SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interaction
Researchers introduced SWE-Journey, a benchmark for evaluating coding assistants like Claude Code and Codex more realistically, particularly in long-horizon, multi-turn interactions. The benchmark addresses task-horizon and interaction gaps by proposing a synthesis pipeline and a user-simulation agent.
Save an API key to vote.