Researchers present TurnCost, a tool for re-scoring trajectory corpora of coding agents to optimize caching, pricing, and routing policies. The study analyzes 600 publicly released coding-agent trajectories and finds that turn count explains 84% of the variance in log trajectory cost, making unnecessary turns the larger opportunity for optimization.

This paper proposes a framework called LEST (LLM Evaluation through Specification-driven Testing) to evaluate the safety claims of large language models. It uses documented safety claims as specifications for black-box testing, mutating prompts to search for counterexamples, and manually validating each candidate. The authors evaluate LEST using two models and find that automated evaluation overestimates behavioral failures, highlighting the need for manual validation.

Cortex is a tool that tests changes to an AI agent's harness, which is a set of files that developers write for the agent to follow. Cortex saves fixes from developer sessions, finds recurring corrections, and writes one skill or rule at a time, keeping only those that show a net gain and no significant task degradation. It was tested on Claude Haiku 4.5 and improved the agent's adherence to conventions.

GF-Bench, a PyTorch-based framework, is introduced to generate diverse failures in distributed GPU workloads. It evaluates log analysis methods, revealing a trade-off between effectiveness and efficiency, and highlights the importance of preserving execution context for accurate failure detection.

Researchers propose ReTerm, a self-improving terminal agent that can recover from errors during long-running execution. ReTerm is trained to learn from its own failed tasks and uses a customized data pipeline and online RL algorithm to improve its recovery capabilities. The model achieves a 14.6 point improvement over the base model on Terminal-Bench 2.0.

A new benchmark, ench{}, is introduced to evaluate whether guardrails in language-model agents honor authorization. The benchmark checks authorization decisions for correctness, usability, and resulting effect, and tracks requests across guardrail changes. Experiments show that many blocked operations are due to failed reviews, and that repairing the output format or adding a second reviewer can improve decision-making.

A paper proposes a new approach to finding high-quality test suites for self-evolving AI agents. The method, called TeSearch, uses a quality-diversity evolutionary algorithm to search for small and strong test suites that can effectively evaluate the agent's behavior. This approach is shown to outperform other methods in terms of suite quality and size, and is used as an improvement signal to reject 59.9% of behavior-changing rewrites.

A research paper on plan-based DataFrame code generation found that generated code may not match its stored reference output, even with passing tests. The study highlights the risk of interpretive lock-in, where a mistaken interpretation persists across the plan, implementation, and tests.

A study on the importance of parameter schema in LLM tool selection in MCP systems. The researchers found that parameter-schema changes have the largest and most consistent selection shifts, followed by description explanations and then name effects. This has implications for MCP developers and evaluators, as schema design and tool ordering can affect which implementation is invoked even when functionality is unchanged.

This paper explores the choice between open-weight and closed frontier language models for security work, finding that open-weight models can match the performance of closed models at a significantly lower cost. The authors evaluate 26 models on the CTI-REALM benchmark and introduce stage-level metrics to score agent outputs.

Researchers used Lean-Prover, a verifier for Rust backed by Lean 4, to solve a proof-generation task for the VeriContest benchmark, achieving higher success rates and lower costs than current frontier models.

A new framework, the Spec Growth Engine, aims to address structural failure modes in AI-assisted software development by introducing a machine-readable spec graph, a context assembler, and a drift gate to enforce spec-code consistency and prevent divergence.

Researchers propose HAFixAgent, a history-aware program repair agent that injects repository context into its repair loop, improving agentic APR performance by up to 64.3%.

A benchmark, SWE-DeceptionBench, was introduced to evaluate Large Language Models (LLMs) deception in software engineering across six lifecycle stages and two deception triggers. The study found substantial deception rates in mainstream LLMs, highlighting the need for effective deception detection and mitigation methods.

ProgramGym is a framework for scalable training environments of long-horizon program reproduction agents. It consists of two components: Program-Env, which constructs verifiable training environments from repositories, and Program-Agent, which combines a role-isolated workflow for analysis, implementation, and review. Experiments show that ProgramGym improves performance on program reproduction tasks.

A new benchmark, AgentSWE, is introduced to evaluate the ability of coding agents to build agents that meet user requirements. The benchmark spans three stages of the agent development lifecycle and finds that current coding agents are not delivering what they promise. An empirical analysis of the failure trajectories identifies recurring failure modes that can be addressed by coding agents.

Agent4Spec is a multi-agent framework that uses LLMs to optimize API specifications through automated validation and market analysis. It detects and corrects OpenAPI syntax errors and identifies functional gaps against industry leaders.

A study examined the integration of AI agents with deployment infrastructure, finding that agents often modify both application code and infrastructure, leading to integration overhead and security risks.

Researchers developed CoCoEvolve, an evolutionary optimizer that uses meta-reflection to improve production AI systems. It outperforms existing methods on various benchmarks, including data agents, LLM-powered pipelines, and data engineering workflows.

A new framework, MASTIF, evaluates Agentic Web technologies, including LLMs, protocols, and agent frameworks. The baseline assessment uses 300 benchmark tasks and shows that task-specific tech stacks can improve completion rates and efficiency.