A study compares motion planning methods for autonomous driving on the CARLA Leaderboard, identifying strengths, weaknesses, and trends. This research is relevant to AI agents as it contributes to the development of autonomous driving capabilities.

A new evaluation paradigm, the Multilayered Evaluation Framework, is introduced to assess the correctness of Text-to-SQL systems with AI operators. Traditional metrics fail to accurately evaluate these systems, leading to a 25% detection rate for correct translations. The proposed framework decouples deterministic database logic from flexible AI semantics, achieving state-of-the-art accuracy across industry and academic systems.

GameASG-Bench is a benchmark for evaluating the behavioral testability of autonomous software generation in game development. It introduces an evaluation interface specification and includes static and dynamic checks to assess compliance and performance. The benchmark is implemented as 47 browser-native game-generation tasks, and experiments demonstrate compliance gaps between average check pass rates and strict task success rates.

This study explores efficient benchmarking methods for production LLM agents, including random sampling, historical caching, fixed representative subsets, and IRT-based adaptive testing. The authors report that multidimensional 2PL adaptive testing achieves the best score fidelity, but also highlight the operational simplicity of difficulty-stratified fixed subsets. The study provides practical recommendations for recurring production-agent evaluation.

WorldRoamBench is a new open-world benchmark for evaluating the long-horizon stability of interactive world models, addressing limitations in existing benchmarks. It assesses models across four dimensions: action, vision, physics, and memory, and reveals that none of the tested models reliably satisfy all dimensions.

A new benchmark, SCICONVBENCH, evaluates LLMs on multi-turn clarification for task formulation in computational science. It assesses their ability to elicit missing information and correct internally contradictory requests. Current frontier models show room for improvement.

A new benchmark, REASONS, is introduced to evaluate the reliability of scientific citation attribution in large language models (LLMs). The benchmark assesses LLMs' ability to correctly attribute citations and abstain from providing incorrect information when uncertain. The evaluation framework includes various settings, such as zero-context and retrieval-augmented prompting, to test LLMs' performance. The results show that certain configurations can significantly improve citation attribution accuracy, but also highlight the challenges of balancing reliability and responsiveness.

LM Fight Arena is a novel framework for evaluating large multimodal models (LMMs) by pitting them against each other in a game competition. This benchmark assesses an LMM's strategic reasoning capabilities in a dynamic setting, providing a fully automated, reproducible, and objective evaluation.

A new approach called Counterfactual Harness Search and Evolution (CHASE) is introduced to prevent cheating in agent evaluation by addressing automatic harness optimization. CHASE generates counterfactuals that preserve task semantics while searching for protocol transformations that reduce gain destruction.

Researchers proposed a framework to generate context-specific large language model (LLM) benchmark datasets by combining expert input with synthetic data generation. This approach aims to balance validity and scalability, addressing the limitations of existing benchmark construction methods. The framework uses a schema to elicit key information about evaluation tasks and guides synthetic data generation with four measurement validity criteria: coverage, diversity, content realism, and stylistic realism.