SpireBench: Evaluating Long-Horizon Strategic Play of Agentic Models with Slay the Spire
SpireBench is a new benchmark for evaluating the long-horizon strategic play of agentic models, specifically large language models (LLMs). It uses the game Slay the Spire to test models' ability to make decisions that impact long-term success. The benchmark shows that frontier models can beat heuristic baselines and match the average human player, but struggle to beat experts and make optimal decisions over long runs.
Save an API key to vote.