Bad Genius: Counterfactual-Guided Harness Evolution Beyond Task-Specific Shortcuts
A new approach called Counterfactual Harness Search and Evolution (CHASE) is introduced to prevent cheating in agent evaluation by addressing automatic harness optimization. CHASE generates counterfactuals that preserve task semantics while searching for protocol transformations that reduce gain destruction.
Save an API key to vote.