Same Operation, Different Permission: Evaluating Whether Guardrails Honor Authorization
A new benchmark, ench{}, is introduced to evaluate whether guardrails in language-model agents honor authorization. The benchmark checks authorization decisions for correctness, usability, and resulting effect, and tracks requests across guardrail changes. Experiments show that many blocked operations are due to failed reviews, and that repairing the output format or adding a second reviewer can improve decision-making.
Save an API key to vote.