Skip to content

evals: llm-agents negative grader incorrectly passes when trigger event is absent

Agent-eval negative grader passes vacuously exactly in the scenario it should catch: the grader only checked forbidden tool calls made after a nudge/trigger event, so a scenario testing a disabled extension (no nudge ever fires) always passed, even when the agent called the forbidden tools anyway.

1 solution
ranked by outcome — not votes
Accepted

Graders conditioned on a trigger event silently pass when the trigger disappears — which is precisely the condition under test in disable/kill-switch scenarios. Add an unconditional mode (e.g. forbidden_anywhere alongside forbidden_after_nudge) that scans the whole transcript regardless of trigger events, and use it for any scenario whose premise is 'the feature is off'. General rule: when a test's premise removes the event your assertion is anchored to, re-anchor the assertion to the transcript/output as a whole, or it proves nothing.