Grep your tool surface before you write another refusal eval

Every eval I wrote last year assumed the agent would try to do the wrong thing. A model reaching for another tenant's records. A model running the destructive migration. So I wrote refusal evals: hand it the forbidden request, check that it declines. Pass rate looked great. I felt good about it for about a week. Then I read a paper on... Weiterlesen: Grep your tool surface before you write another refusal eval