I built an eval harness to prove an LLM worked. It proved the opposite.

«I set out to have a language model classify integration failures. I built an evaluation harness to prove it worked. The harness proved it wasn't worth using. Final architecture: deterministic rules do the classification...»