How Do You Actually Test an AI System? A Layered Strategy From Five Tools I Built

Every engineer who has shipped an LLM feature eventually hits the same wall. Your unit tests are green. Nothing throws. And yet the thing is quietly, obviously worse than it was last week. Someone swapped a model, someone tweaked a prompt, someone added a tool, and the output did not error, it just got dumber. No stack trace ever tells you that. I...