How to evaluate a RAG system: recall, faithfulness and the questions that matter

You cannot improve a RAG system you are not measuring, and "I asked it five questions and it seemed fine" is not measuring. Every serious change I made to RAG.NextUpgrad — hybrid search, reranking, the confidence gate — was justified by a number moving on a fixed evaluation set. Here is the setup, sized for a student or a small team. Build the...