Evals: how you actually know your AI product works

Traditional software fails loudly — a null pointer, a stack trace, a failing test. An LLM application fails quietly: it returns a fluent, confident, plausible-sounding answer that happens to be wrong, unhelpful, or subtly off-topic. Nothing crashes. Nothing turns red. Without a deliberate way to measure quality, you're shipping changes — a new prompt, a new retrieval strategy, a model upgrade — with no idea whether they made things better or worse. Evals are the test suite for that gap.

What makes evals different from unit tests

A unit test asserts output === expected. Language is rarely that rigid — two correct answers to the same question can be worded completely differently. Evals instead score outputs along dimensions that tolerate that variation: is the answer faithful to the source material, relevant to the question, and correct, even if it's not a word-for-word match to some reference?

The core metrics for a RAG or LLM app

• Faithfulness / groundedness — does every claim in the answer trace back to the retrieved context, or did the model add something it wasn't given? This is the direct measure of hallucination.
• Answer relevance — does the answer actually address the question asked, independent of whether it's factually correct?
• Context precision / recall — of the chunks retrieved, how many were actually relevant (precision), and of the chunks that should have been retrieved, how many were found (recall)? This isolates retrieval quality from generation quality.
• Correctness — for questions with a known-good answer, does the response match it in substance?

Reference-based vs. reference-free

Some metrics need a golden answer to compare against — correctness, for instance, usually does. Others are reference-free: faithfulness only needs the context and the answer, not a pre-written correct response, which makes it far cheaper to run continuously in production, on real traffic, without needing to hand-label everything in advance.

LLM-as-judge

The most common way to score subjective qualities like faithfulness or relevance at scale is to use another LLM as the grader — give it the question, the context, the answer, and a precise rubric, and ask it to return a structured score.

JUDGE_PROMPT = """You are grading an AI answer against retrieved context.

Context: {context}
Question: {question}
Answer:   {answer}

Score "faithfulness" from 1-5: does every claim in the answer
appear in or follow from the context? Respond as JSON:
{{"score": <int>, "reason": "<one sentence>"}}"""

def score_faithfulness(question, context, answer):
    result = judge_llm.generate(JUDGE_PROMPT.format(**locals()))
    return json.loads(result)

Building an eval harness

In practice, evals become useful once they're a fixed part of your workflow rather than a one-off check:

1. Build a test set — real questions your users actually ask, including edge cases and known-hard queries, not just easy examples.
2. Run it on every change — a new prompt, a different chunk size, a model swap. Treat it like CI: no change ships without the eval suite running against it.
3. Track scores over time — a single run tells you almost nothing; the trend across changes is what tells you if you're actually improving.
4. Mix automated and human review — automated evals scale, but a small slice of human review keeps the automated judge honest.

Evals are what close the loop on everything else in this series: they're how you'd know whether a change to your RAG pipeline — a different chunking strategy, a new embedding model — actually helped.