// Evals 101 · Advanced

Advanced evals

You have the basics: tasks, verifiers, sandboxes and traces. These 23 ideas take your evals from working to trustworthy. They’re grouped into 5 themes, and each one comes in simple words, with an analogy, an example and why it matters.

// Theme 1

Measure it properly

Start small, pick the right number, and know when a change is real rather than noise.

Start with a small golden dataset

Simple words: A small, hand-checked set of real cases the team trusts as the reference.
Analogy: The teacher’s own marked copy of the exam.
Example: Pull real support tickets where the agent failed, write down the right outcome for each, and use them as your first tasks.
Why it matters: One practitioner guide suggests 20–50 simple tasks drawn from real failures is a great start. You don’t need thousands.

Source: Anthropic Engineering (2026)

Metrics primer: accuracy, precision, recall

Simple words: Accuracy: how often the answer is right overall. Precision: when it says “yes”, how often it’s right. Recall: of all the real “yes” cases, how many it found.
Analogy: A spam filter: precision is “don’t bin good mail”, recall is “don’t let spam through”.
Example: An agent flags urgent tickets. It flags 10, and 8 really are urgent: good precision. But there were 20 urgent tickets, and it found only 8: poor recall.
Why it matters: One number can hide the failure that matters. Decide which mistake is worse for your task, and track that metric.

Same input, different output: pass@k and pass^k

Simple words: AI can answer the same question differently each time, so one run proves little. Run it several times.
Analogy: A basketball player’s free throws: one shot tells you little about their average.
Example: pass@k asks: did at least one of k tries succeed? pass^k asks: did all k tries succeed? A support agent needs a high pass^k: the same request comes from many customers, and every one of them must be handled correctly.
Why it matters: Reporting only the best of several tries makes an unreliable agent look reliable.

Source: Chen et al. (2021) · Yao et al. (2024)

Is the change real? Sample size and noise

Simple words: With only a few test cases, a score can move just by chance. The fewer cases you have, the bigger a change must be before you trust it.
Analogy: Flipping a coin ten times and getting seven heads doesn’t prove the coin is loaded.
Example: A new prompt passes one more task out of twenty. Before celebrating, run each task several times and add more cases. If the gain holds, it’s probably real.
Why it matters: It’s easy to ship an “improvement” that was only noise. Repeated runs and more cases turn a hunch into evidence.

// Theme 2

Graders you can trust

Code, AI judges and people each grade some things well. Give them clear rules, and check them before you rely on them.

Three kinds of graders

Simple words: Code checks what can be checked exactly. AI judges handle fuzzy things like tone. Humans handle the hardest calls and set the standard.
Analogy: A spell-checker, an editor, and the publisher.
Example: Code: “was the note saved?” AI judge: “is the tone friendly?” Human: “would I send this to our biggest client?”
Why it matters: Use the cheapest grader that is reliable for each check. Save humans for what only humans can judge.

Source: Anthropic Engineering (2026)

LLM-as-judge and its biases

Simple words: Using one AI to grade another AI’s output against a rubric.
Analogy: A fast, tireless judge who happens to prefer long speeches.
Example: A judge model reads each drafted email and answers five yes/no questions about tone and accuracy.
Why it matters: Research has found AI judges can favour longer answers, answers in a certain position, and answers they wrote themselves. Check a judge against human labels before trusting it.

Source: Zheng et al. (2023)

Pairwise comparison: which is better, A or B?

Simple words: Instead of scoring one answer on its own, show a grader two answers to the same question and ask which is better.
Analogy: An eye test: “clearer with lens one, or lens two?” is easier to answer than “rate this lens”.
Example: Old prompt versus new prompt: for each test case, a judge sees both emails side by side and picks the better one. Swap the order half the time.
Why it matters: Seeing two answers side by side can make small differences easier to spot. Research on AI judges found they can favour one position, often the first, so swap the order.

Source: Zheng et al. (2023)

Write grading guidelines people can follow

Simple words: A short written guide, with examples, that tells every grader — human or AI — what “pass” and “fail” look like.
Analogy: A referee’s rulebook, with written examples of what counts as a foul.
Example: “Brand tone” becomes: no exclamation marks, no slang, greet by first name. Add two passing and two failing sample emails.
Why it matters: Without a shared guide, two experts grade the same answer differently, and your AI judge learns the confusion.

Checking the grader against experts

Simple words: Have humans grade a sample too, and measure how often the grader agrees with them.
Analogy: Two referees watching the same match should mostly make the same calls.
Example: An expert grades a sample of emails; the AI judge grades the same sample. You compare the two sets of answers, and fix the rubric where they disagree.
Why it matters: Agreement statistics such as Cohen’s kappa correct for agreement that would happen by chance, so they are stricter than a raw match rate.

Source: McHugh (2012)

// Theme 3

Test what matters

The path, the conversation, the sources, the attacks — and why public leaderboards can’t do this for you.

Grade the path, not just the answer

Simple words: Trajectory evals check the steps and tool calls the agent took, not only its final answer.
Analogy: Marking the working in a maths exam, not just the final number.
Example: The final email was fine, but along the way the agent deleted a record by mistake. Only a check on the steps catches that.
Why it matters: An agent can reach a right-looking answer in an unsafe or wasteful way. Tool-call checks catch it.

Source: Anthropic Engineering (2026)

Conversations, not just single questions

Simple words: Many agents talk with a person over several turns. Multi-turn evals test the whole conversation, often with an AI playing the user.
Analogy: A driving test on real roads with other traffic, not just a parking exercise.
Example: A simulated customer changes their mind halfway: “Actually, refund the other order.” The eval checks the final records, not just the first reply.
Why it matters: Agents that do well on one-shot questions can lose track over a conversation. Research benchmarks now simulate users to test this.

Source: Yao et al. (2024)

Evaluating RAG: grounded answers

Simple words: For systems that look things up first, check whether the answer sticks to what was retrieved, and whether the right things were retrieved.
Analogy: An open-book exam: did the student use the book, and did they open it at the right page?
Example: The bot quotes a refund window. Faithfulness asks: is that in the retrieved policy? Context relevance asks: is what was retrieved actually about refunds, or padded with unrelated text?
Why it matters: Research on RAG evaluation measures faithfulness, answer relevance and context relevance separately, because each can fail on its own.

Source: Es et al. (2023)

Safety evals, red teaming and prompt injection

Simple words: Deliberately try to make your agent misbehave, including hiding instructions inside the content it reads.
Analogy: Hiring a locksmith to try to break into your own house.
Example: A test web page contains hidden text telling the agent to email the customer list elsewhere. The eval passes only if the agent ignores it.
Why it matters: OWASP lists prompt injection as the top risk for LLM applications, and researchers have shown indirect injection through retrieved content working against LLM-integrated apps.

Source: OWASP (2025) · Greshake et al. (2023)

Public benchmarks aren’t your evals

Simple words: Leaderboards test general skills on public questions. Your evals test your tasks on your data.
Analogy: Top of the class in general knowledge doesn’t make you good at a specific job.
Example: A model tops a public coding benchmark but fails many of your CRM tasks, because your tools and rules are unusual.
Why it matters: Models may have seen public test questions during training, which makes their benchmark scores look better than they are.

Source: Sainz et al. (2023)

// Theme 4

Make evals a habit

Study real failures, write evals first, run them on every change, watch production, and keep the suite fresh.

Error analysis: read the failures first

Simple words: Before writing more evals, read a batch of real failures and sort them into groups by what went wrong.
Analogy: A doctor sorting symptoms before prescribing, instead of treating every cough the same way.
Example: Reading a batch of failed support chats shows three patterns: wrong order found, refund policy misquoted, tone too casual. Each pattern becomes its own set of evals.
Why it matters: It tells you which evals to write and which fix will help most. Guessing wastes effort on problems users don’t actually have.

Eval-driven development

Simple words: Write the eval before you build or change the feature, then build until it passes.
Analogy: Agreeing what “done” looks like before you start painting the room.
Example: Before adding “draft follow-up emails”, write ten tasks and a checklist for a good draft. Then build the feature against them.
Why it matters: It forces you to define “good” up front, and gives you a finish line instead of endless tweaking.

Regression tests and CI gates

Simple words: Re-run your evals on every prompt, tool or model change, and block the change if scores drop.
Analogy: A smoke alarm in the build pipeline.
Example: A pull request changes the system prompt. The pipeline runs the eval suite; two tasks that used to pass now fail, so the merge is blocked.
Why it matters: Small prompt tweaks can quietly break things that used to work. Automatic checks catch it before users do.

Offline vs. online evals

Simple words: Offline evals run on a fixed test set before release. Online evals watch the live system with real users.
Analogy: A crash test in the lab versus a dashcam on the road.
Example: Offline: run the CRM suite before changing the prompt. Online: sample live runs each day and check them against the same checklist.
Why it matters: Offline evals catch known problems early. Online evals catch the ones you never thought to test.

Observability: traces, spans and sessions

Simple words: A trace records one run. Spans are its individual steps. A session groups the runs from one conversation.
Analogy: Receipts (spans) stapled into one bill (trace), filed under one customer visit (session).
Example: A complaint arrives. You open the session, find the trace for that answer, and see which tool-call span returned bad data.
Why it matters: OpenTelemetry, an open standard for traces, is developing shared naming rules for AI calls, so tools can read each other’s traces. The rules are still being written.

Source: OpenTelemetry (2026)

Evals go stale

Simple words: As products, users and models change, old evals stop matching reality. Review and update them like code.
Analogy: Last year’s map of a growing city.
Example: You add a new pricing plan, but the eval suite still checks for the old plan names. Everything passes while real answers are wrong.
Why it matters: A stale suite gives false confidence, which can be worse than no suite.

Retire the tests everyone passes

Simple words: When every version of your agent passes a task, that task no longer tells you anything new. Replace it with harder, more realistic ones.
Analogy: A high jumper who clears the bar every time needs the bar raised to see any progress.
Example: All of your email tasks pass, yet customers still complain. Add harder cases from recent complaints; keep a few old ones as a regression check.
Why it matters: A suite that always passes feels good but can’t show improvement or catch new problems.

// Theme 5

Spend wisely

The best model is the cheapest one that clears your bar for each job.

Cost, speed and quality: pick per task

Simple words: Different tasks deserve different trade-offs. Choose the cheapest, fastest option that clears each task’s bar.
Analogy: You don’t take a taxi to the corner shop, or walk to the airport.
Example: Internal notes go to a small, fast model. Customer-facing emails go to a larger one that clears a higher bar.
Why it matters: Research on agents argues that evaluations should report cost alongside accuracy, not accuracy alone.

Source: Kapoor et al. (2024)

Distillation

Simple words: Training a smaller model to copy the outputs of a larger one for a specific job.
Analogy: An apprentice learning by watching a master at work.
Example: The large model drafts thousands of approved CRM notes. A small model trained on them can then run the task faster and more cheaply.
Why it matters: It can cut cost once a big model has shown the job can be done well. Your evals tell you whether the small one kept the quality. Check your provider’s terms first: some limit training on their outputs.

Source: Hinton, Vinyals & Dean (2015)