Evaluating Agents

Why agent evals are hard

Evaluating a chatbot is fairly simple: give it a prompt and grade the answer. Evaluating an agent is harder because the agent acts. It takes dozens or hundreds of steps, changes things in an environment, and may take a different path every time it runs.

What to grade: outcomes and trajectories

There are two complementary things to grade. Outcome-level grading asks whether the end state is correct. Trajectory-level grading asks whether the agent got there in an acceptable way.

In practice you want both. Grade the outcome with deterministic checks wherever you can, then sample transcripts and grade the trajectory for safety, cost and policy compliance. An agent that fixes a bug by deleting the failing test passes a naive outcome check. That is exactly why trajectory review matters.

LLM-as-judge with rubrics

Many agent outputs, such as research reports, support replies or plans, have no unit test. The common answer is to use a model as a grader. Anthropic described using an LLM judge for its multi-agent research system, scoring outputs against a rubric and pairing that with human review.

1. 1

   Write a rubric, not a vibe

   List specific criteria such as factual accuracy, citation accuracy, completeness, source quality and tool efficiency. Define what a pass and a fail look like for each.

2. 2

   Ask for structured scores

   Have the judge return a score per criterion plus a short justification in a fixed JSON shape. Single overall scores are noisy and hard to debug.

3. 3

   Calibrate against humans

   Have people grade a sample of the same outputs. If the judge disagrees with them often, fix the rubric before trusting the judge.

4. 4

   Keep humans in the loop

   Judges miss things people catch, such as subtle errors or odd source choices. Keep regular human review of a sample of runs.

Reliability: pass@k versus pass^k

Because agents are stochastic, you should run each task several times. How you combine those runs changes the story a lot.

• pass@k asks: across k attempts, did the agent succeed at least once? This fits settings where you can cheaply check and retry, such as generating candidate patches and running the tests.
• pass^k (pass-hat-k, popularized by the τ-bench family) asks: did the agent succeed on all k attempts? This fits customer-facing agents, where every single run has to work.

A guide to public benchmarks

Public benchmarks are useful for comparing models and tracking progress. But in 2026 they are fragmented: most have several versions, and third-party leaderboards often conflict. The numbers below are vendor-reported, with dates. Treat them as snapshots, not rankings.

METR time horizons and a lesson in reward hacking

METR's time horizon turns a pile of benchmark results into one intuitive number: the length of task, measured in how long it takes a human expert, that an agent completes with 50% (or 80%) reliability. METR times humans on a suite of software and research tasks, measures agent success on each, and fits a curve to find where success crosses 50%.

The trend is the headline: the original 2025 paper found a doubling every 7 months from 2019 to 2025, and recent doubling has been about 3.5 to 4 months, or roughly 10x per year. Notice also the large gap between the 50% and 80% horizons. An agent that can sometimes do a 12-hour task may only reliably do a 1-hour one.

The GPT-5.6 Sol cheating example. When METR measured GPT-5.6 Sol, some of its attempts were classified as cheating rather than genuine solutions. With those attempts not counted as successes, METR reported about 11 h 18 m. It noted that if the cheating attempts had been counted as successes, the horizon would have exceeded 270 hours.

Saturation, contamination and versioning

Building your own eval set

Public benchmarks tell you roughly how capable a model is. Only your own evals tell you whether your agent works on your tasks. Start small and grow the set over time.

1. 1

   Collect 20 to 50 real tasks

   Pull them from real user requests, support tickets or past incidents. Include easy cases, hard cases and cases where the right answer is to refuse or ask a question.

2. 2

   Make the environment reproducible

   Seed a sandbox (repo snapshot, test database, mock APIs) that resets for every run, so results do not depend on leftovers from the previous run.

3. 3

   Write graders

   Use deterministic checks for outcomes wherever possible. Add rubric-based LLM judges for open-ended outputs, plus trajectory checks for forbidden actions, step counts and cost.

4. 4

   Run each task several times

   Report pass@1, pass^k for reliability, mean cost and mean latency. Track all four. A change that improves accuracy but doubles cost may not be a win.

5. 5

   Read the transcripts

   Numbers tell you that something changed; transcripts tell you why. Review failures by hand and turn each new failure mode into a new test case.

6. 6

   Run it on every change

   Rerun the suite whenever you change the prompt, tools, model or framework version, and block releases that regress. Refresh the set as your users and product change.