Without evaluation, you cannot tell improvement from gaming

An agent calls tools, changes an environment and completes work across steps. Evaluating the final prose alone can never be enough.

Define a verifiable task first; keep failure trajectories first; optimize only then.

01 / ENVIRONMENTBuild a resettable environment first

Tool-using tasks must simulate real interfaces and side effects; interactive tasks also need intermediate state. Every run starts from a controlled point, exposes what happened, and can be scored against the final environment rather than the agent’s own summary. If a test cannot reset, observe and judge the outcome, it cannot distinguish repeatable ability from a lucky run.

02 / TASK SETMake the task set precise and representative

A dataset needs common paths as well as boundary and failure cases. Each task needs an objective, verifiable result, while ambiguous, leaked or stale samples must be removed. Stratify simple, compositional and long-horizon work; cover real tools, data scale and permission conditions; prevent answers entering prompts or training data; and version the task set so comparisons remain honest.

03 / METRICSMeasure cost and process as well as success

Record task success, steps, tool errors, latency, tokens and monetary cost together. An LLM-as-a-Judge can help with subjective quality, but its rubric, position bias and model version must be controlled. Paired comparisons are often steadier than absolute scores. With enough samples, report confidence intervals and statistical significance instead of declaring a tiny difference a win.

04 / OBSERVE & IMPROVEUse trajectories to explain and improve

A total score says whether something moved; the trajectory explains why. Observability should retain model inputs, tool calls, errors, timing and key state. Then ablations and A/B tests identify which change truly caused the result. First exercise: give a case-organizing agent 20 verifiable tasks, and record completeness, citation accuracy, duration and human correction count. Keep the failed runs: they are the map for the next iteration.

Sources

bojieli/ai-agent-book.
Chapter 6 — Evaluation and observability.
This is an applied reorganization, not a replacement for the source book; use the source text and code for exact claims.

Next: when post-training is actually needed.

Continue to post-training

Without evaluation, you cannot tell improvement from gaming.