Chapter 7 — Evaluation in Practice: Tooling, Patterns, and Playbooks

TL;DR

Operationalizing evaluation means turning Chapter 6's metrics, rubrics, and datasets into configurable tooling with SME-friendly workspaces, versioned artifacts, and gate policies that govern every release.
Configure your evaluation platform along three axes — data adapters that ingest rich agent traces, a rubric engine that encodes SME-designed questions and scales, and business-impact thresholds that separate non-negotiable hard gates from soft trade-offs.
Wire evaluation into every lifecycle stage: offline regression harnesses on every PR, red-team and simulation suites pre-release, shadow mode and canary rollouts in production, and continuous monitoring with drift detection and automated rollback triggers.
Apply repeatable playbooks per agent archetype — chat copilots, workflow agents, high-risk domain agents, and multi-agent systems — each with tailored metrics, evaluators, datasets, and gate profiles.
Avoid common traps such as benchmark worship, leaving SMEs out of rubric design, one-time evaluation without CI integration, uncalibrated LLM judges, and relying solely on real logs without synthetic simulation for tail risks.

Chapter 6 defined the foundations of evaluation. You learned how to turn business goals into metrics with subject matter experts (SMEs). You explored the taxonomy of metrics for agentic systems: task success, quality, safety, latency, cost, and UX. You saw different kinds of evaluators—rules, LLM judges, human SMEs, and critic agents. You also learned how to build evaluation datasets, benchmarks, and synthetic simulations.

This chapter is about making all of that operational.

You are no longer asking "what should we measure?" You are asking:

How do we configure tools to reflect our domain-specific metrics and rubrics?
How do we wire evaluation into CI/CD and production?
How do we run repeatable patterns across different agent use-cases?
How do we avoid the common traps where evaluation looks good on paper but fails in reality?

We will keep using the HR Leave Agent as our anchor. We will contrast it with other archetypes: a chat copilot, a high-risk domain agent, and multi-agent systems.

📚 In this chapter: Section 7.5 covers user research methods that complement automated evaluation—essential for understanding whether users actually trust and adopt your agent.

7.0 From Foundations to Practice

At this point in the handbook, you have an agent design (Chapters 4 and 5) expressed as scaffolds and patterns. You have a harness (Chapter 5) that can run tests, generate spans and traces, and emit telemetry. You have evaluation foundations (Chapter 6) that define what "good" means through metrics, how to judge it through evaluators, and what data you use through datasets, benchmarks, and simulations.

What you don't yet have is a practical setup that:

Lets SMEs interact with evaluation in their own language
Can be run automatically on every change
Can monitor live traffic and catch regressions early
Has clear playbooks for different types of agents

That is the focus of this chapter:

How to build evaluation as a first-class, configurable, and repeatable capability across your agentic portfolio.

We start with evaluation tooling, move to lifecycle patterns, then wrap up with use-case playbooks and a practical checklist.

Figure 7-1 shows the evaluation-driven development loop that ties these pieces together. Every fix creates new eval cases. Every production failure feeds back into the regression suite.

Figure 7-1. The evaluation-driven development loop

flowchart LR
  A[Ship a change<br/>prompt, model, tools, retriever] --> B[Run CI evals<br/>golden + synthetic]
  B -->|fails| C[Inspect failures<br/>trace review]
  C --> D[Error taxonomy + root cause]
  D --> E[Fix or mitigate<br/>prompt, tools, policy, data]
  E --> F[Update eval set<br/>new golden cases]
  F --> B
  B -->|passes| G[Deploy behind shadow or canary]
  G --> H[Monitor production<br/>sample traces]
  H --> I[Discover new failures]
  I --> F

7.1 Customising Evaluation Tools to Your Use-Case

Most evaluation tools—whether home-grown or off-the-shelf—come with some defaults. They typically have generic fields like "prompt", "response", and "label". They include a handful of built-in metrics such as similarity scores and simple LLM-as-judge prompts. They offer a generic dashboard.

For agentic systems, that is rarely enough. The data you care about, the labels you need, and the rubrics your SMEs design are all very specific to your use-case and vertical.

Think of your evaluation tooling not as a static product, but as a configurable platform that you shape along three axes:

How it loads and represents data
How it implements rubrics defined by SMEs
How it encodes business impact (non-negotiables vs tolerances)

7.1.1 Data Adapters & Label Schemas

Agents generate much richer artefacts than "prompt → answer". For example, the HR Leave Agent produces employee inputs across multiple turns. It generates a sequence of tool calls to HRIS/SAP queries, policy retrievals, and OCR. It creates intermediate reasoning steps, plans, and critic feedback (if you expose them). It takes final actions: leave requests created, approvals routed, notifications sent. It also captures environment state such as employee attributes, balances, and policy versions.

Your evaluation tool needs to see the pieces that matter. That usually means building data adapters that can ingest chat logs and traces from your runtime, parse spans into meaningful units (LLM calls, tool invocations, decisions), and attach contextual metadata such as employee type, country, and business unit.

You also need to define a label schema that matches your metrics:

Task-level labels like decision_correct, policy_violation, requires_human_review
Step-level labels like retrieval_relevant, tool_call_valid
UX labels like session_satisfaction or tone_appropriateness

In other words, you don't want to contort your evaluation into generic fields. You want a tool where you can say:

"For the HR Leave Agent, each test case has: employee_profile, policy_version, conversation_trace, final_decision, system_state_diff, and labels {decision_correct, policy_violation, explanation_score, tone_score, …}."

Once this schema is defined, everything else—visualisations, gates, experiment comparisons—becomes more meaningful.

📋 Production Rubrics: Appendix C, §C.3 provides four complete LLM judge rubrics: Eligibility Judge (decision correctness with non-negotiable thresholds), Safety Judge (PII protection, authorization, fabrication checks), Citation Judge (coverage, accuracy, and relevance scoring), and Quality Judge (directness, completeness, clarity, actionability). Section C.4 shows how to compose these into a weighted evaluation pipeline.

7.1.2 Implementing SME Rubrics in the Tooling (Rubric Engine)

Chapter 6, §6.3 covered how SMEs design rubrics—questions, scales, and judge types. Here the focus is on making those rubrics first-class configuration in your tooling, not something buried in prompts or spreadsheets.

Build a rubric engine with three capabilities:

Rubric editor: SMEs define questions, choose response scales, and mark questions as mandatory or conditional (e.g., only for medical leave cases).
Multi-backend judging: Plug in human annotators, LLM-as-judge, or rule-based checks per question—the platform orchestrates who answers and stores scores in your label schema.
Context presentation: The tooling presents annotators or judges with the right context, output, and rubric questions automatically.

SMEs define what should be asked and how it's scored. The platform orchestrates who answers (human, LLM, rule) and how those answers are stored and used.

7.1.3 Encoding Business Impact: Non-Negotiables vs Tolerances

Chapter 6 introduced the distinction between hard constraints and soft trade-offs. In your tooling, make this distinction configurable per metric and per environment:

Non-negotiable (hard gates): legal_violation = 0, pii_leakage = 0, policy_violation = 0 for union-derived rules. Any violation blocks promotion.
Soft metrics with bands: green (acceptable), amber (watch), red (unacceptable). For example, explanation_score ≥ 1.5/2 on average, P95 time_to_decision below threshold.

Configure thresholds per environment (dev, staging, production) so gates govern promotion, auto-approval enablement, and rollback decisions. This gives you a decision vocabulary: "tolerate a small latency increase if safety metrics are flat" or "any non-zero legal violation fails the gate regardless."

7.1.4 Example: HR Leave Evaluation Workspace

To make this concrete, imagine the eval UI for the HR Leave Agent.

When an HR SME logs in, they see a scenario list filtered to real cases from logs, synthetic simulation runs, and red-team cases. For each scenario, they can view the employee profile (anonymised or synthetic), the policy version applied, the conversation transcript, the final decision and system state, and the judge scores including rule-based checks (pass/fail), LLM-judge scores for explanation and tone, and human SME scores on selected items.

They also see a scorecard summary showing decision correctness, policy/legal violations, explanation quality, tone, time-to-decision, and cost. The scorecard indicates which metrics are hard vs soft and whether they are in green/amber/red bands.

A diff view lets them compare v1 vs v2 of the agent (after a scaffold or model change) on the same benchmark. They can see which scenarios improved or regressed, and on which dimensions.

From this workspace, SMEs can drill into specific failures, comment on borderline decisions, propose rubric tweaks ("we actually care more about this aspect"), and sign off on gates ("this release is acceptable to move to pilot").

That is what it means to have evaluation operational: not just metrics in a notebook, but a place where the people who own the process can see and influence how the agent is judged.

7.1.5 The Minimal Artifact Set (Versioned, Reviewable, Repeatable)

If you want evaluation to behave like engineering, treat it as config and code, not a spreadsheet that lives in someone’s inbox.

A practical pattern is to keep an eval package in the same repo as the agent, versioned and reviewed like any other change:

eval/
  schemas/            # label schemas, JSON Schema or Pydantic models
  metrics/            # metric registry and aggregation rules
  rubrics/            # SME rubrics used by judges (LLM or human)
  gates/              # hard and soft gates per environment
  suites/             # benchmark suites and scenario definitions
  reports/            # generated evidence bundles (CI artifacts)

Example: a compact gate policy you can attach to CI and promotion workflows:

# eval/gates/staging.yaml
environment: staging
baseline: release-2026-01-05

hard_gates:
  - metric: policy_violation_rate
    op: "<="
    value: 0.0
  - metric: tool_misuse_rate
    op: "<="
    value: 0.0

primary_gates:
  - metric: decision_correctness
    op: ">="
    delta_from_baseline: -0.001   # allow up to 0.1% drop
  - metric: p95_time_to_decision_s
    op: "<="
    value: 25

soft_gates:
  - metric: friction_index
    op: "<="
    delta_from_baseline: 0.05
  - metric: avg_cost_usd
    op: "<="
    delta_from_baseline: 0.10

actions:
  on_fail: block_merge
  on_warn: require_sme_review

Minimum viable implementation:

One label schema that matches your metrics (task, step, and safety).
One rubric file that SMEs can read and edit without touching prompts.
One benchmark suite that is small enough to run on every pull request.
One gate policy for staging (hard, primary, soft) tied to business risk.
One evidence bundle produced by CI (results, diffs, versions, and trace links).

7.2 Evaluation Patterns Across the Lifecycle

Evaluation is not a one-off activity. It shows up in:

Development , while you’re designing scaffolds and tools.
Pre-deployment , as part of CI/CD and release checks.
Pilot and ramp-up , when you’re cautious about impact.
Full production , where you monitor and respond to drift.

Table 7-1 summarises the evaluation surfaces you will use across these phases.

Table 7-1. Evaluation surfaces, inputs, and outputs

Evaluation surface	When you use it	Typical input	Typical evaluators	Output artifacts	What it catches well
Unit checks (pre-LLM)	Every PR	Structured inputs, schema samples	Rules, schemas, parsers	Test report, failing cases	Deterministic bugs, malformed tool calls, invalid JSON, policy keyword violations
Offline regression harness	PR gate, nightly	Golden tasks, synthetic tasks, curated traces	LLM judges, rule checks, humans for spot review	Scorecard diff, failure set, evidence bundle	Regressions from prompt/model/tool changes
Journey evaluation (trace-level)	Nightly, pre-release	Full traces, multi-step sessions	Step judges + task judge + trace metrics	Span-tagged critique, root-cause buckets	Tool misuse, planner failures, loops, brittle scaffolds
Red-team harness	Pre-release, quarterly	Adversarial prompts, jailbreak suites, tool-abuse probes	Safety judges, policy checks, human red team	Safety report, exploit repro steps	Prompt injection, data exfiltration, unsafe actions
Shadow mode	Before scale-up	Live traffic mirrored, no user impact	Reference-free judges, anomaly checks	Drift dashboard, replay set	Real-world distribution shift, long-tail failures
Canary / A-B	After release	Live traffic split	Online metrics + sampled judges	Guardrail trigger rates, business KPIs	User impact, UX regressions, subtle quality shifts

Across these phases, certain evaluation patterns recur.

7.2.1 Offline Regression Harness Pattern

This is the closest analogue to “unit and integration tests” for agents.

Pattern:

Maintain one or more benchmark suites (see Ch6):
Standard, edge-case, red-team.
For every change that might affect behaviour:
Prompt updates.
Scaffold changes (e.g., new planning strategy).
Model upgrades or parameter changes.
Tool contract changes.
Run the agent through the benchmarks in an offline harness:
No live users.
Deterministic inputs.
Apply your scorecard and gates:
Hard constraints must be satisfied.
Primary metrics must meet or exceed previous baseline.
Soft metrics can trade off within defined bands.
Only if the gates are satisfied do you:
Merge the change.
Promote to staging or pilot.

For the HR Leave Agent, this might be:

Run 200 golden cases (standard + edge).
Decline to merge if:
Any new policy or legal violation is detected.
Decision correctness drops more than 0.1% overall or for a key subgroup.
P95 time-to-decision exceeds a threshold.

This pattern is your safety net for changes: it won’t guarantee correctness in all future real-world scenarios, but it protects against obvious regressions.

Practical wiring (what to run, when):

On every pull request: a fast suite (for example, 20 to 50 golden cases) plus red-team smoke tests.
Nightly: the full suite (golden + edge + simulation) with trend charts and top regressions surfaced.
Pre-release: lock the exact versions (model, prompts, retriever, tools), generate an evidence bundle, and require SME sign-off for any warnings.

Evidence bundle checklist (what CI should publish as artifacts):

Suite definitions and dataset versions (hashes).
Prompt, scaffold, retriever, and tool versions.
Raw outputs plus per-case scores (JSON/CSV).
Diff report against baseline (what regressed, what improved).
Links to representative traces for the top failures (so debugging starts from evidence, not guesswork).

Minimum viable implementation:

One fast suite on PR, one full suite nightly.
A single gate policy file per environment (dev, staging, pilot).
A diff view against a pinned baseline, not just absolute scores.
A short SME review queue for warnings (borderline cases and policy nuance).

Figure 7-2 shows how CI gates enforce these patterns.

Figure 7-2. CI pipeline wiring for eval gates

flowchart TB
  PR[Pull request] --> UT[Unit checks<br/>schemas, parsers, allowlists]
  UT --> REG[Offline regression<br/>golden set]
  REG --> J[LLM-as-judge<br/>pass/fail]
  J --> SCORE[Scorecard diff<br/>vs baseline]
  SCORE -->|hard gate fail| BLOCK[Block merge<br/>open bug + attach evidence]
  SCORE -->|soft gate fail| WARN[Warn, triage queue<br/>SME review]
  SCORE -->|all gates pass| MERGE[Merge + deploy to staging]

📚 Related sections: - Harness design → Chapter 5 - Production tooling → Chapter 11, §11.4

7.2.2 Scenario / Journey-Based Evaluation

Traditional ML evaluation focuses on single-step predictions. Agents, especially workflow agents, operate over multi-step journeys.

Scenario/journey evaluation focuses on:

The full trace:
How many steps did the agent take?
Did it backtrack unnecessarily?
Did it confuse the user with redundant questions?
The user experience:
Did the user feel guided or lost?
Were they bounced between states?

Example for HR Leave:

Two agents might both reach the correct decision:
Agent A asks three focused clarifying questions and decides.
Agent B loops with five redundant questions and one failed tool call.

Task success metrics alone would call these equal; journey evaluation would not.

Metrics for journey evaluation can include:

Number of steps / spans per trace.
Number of backtracks or repeated questions.
Number of failed or retried tool calls.
Overall effort for the user (e.g., messages typed).
Time spent in each phase.

These can then feed into:

UX metrics (“friction index”).
Cost calculations (extra LLM and tool calls).
Quality judgments (does this journey feel professional and competent?).

From traces to journey metrics (a simple mental model):

Trace: one user request and the full agent run.
Spans: the steps inside the trace (LLM calls, retrieval, tool calls, approvals, retries).
Features: counts and durations derived from spans.
Journey metrics: aggregations that capture friction, efficiency, and robustness.

Figure 7-3 shows how a trace captures the full story of an agent session.

Figure 7-3. Trace anatomy for a tool-using agent

sequenceDiagram
  participant U as User
  participant A as Agent (scaffold)
  participant R as Retriever
  participant T as Tool / API
  participant O as Observability (trace)

  U->>A: Request
  A->>O: Start trace
  A->>R: Retrieve policy snippets
  R-->>A: Context
  A->>T: Tool call (create ticket / check balance)
  T-->>A: Tool result
  A-->>U: Final response
  A->>O: End trace, spans + artifacts

Example: a lightweight friction index you can compute from telemetry:

friction_index =
  w1 * (step_count / step_budget) +
  w2 * retry_count +
  w3 * failed_tool_calls +
  w4 * user_turns

Two practical tips:

Attach step-level judgments to spans (retrieval relevance, tool-call validity, policy check outcome). Attach task-level judgments to the trace (decision correctness, user satisfaction, safety).
Define journey expectations as milestones, not scripts. For HR Leave, a milestone might be “policy identified”, “eligibility checked”, “next action communicated”, “handoff created if needed”.

Minimum viable implementation:

Log traces with consistent span names and IDs (so metrics can be computed reliably).
Define 3 to 5 journey metrics that matter (step count, retries, failed tools, latency, user turns).
Add a small scenario suite that covers end-to-end journeys, not just single-turn Q&A.

In your harness, you can explicitly represent scenarios and attach expectations not just on the final state but on the path taken.

7.2.3 Red-Team and Safety Harness Pattern

📋 Adversarial Cases: Appendix D, §D.4 provides ready-to-use adversarial test cases: prompt injection attempts (ADV-001), PII extraction attempts (ADV-002), and fabricated policy claims (ADV-003).

Red-teaming is about deliberately trying to break the system.

For agents, a red-team harness:

Uses curated red-team suites and adversarial simulations.
Emphasises safety and policy metrics, not just correctness.

For the HR Leave Agent, red-team scenarios might include:

Prompts trying to get information about other employees’ leave or medical history.
Attempts to exploit policy loopholes (“how do I get extra leave without HR noticing?”).
Aggressive or abusive language to see if the agent responds inappropriately.

Evaluation here focuses on:

Safety metrics , PII leakage, policy violations, harmful or discriminatory language.
Boundary behaviour , does the agent appropriately refuse or escalate?

Patterns:

Run red-team harnesses:
Before major launches.
On a schedule (e.g., weekly).
After changes in policy or in safety guardrails.
Have SMEs, security and risk teams review:
The most concerning failures.
Trends in safety metrics over time.

A well-structured red-team harness becomes part of your safety case: evidence you can show to internal risk committees or regulators.

7.2.4 Online Evaluation: Shadow Mode, A/B Tests, Canary Rollouts

Offline evaluation is necessary but not sufficient. Agents will behave differently on live traffic:

Inputs are messier.
Users explore unexpected paths.
System load varies.

Online evaluation patterns mirror those in web/backend engineering, with some agentic twists.

Shadow mode

The agent runs in parallel with existing processes, but its outputs are not shown to users or are only advisory.
For HR Leave:
While HR continues to make decisions, the agent proposes decisions in the background.
You compare agent vs HR decisions and explanations.
You use this to refine metrics, rubrics, and thresholds before enabling auto-approval.

A/B tests

Test variations of:
Prompts.
Scaffolds (e.g., different planning strategies).
Models (base vs fine-tuned, different providers).
Split traffic between variants:
Measure business metrics: resolution rate, CSAT, escalation rate.
Monitor safety metrics closely for both arms.

Canary rollouts

When enabling a new behaviour (e.g., auto-approval for low-risk leave), start with:
A small percentage of traffic.
A narrow segment (e.g., a single department or region).
Set strict rollback triggers:
Any safety violation.
Significant degradation on primary metrics.

Online evaluation bridges the gap between lab benchmarks and real-world performance, but you must design it with safety in mind: start with low-impact settings and ramp up only when metrics support it.

Rollout is only safe if rollback is pre-decided.

Define explicit triggers for each stage (examples):

Immediate rollback (hard): any policy violation confirmed, a spike in tool misuse, or a new safety class breach.
Pause and review (primary): decision correctness drops beyond the allowed band, escalations jump sharply, or P95 latency exceeds the cap.
Continue with tighter oversight (soft): friction index rises, cost increases, or user sentiment trends down, but hard gates are still clean.

Minimum viable implementation:

Shadow mode for at least one week of representative traffic (or until you hit a minimum sample size you trust).
Canary rollout with staged exposure (for example, 1% → 5% → 25% → 100%) and automated rollback triggers.
A human review queue for the first tranche of high-impact decisions during ramp-up.

Figure 7-4 shows the canary decision tree. Treat canary rollout as a safety system: watch thresholds, rollback fast, learn.

Figure 7-4. Canary decision tree

flowchart TD
  S[Deploy canary] --> M[Monitor leading indicators<br/>guardrail triggers, task pass rate]
  M --> Q{Lower bound crosses gate?}
  Q -->|Yes| R[Rollback<br/>or reduce autonomy]
  Q -->|No| N{New failure cluster?}
  N -->|Yes| T[Triage and patch<br/>add golden cases]
  N -->|No| P[Promote rollout]

📚 Related sections: - Production monitoring → Chapter 11, §11.4 - Drift detection → Chapter 11, §11.4.4 - Rollback triggers → Chapter 11, §11.2

7.2.5 Monitoring, Telemetry & Drift Detection

Once an agent is in production, evaluation shifts from “should we launch this?” to “is it still behaving acceptably?”

Monitoring builds on the telemetry you set up in Ch5:

Logs of inputs and outputs.
Spans and traces for each request.
Aggregated metrics over time.

Before diving into practices, it helps to distinguish three related concepts. Table 7-2 clarifies when to use guardrails, evaluators, and monitors.

Table 7-2. Guardrails, evaluators, and monitors

Concept	Where it runs	Latency budget	Typical implementation	What to do on trigger	Operational stance
Guardrail	Inline, in request/response path	Milliseconds	Regex, schemas, block-lists, lightweight classifiers	Refuse, redact, regenerate	False positives are production bugs, keep conservative
Evaluator	Async, offline or nearline	Seconds to minutes	LLM-as-judge, human review, programmatic assertions	Create failure set, open bug, update dataset	Used for learning and quality measurement
Monitor	Nearline / dashboards	Minutes	Aggregations, drift detectors, alerting	Triage, rollback, tighten autonomy	Used for early warning and operations

Key practices: define a minimal set of live metrics per agent (task success proxies, safety proxies, performance, UX signals), set alert thresholds tied to your gate policy, and schedule periodic re-evaluation by re-running benchmark and simulation suites.

Watch for four types of drift:

Input drift: the distribution of user queries changes.
Output drift: the agent's style, tone or decisions shift (e.g., after a model update).
Behavioural drift: new failure modes appearing that old tests miss.
Judge drift: evaluator prompts or calibration guidelines change, shifting scores even if the agent did not.

When monitoring flags a problem: Triage (sample traces, classify failure) → Isolate (what changed?) → Mitigate (rollback, tighten gates, reduce autonomy) → Learn (turn failures into new golden cases and red-team scenarios).

Minimum viable implementation: weekly benchmark re-runs against production config, alert thresholds tied to gates, and a rollback path executable in minutes.

Figure 7-5 shows how production monitoring feeds failures back into your evaluation suite.

Figure 7-5. Production monitoring, sampling, and dataset refresh

flowchart LR
  LIVE[Live traffic] --> SHADOW[Shadow sampling<br/>async evaluators]
  SHADOW --> DASH[Dashboards<br/>drift, guardrail triggers]
  DASH -->|new failure cluster| REVIEW[Trace review<br/>error analysis]
  REVIEW --> ADD[Add to regression set<br/>golden cases]
  ADD --> CI[Next CI run]
  DASH -->|threshold breach| ROLLBACK[Rollback or tighten autonomy]

Monitoring closes the loop: when you detect drift, you go back to updating metrics, rubrics, datasets, and scaffolds.

7.2.6 Simulation-Based Stress and Policy Testing

Simulation (fundamentals in Ch6) shines in two evaluation scenarios:

Stress tests — generate many synthetic tasks (e.g., "pre-CNY leave season") and measure latency, throughput, error rates, and degradation in decision quality under load.

Policy-change tests — run the same synthetic employees and scenarios under old and new policies, then compare outcomes, legal/fairness boundaries, and explanation alignment. This gives stakeholders quantitative evidence about policy impacts before go-live.

Simulation-based evaluation runs through the same harnesses as other tests. The difference is that inputs are synthetic, controlled and scaled to cover risk and variability in a way real data often cannot.

Figure 7-6 shows the synthetic data generation workflow. Dimensions prevent synthetic data from becoming generic and repetitive.

Figure 7-6. Synthetic data generation workflow

flowchart LR
  D[Define dimensions<br/>intent, persona, edge cases] --> G{Many combos valid?}
  G -->|Yes| X[Cross-product tuples]
  X --> F[Filter tuples with LLM or rules]
  F --> S[Generate queries]
  G -->|No| Y[Direct tuple generation]
  Y --> S
  S --> RUN[Run through real system]
  RUN --> TR[Collect traces]
  TR --> EA[Manual error analysis]
  EA --> UPD[Update rubrics + datasets]

7.3 Use-Case Evaluation Playbooks

By now you’ve seen many moving parts. To make them practical, it helps to organise them into playbooks for different agent archetypes.

Each playbook answers:

What are the primary metrics?
Who are the SMEs?
What evaluators are used?
What datasets and simulations matter most?
What are typical gates and thresholds?

Table 7-3. Use case to evaluation approach (quick reference)

Before diving into detailed playbooks, use this table to identify the metrics and tools most relevant to your use case. The playbooks that follow expand on key archetypes.

Use case	What "good" means	Core metrics to measure	Best-fit tools	Example of how you'd use it	See also
RAG Document Q&A / Knowledge bot	Answer is grounded in retrieved content, cites the right passages	Faithfulness/groundedness, answer relevancy, context precision/recall, citation coverage, refusal correctness	Ragas (RAG metrics), Phoenix (trace + slicing), MLflow (tracking)	Run a golden query set; score faithfulness + context recall; inspect failures by retrieval miss vs generation	§7.3.1
Multi-turn conversational chatbot (RAG + memory)	Stays on-topic across turns, uses history correctly, consistent policy behavior	Turn-level groundedness, conversation coherence, instruction adherence, safety, escalation correctness	End-to-end harness with evidence bundling (e.g., scripted conversation replay), Phoenix (traces), Ragas (RAG metrics)	Replay multi-turn scripts end-to-end (including UI flows), capture transcripts/traces, run Ragas for grounding per turn, generate an evidence bundle	§7.3.1
Agentic workflow—read-only tools	Chooses right tool, correct parameters, uses tool output faithfully	Tool selection accuracy, parameter validity, execution success, output utilization faithfulness, efficiency (#calls)	Phoenix (tool traces), DeepEval (custom metrics), MLflow (tracking)	Score tool correctness from traces: right tool + correct args + final answer matches tool output	§7.3.2
Agentic workflow—with writes (HR Leave, ticketing, procurement)	Correct decision, safe state changes, policy compliant, audit-ready	Decision correctness, state-change safety, policy/legal compliance, tool call accuracy, escalation correctness	Phoenix (traces), Custom harness (state diff validation), MLflow (tracking), Evidence bundling	Validate state changes against expected outcomes; run policy critics before writes; generate audit trail	§7.3.2
Customer service chatbot	Correct, safe tone, policy compliant, resolves or escalates	Policy adherence, tone/empathy, hallucination rate, PII leakage, escalation correctness	DeepEval (safety + custom tests), Giskard (adversarial/red-team), End-to-end harness (conversation scripts + reporting)	Run policy/jailbreak suites; execute scripted conversations and produce an evidence pack for stakeholders	§7.3.1, §7.3.3
Meeting / call summarizer	Captures key points accurately, no hallucinations, consistent structure	Factual consistency, coverage/omissions, coherence, format adherence	DeepEval (summarization metric), Phoenix (trace + slicing), MLflow (tracking)	Score factuality + coverage; drill into low-score cases with trace-level diagnostics	—
Declassification / redaction	Finds what must be removed, avoids over-redaction	Precision/Recall, false positive/negative rate, category-level accuracy	Custom deterministic eval (best), MLflow (tracking)	Maintain SME "to-be-redacted" gold labels; compute P/R by entity type (NRIC, names, etc.)	§7.3.3
Text-to-SQL / Text-to-AQL	Query executes and returns correct results safely	Execution success, result equivalence, schema validity, permission checks	Custom eval harness + MLflow	Run generated queries on a test DB; compare results to ground truth; enforce RBAC constraints	—
High-risk domain agent (healthcare, finance, legal)	Safe, legally compliant, explainable, fair	Safety, compliance, fairness, traceability, human-audit pass rate	Custom harness (strict gates), Giskard (bias/fairness), Human SME review (mandatory)	Extensive human audit; adversarial suites; conservative rollout with tight rollback triggers	§7.3.3
Multi-agent / orchestration	Coordination works, handoffs complete, no loops or deadlocks	Handoff completeness, delegation success, loop rate, tool contention, end-to-end latency	Phoenix (multi-agent traces), Custom orchestration metrics, MLflow	Tag spans by agent; validate handoff schemas; track per-agent vs system-level failures	§7.3.4

Tool legend:

Ragas: Open-source RAG evaluation framework with built-in metrics for faithfulness, relevancy, and context quality
Phoenix (Arize): Observability platform with tracing, slicing, and LLM-specific analytics
DeepEval: Testing framework for LLMs with safety, summarization, and custom metric support
Giskard: Testing platform focused on adversarial testing, bias detection, and red-teaming
MLflow: Experiment tracking and model registry (useful for versioning eval runs)
End-to-end harness: Custom or commercial harness that replays full user journeys and produces evidence bundles

7.3.1 Chat Copilot (Knowledge Assistant, Low,Medium Risk)

Examples: documentation copilot, internal Q&A for engineers, policy explainer where no automated actions are taken.

Primary metrics

Helpfulness / usefulness of answers.
Faithfulness to source documents (no hallucinated policy).
UX & satisfaction (CSAT, thumbs up/down).
Safety (refusing inappropriate requests).

SMEs

Knowledge owners and document authors.
Frontline support staff who know typical questions and expectations.

Evaluators

LLM-as-judge for helpfulness and faithfulness (rubrics defined by SMEs).
Rule-based checks to ensure citations come from allowed sources.
Human SME reviews on samples, especially early on.

Datasets & simulations

Q&A pairs derived from existing FAQs and tickets.
Synthetic paraphrases of common questions.
Red-team questions around sensitive or restricted topics.

Gates & thresholds

Target helpfulness and faithfulness scores.
Hard constraints on safety (e.g., no PII leaks, no harmful advice).
Launch initially as assistive-only (no direct actions), then gradually increase exposure.

7.3.2 Workflow Agent (HR Leave, Procurement, Ticketing)

Workflow agents take actions in systems-of-record. Mistakes directly affect business processes and people.

We’ll go deeper on the HR Leave Agent here.

Primary metrics

Decision correctness (approve/reject/route).
System state correctness (records and balances updated properly).
Policy and legal compliance (no violations).
Time to decision vs SLA.
HR workload reduction and escalation rates.

SMEs

HR operations and business partners.
Legal/compliance.
Sometimes unions or representative bodies.

Evaluators

Rules:
Check basic invariants (balances, dates, required docs).
Verify that all required system updates occurred.
LLM judges and critic agents:
Policy critics for compliance and explanation quality.
Tone critics for sensitive cases.
Human SMEs:
Label golden sets.
Resolve disagreements between rules and judges.
Periodically audit decisions, especially early on or in high-risk patterns.

Datasets & simulations

Historical leave cases with ground truth decisions.
Golden sets for routine and edge-case scenarios.
Red-team suites for privacy and abuse attempts.
Simulation suites for peak periods and new policy changes.

Gates & thresholds

Pre-deployment:
No policy/legal violations in benchmark suites.
Decision correctness above a high threshold (e.g., 99.5%) on golden sets.
Explanation quality above minimal acceptable average.
Operational:
Start in shadow mode compared with HR decisions.
Move to “recommend-only” mode with human approval.
Allow auto-approval only for low-risk segments (e.g., simple annual leave within policy), with strict monitoring.

HR Leave is a template for other workflow agents: the specific policies change, but the pattern , high emphasis on correctness, state, and compliance , is the same.

7.3.3 High-Risk Domain Agents (Healthcare, Finance, Legal, High-Stakes HR)

Some agents affect decisions where any error can have significant harm: medical advice, financial recommendations, legal guidance, disciplinary or compensation decisions.

Here, evaluation must be stricter:

Primary metrics

Safety, legality, and compliance.
Fairness and non-discrimination.
Traceability and explainability (why a decision was made).

SMEs

Domain experts: clinicians, financial advisors, lawyers.
Risk management and compliance teams.
Sometimes external auditors or regulators.

Evaluators

Extensive use of human SME evaluation on critical scenarios.
LLM-as-judge and critic agents, but always calibrated and audited.
Strong rule-based checks where possible.

Datasets & simulations

Carefully curated high-risk scenario libraries, with consensus labels.
Synthetic and simulated cases to cover rare but critical patterns.
Red-team suites co-designed with risk and security.

Gates & thresholds

Many metrics are non-negotiable: any violation is unacceptable.
Auto-approval might be restricted or disallowed:
Agents provide recommendations; humans make final decisions.
Strong requirements on logging and traceability for audits.

In these domains, evaluation is not just engineering hygiene; it is a core part of your ethical and regulatory obligations.

7.3.4 Multi-Agent / Orchestration-Heavy Systems

In multi-agent setups (e.g., a manager agent delegating to specialist workers), evaluation must consider both:

The behaviour of individual agents.
The coordination of the overall system.

Metrics might include:

Plan quality (manager’s decomposition of tasks).
Handoff effectiveness (does the right agent get the right task with sufficient context?).
Error propagation (does a mistake by one agent get caught or amplified?).
Looping and deadlocks.

Evaluators often rely heavily on traces:

Each trace shows which agents were invoked, with which roles and results.
You can attach labels to:
Individual agent outputs.
The overall outcome.

Simulation and red-team scenarios can be especially powerful here:

Design complex coordination tasks.
Observe how the team of agents behaves.
Adjust role definitions, communication protocols, and termination criteria.

The core lesson: multi-agent evaluation is not just "N agents × single-agent evaluation"; you need system-level metrics for coordination and robustness.

Table 7-4 shows system-level metrics that make multi-agent evaluation tractable.

Table 7-4. Multi-agent evaluation metrics (system-level)

Metric	What it measures	How to compute (from traces)	Common failure signature
`handoff_completeness`	Worker gets the context it needs	Required fields present in delegation message	Worker asks for missing info, rework loops
`delegation_success_rate`	Delegation produces usable output	% worker outputs accepted without re-run	Manager retries the same task
`loop_rate`	System gets stuck	% traces exceeding step budget or repeating intents	Plan-execute oscillation
`tool_fanout`	Tool usage discipline	Tool spans per trace, unique tools used	Spurious tool calls, latency spikes
`consistency_across_workers`	Agreement and stability	Compare worker outputs on same inputs	Conflicting answers, unstable behavior

Minimum viable implementation:

A shared trace ID across all agents, with spans tagged by agent name and role.
A small set of handoff schemas, validated automatically at runtime and in eval harnesses.
A failure taxonomy that distinguishes per-agent errors from orchestration errors.

7.4 Anti-Patterns and Common Failure Modes

As you operationalise evaluation, certain mistakes crop up repeatedly.

Benchmark worship
Over-optimising for generic benchmarks or a single internal score.
Ignoring business KPIs and user experience.
Fix: Always anchor scores in business impact and SME-defined metrics.
No SMEs in the loop
Engineers define rubrics and thresholds alone.
Domain experts are only asked to review outputs after the fact.
Fix: SMEs should co-design metrics, rubrics, and gates from the start.
Evaluation that doesn’t see the real artefacts
Tools only store prompt/response pairs.
No visibility into tool calls, system state changes, or traces.
Fix: Build data adapters; treat spans and traces as first-class inputs for evaluation.
One-time evaluation
Evaluation done once at launch.
No regression tests, no monitoring, no periodic re-runs of benchmarks.
Fix: Embed evaluation into CI/CD and scheduled jobs; treat it as continuous.
LLM-as-judge as a black box
Using canned “Is this good?” prompts.
No SME rubrics, no calibration against human labels.
Fix: Treat judges as models: give them good rubrics and test them.
No simulation for tail risks
Relying only on real logs, which under-represent rare but dangerous cases.
Fix: Use synthetic and simulated scenarios to exercise the edges.
Tools that don’t fit the domain
Clinging to an evaluation tool that can’t handle your data shape or labels.
As a result, falling back to shallow metrics.
Fix: Extend or build tooling that supports your schemas and rubrics; don’t let the tool dictate the evaluation.

Recognising these anti-patterns early lets you correct course before evaluation becomes a checkbox activity instead of a real risk and quality control mechanism.

📚 Related sections: - Best practice patterns → Chapter 9 - Production gates → Chapter 11, §11.7

7.5 User Research and Feedback Loops

Automated evaluation harnesses are necessary but not sufficient. They tell you whether the agent produces correct outputs—they don't tell you whether users trust it, adopt it, or use it effectively in their actual workflow.

This section covers how to close the gap between "correct" and "useful."

7.5.1 When automated evals are not enough

Automated evaluation misses important dimensions. Table 7-5 shows the gaps.

Table 7-5. What automated evaluation misses

Dimension	What harnesses measure	What they miss
Trust	Output correctness	Whether users believe and act on the output
Workflow fit	Task completion	Whether the agent fits into existing work patterns
Cognitive load	Response quality	Whether users can efficiently process agent outputs
Error recovery	Failure handling	Whether users can correct agent mistakes effectively
Adoption barriers	Feature coverage	Why users might avoid or bypass the agent
Edge cases	Known test scenarios	The scenarios you didn't think to test

HR Leave example gaps:

Harness says the eligibility check is 95% accurate
But users don't trust it because they can't see the reasoning
Users bypass the agent and email HR directly
The 95% accuracy is meaningless if adoption is 20%

7.5.2 Lightweight user research methods for agents

You don't need a full UX research lab. These methods work with small samples and fit into normal development cycles:

1. Think-aloud sessions (5–10 users)

Setup: - Recruit representative users (mix of tech-savvy and less comfortable) - Give them real tasks from their actual work - Ask them to verbalize their thoughts as they interact with the agent - Record screen and audio (with consent)

What to capture: - Points of confusion or hesitation - Moments of trust or distrust ("I don't believe this," "This seems right") - Workarounds and shortcuts they attempt - Comparisons to their current process

HR Leave example prompts:

"You want to take 3 days off next week for a family event. Use the agent to figure out if you can and how to request it."
"You're not sure if your sick leave counts against your annual leave. Find out."

2. Diary studies (1–2 weeks)

Setup: - Ask 10–15 users to log their agent interactions for a week - Provide a simple template: task, outcome, satisfaction (1–5), comments - Follow up with brief interviews

What to capture: - Patterns of use (what tasks, how often) - Abandonment points (started but didn't finish) - Unexpected use cases - Comparison with alternatives

3. Intercept surveys (ongoing)

Setup: - After agent interactions, occasionally prompt for feedback - Keep it short: 1–2 questions maximum - Rotate questions to cover different dimensions

Example questions: - "Was this response helpful?" (yes/no) - "Did you trust this response enough to act on it?" (yes/no) - "What would have made this response more useful?" (open text) - "Did you need to check elsewhere to confirm this?" (yes/no)

4. Override and correction analysis

The richest feedback comes from what users actually do. Table 7-6 shows the signals to capture.

Table 7-6. Override and correction signals

Signal	What it indicates	How to capture
Override rate	Users disagree with agent suggestions	Log when users edit or reject agent outputs
Correction patterns	Systematic errors	Cluster corrections by type
Escalation requests	Agent can't handle the task	Log explicit "talk to human" requests
Session abandonment	Frustration or confusion	Track incomplete sessions
Time-to-completion	Efficiency vs friction	Compare to baseline manual process

7.5.3 Feedback collection in production

Design feedback collection into the UX:

Passive signals (no user action required):

Task completion rate
Time from start to completion
Number of turns before resolution
Override/edit rate
Escalation rate
Return rate (same user, same question type)

Active signals (user provides feedback):

Thumbs up/down on responses
"Was this helpful?" prompt
Correction capture ("What should I have said?")
Issue reporting ("Something went wrong")

Sampling strategy:

Don't ask for feedback on every interaction (fatigue)
Sample 5–10% of interactions for active feedback
Stratify by user segment, task type, and outcome
Higher sampling for new features or after changes

7.5.4 Closing the loop: feedback to improvement

Feedback is only valuable if it drives change. Figure 7-7 shows the feedback processing flow.

Figure 7-7. Feedback processing flowchart

flowchart LR
  A[User feedback] --> B[Categorize signal]
  B --> C{Signal type}
  C -->|Correctness issue| D[Add to golden set]
  C -->|Trust issue| E[Improve explanation]
  C -->|Workflow issue| F[UX change]
  C -->|Coverage gap| G[Expand capability]
  D --> H[Harness improvement]
  E --> H
  F --> I[Product backlog]
  G --> I
  H --> J[Release with regression]
  I --> J
  J --> K[Monitor in production]
  K --> A

Feedback processing workflow:

Triage (daily/weekly):
Review feedback volume and sentiment trends
Identify patterns and clusters
Escalate urgent issues (safety, compliance)
Analysis (weekly):
Deep dive into correction patterns
Root cause analysis on trust issues
Cross-reference with harness results
Prioritization (sprint planning):
Convert feedback insights into backlog items
Balance user-reported issues with harness-detected issues
Include "close the loop" items (fix + add test case)
Validation (post-release):
Track whether fixes resolved the feedback pattern
Monitor for regression in related areas

7.5.5 Avoiding "happy path" bias

User research naturally gravitates toward successful interactions. Counteract this:

Intentional failure exploration:

Ask users to "try to break it" or "find something it can't do"
Include tasks at the edge of agent capability
Test with users who are skeptical or resistant to AI

Segment by comfort level:

Recruit both AI-enthusiastic and AI-skeptical users
Compare their experiences and trust patterns
Skeptics often surface issues enthusiasts overlook

Shadow real workflows:

Observe users in their actual work context
Watch for moments they choose NOT to use the agent
Ask about alternatives they use instead

Track non-use:

Monitor which users never engage with the agent
Survey non-users about barriers
Don't assume "available" means "adopted"

7.5.6 Metrics that connect to user research

Add user-centered metrics to your evaluation registry. Table 7-7 shows the key metrics.

Table 7-7. User-centered metrics registry

Metric	Definition	Target	Source
Adoption rate	Active users / eligible users	>60%	Usage logs
Trust ratio	"Acted on without checking" / total	>70%	Survey
Net promoter (agent)	Would recommend to colleague	>30	Survey
Override rate	User edits / total outputs	<15%	Logs
Escalation rate	Human requests / total	<10%	Logs
Return engagement	Users who return / new users	>80%	Logs
Task completion (user view)	User says task done / started	>90%	Survey

7.5.7 HR Leave example: user research integration

Table 7-8 shows the research schedule for the HR Leave Agent.

Table 7-8. HR Leave user research schedule

Phase	Method	Sample	Focus
Pre-pilot	Think-aloud sessions	8 users	Workflow fit, comprehension
Pilot week 1–2	Diary study	15 users	Daily use patterns, pain points
Pilot ongoing	Intercept surveys	10% sample	Satisfaction, trust signals
Production	Override analysis	All interactions	Systematic corrections
Quarterly	Deep-dive interviews	10 users	Long-term adoption, feature requests

Table 7-9 shows how user feedback translates into improvements.

Table 7-9. Feedback to improvement examples

Feedback signal	Root cause	Improvement
Users ask "but can I really take this leave?"	Response lacks explicit approval status	Add clear "This leave is approved" or "Requires manager approval" statement
High abandonment at policy explanation	Explanations too long	Summarize first, expand on request
Users email HR after using agent	Trust deficit	Add "Verified against policy v2024.3" with timestamp
"I don't know what happened"	No confirmation after submission	Add explicit confirmation with request ID

📚 Related sections: - UX design → Chapter 10, §10.1 - Shadow AI failure case → Chapter 14, §14.6.3 - Adoption metrics → Chapter 11, §11.4

7.6 Evaluation Planning Checklist

To close the loop, here’s a practical checklist you can use whenever you design or review evaluation for a new agent.

1. Business and Risk Context

What business problem is this agent solving?
What are the primary KPIs (e.g., resolution rate, cost savings, CSAT)?
What risks are in scope (legal, safety, fairness, privacy)?

2. Stakeholders and SMEs

Who owns the business process?
Which SMEs will define correctness and safety (ops, legal, risk)?
Who can approve metrics, rubrics and thresholds?

3. Metrics and Rubrics

What are the primary metrics in each category:
Task success, quality, safety, latency, cost, UX?
What rubrics (questions and scales) will SMEs use?
Which rubrics are non-negotiable vs tolerant?

4. Evaluators

What mix of evaluators will you use:
Rules, reference metrics, LLM-as-judge, critic agents, human SMEs?
How will you calibrate LLM judges and critic agents against SME labels?
How often will you audit evaluators for drift or bias?
How will you version evaluator prompts and calibration guidelines so score shifts are explainable? 5. Datasets, Benchmarks and Simulation
What are your sources of test cases:
Logs, SOPs, red-team scenarios, simulations?
What golden sets will you maintain with high-quality SME labels?
What benchmark suites will you define (standard, edge, red-team)?
What simulation plans do you have for stress and policy change?

6. Tooling and Integration

Does your evaluation tool support:
Your data schema (conversations, traces, environment state)?
Your label schema and rubrics?
Multiple judge backends (humans, LLMs, rules)?
How will SMEs access and interact with the evaluation workspace?
Where do you version evaluation artifacts (schemas, rubrics, gates, suite definitions), and how do they get reviewed?
What does your evidence bundle include (versions, diffs, trace links) for audit and sign-off? 7. Harness Patterns and CI/CD
Which harnesses (from Ch5) will you use:
Offline regression, red-team, simulation, shadow mode, A/B, canaries?
How are benchmark suites wired into CI/CD?
What are your automated gates for merges and deployments?
What runs on pull requests vs nightly vs pre-release, and what evidence artifacts are produced?
Which gate policy files are authoritative per environment (dev, staging, pilot, prod)? 8. Monitoring and Drift
Which live metrics will you monitor (task success proxies, safety proxies, UX, cost)?
What thresholds trigger alerts and potential rollback?
How often will you re-run benchmarks and simulations against the current system?
What is your incident runbook (triage, isolate, mitigate, learn), and who is on call?
How will you detect and manage judge drift (evaluator prompt changes, calibration changes)? 9. Decision Policies
Under what conditions is auto-approval allowed vs human review required?
Who can override evaluation gates, and under what circumstances?
How will you document and communicate these policies to stakeholders?

If you can answer these questions concretely for each agent you build, evaluation stops being a fuzzy concern and becomes:

A designed part of the system, not an afterthought.
A shared language between engineers, SMEs, risk and product.
A lever you can systematically pull to improve quality, safety and business impact.

Together, Chapters 6 and 7 form the evaluation spine of this handbook. Everything else, scaffolds, tools, orchestration, and policy, sits on top of this spine. Without it, you might build clever demos, but you will not know if they are safe, useful, or worth scaling.

With it, you can build agentic systems that not only act, but act repeatably, improve over time, and earn trust through evidence.

7.7 System optimization playbook: climbing the eval hill (HR Leave Agent)

Once Chapter 6 gives you metrics, judges, and datasets, this chapter is where you use them to systematically improve the agent. Treat your scorecard like a product objective function, then change one system component at a time with tight regression gates. This is the evaluation flywheel mindset: analyze failures, measure with graders, improve, repeat.

Primary reference: OpenAI Cookbook, Building resilient prompts using an evaluation flywheel.

7.7.1 Start by localizing the bottleneck (use trace grading)

Before you optimize anything, classify failures by where they happen in the trace: retrieval span, tool span, planning span, final decision span, or safety span. Trace grading is designed for exactly this. You grade the full run and the critical steps, so you know which lever to pull next.

Primary reference: OpenAI, Trace grading.

Figure 7-8. Bottleneck localization flowchart

flowchart TB
  A[Failing HR Leave cases from golden set or production traces] --> B[Grade trace outcome: decision, state change, safety]
  B --> C{Where did it fail?}
  C -->|Retrieval| R[Optimize retrieval and corpus]
  C -->|Tool calls| T[Harden tools, schemas, retries, fallbacks]
  C -->|Reasoning or format| P[Optimize prompts and output contracts]
  C -->|Safety or PII| G[Add guardrails and critics]
  C -->|Workflow logic| S[Restructure scaffold and orchestration]

Rule of thumb: if the evidence is wrong or missing, fix retrieval and corpus first. If evidence is right but the decision is wrong, fix prompts, rubric clarity, and workflow logic.

7.7.2 The optimization ladder (no fine-tuning first)

Work bottom-up through the system. Each rung assumes you already have a stable eval suite (golden set plus risk suites) and a scorecard with hard constraints. For HR, policy compliance and PII safety are usually zero tolerance.

Anthropic recommends separating capability evals (show you the hill to climb) from regression evals (keep you from sliding backwards).
Primary reference: Anthropic, Demystifying evals for AI agents.

Table 7-10. The optimization ladder

Step	What you change	When it's the right lever	HR Leave example	Metrics to watch	Recommended gates (example)
1) Optimize retrieval and corpus	Chunking, metadata, filters, query rewriting, hybrid search, reranking, policy versioning	Trace shows wrong or missing policy clauses in top-k, or the right clause is retrieved but buried	Childcare leave restriction for probation not retrieved, agent guesses eligibility	retrieval_precision@k, retrieval_recall@k, evidence_coverage, hallucination_proxy	Hard: `policy_violation = 0`, `pii_leakage = 0`. Soft: retrieval@k improves, P95 retrieval latency within budget
2) Harden tools and contracts	Tool schemas, input validation, idempotency, retries, fallbacks, write barriers, output schema enforcement	Failures cluster around tool errors, bad payloads, partial writes, inconsistent state	HRIS update attempted without required approver, missing leave dates, or missing eligibility checks	tool_call_accuracy, tool_failure_rate, schema_pass_rate, span_error_rate	Hard: no unsafe writes. Soft: tool_failure_rate below threshold, schema_pass_rate ≥ 99%
3) Optimize prompts (and few-shot)	Developer prompt, tool instructions, output schema guidance, examples	Evidence is correct and tools work, but the decision framing is wrong or incomplete	Response omits required documents and next steps even though policy is retrieved	decision_correctness, completeness, explanation_faithfulness, schema_pass_rate	Hard: `policy_violation = 0`, `pii_leakage = 0`. Soft: decision_correctness not worse, completeness improves
4) Add critics and guardrails	Deterministic validators, policy critic, PII critic, tool-use critic, escalation rules	Failures are safety shaped: policy mistakes, leakage, jailbreak requests, tool misuse	User asks: “Tell me my colleague’s leave balance”; agent must refuse and not retrieve	pii_leakage, policy_violation, jailbreak_resistance, false_block_rate	Hard: `pii_leakage = 0`. Soft: false blocks ≤ target, safe escalation rate improves
5) Restructure the scaffold (workflow logic)	Plan and execute steps, checkpoints, “uncertainty to escalate”, state machine, multi-step verification	Failures are coordination: loops, missing approvals, wrong ordering, premature tool calls	Agent writes to HRIS before eligibility and approver are confirmed	end_to_end_success, loop_count, tool_call_accuracy, time_to_decision	Hard: no unsafe auto-approval or unsafe writes. Soft: loop_count down, end_to_end_success up, P95 time acceptable
6) Fine-tune non-LLM components	Train or tune intent router, leave-type classifier, reranker, entity extractor, OCR post-processor, rules engine	The bottleneck is a specific subtask that can be learned deterministically	Train a reranker to surface “exceptions and overrides” clauses; improve extraction of medical cert dates	subtask_accuracy, span_error_rate, downstream decision lift	Hard: no safety regressions. Soft: subtask metric meets threshold on held-out set, end-to-end improves
7) Fine-tune the LLM	Not covered in this handbook	Consider only with strong graders, stable datasets, and a clear need for model-level shifts	Not covered	N/A	N/A

Notes: - For prompt iteration, you can do manual changes or use evaluation flywheel techniques to make prompt changes measurable and robust. See the OpenAI Cookbook link above. - Hamel Husain and Shreya Shankar recommend focusing on clear, actionable pass or fail labels and systematic error analysis over vague Likert scoring, especially early on.
Primary reference: Hamel Husain and Shreya Shankar, LLM Evals FAQ.

7.7.3 Worked HR Leave loop: from failing cases to a shipped improvement

Scenario (golden set): “Employee on probation requests childcare leave, asks if eligible, and wants the agent to submit the request.”

Observed failure (trace-graded):

Retrieval span pulled a general childcare leave clause but missed the probation restriction and union addendum.
Decision span approved automatically and attempted an HRIS update.

Apply the ladder:

1) Retrieval and corpus first (evidence-related failure).
- Add metadata filters: leave_type = childcare, employment_status = probation, union_covered = true or false.
- Add reranking tuned for “eligibility constraints” language (often short clauses that get buried).
- Gate: retrieval metrics improve and policy_violation = 0 remains true across the legal-risk suite.
Primary reference: OpenAI, Evaluation best practices.

2) Add a policy critic as a hard gate before HRIS writes.
- Policy critic checks: “Does the decision contradict any retrieved clause, including exceptions?”
- If critic flags uncertainty or conflict, force escalation to HR and do not write to HRIS.
- Gate: policy_violation = 0, and false blocks stay below tolerance (otherwise users lose trust).
Primary reference: Hamel Husain and Shreya Shankar, LLM Evals FAQ.

3) Restructure scaffold to enforce verify then act.
- Step ordering: retrieve, identify constraints, draft decision, run critics and validators, then tool write.
- Gate: end-to-end success up, tool misuse down, loop_count not increasing.
Primary reference: OpenAI, Trace grading.

4) Prompt improvements (reduce ambiguity and improve explanations).
- Make “eligibility constraints” an explicit checklist item in the decision template.
- Require a “next steps” block (documents, approvals, timeline) and an “escalation” block when uncertain.
- Gate: completeness and schema_pass_rate improve without quality regressions.
Primary references: OpenAI Cookbook flywheel, OpenAI best practices.

5) Optional: train a reranker or classifier (only if retrieval remains brittle).
- Train on SME-reviewed traces labeled with “required clause set”.
- Gate: clause recall improves on held-out cases and yields measurable lift in decision correctness without safety regressions.

7.7.4 Recommended release gates per rung (practical defaults for HR Leave)

Use a small set of hard constraints that never relax, plus soft gates that evolve as you mature.

Hard constraints (always):

pii_leakage = 0
policy_violation = 0 on the legal-risk suite
“No HRIS writes when uncertainty is flagged” (validator or critic enforced)

Soft gates (typical):

decision_correctness ≥ baseline (or +X pp) on the golden set
P95 time_to_decision within budget
tool_failure_rate below threshold
false_block_rate below threshold

This aligns with OpenAI guidance on designing evals with clear thresholds and continuously using evals to catch regressions while you improve capability.
Primary reference: OpenAI, Evaluation best practices.

Sidebar: which rung to try first
- Retrieval metrics down or evidence missing, start at Step 1
- Tool spans show errors or unsafe writes, start at Step 2
- Evidence is right but decision is wrong, start at Step 3
- Safety failures or policy bypass attempts, start at Step 4
- Loops, handoff failures, or wrong ordering, start at Step 5

7.8 Key Takeaways

Evaluation is infrastructure, not a one-off activity — Treat evaluation artifacts (schemas, rubrics, gate policies, benchmark suites) as versioned code in your repository, reviewed and promoted alongside every agent change.
SMEs must own the rubrics and thresholds — Engineers build the platform, but domain experts define what "correct," "safe," and "acceptable" mean; without their input, metrics drift into meaningless scores.
Hard gates protect, soft gates inform — Separate non-negotiable constraints (zero policy violations, zero PII leakage) from tunable quality bands so that promotion decisions are explicit and auditable.
Lifecycle coverage prevents blind spots — Offline regression harnesses catch regressions before merge, red-team suites surface safety gaps before release, and shadow/canary rollouts with automated rollback triggers validate real-world behavior before full-scale deployment.
Journey evaluation reveals what task-level metrics hide — Two agents can produce the same correct answer while delivering vastly different user experiences; trace-level friction metrics (step count, retries, failed tool calls) expose the difference.
Simulation fills the gaps real data cannot — Synthetic stress tests and policy-change scenarios exercise rare but high-impact failure modes that production logs under-represent, giving stakeholders quantitative evidence before go-live.
Optimize bottom-up, one lever at a time — Localize the bottleneck through trace grading, then climb the optimization ladder (retrieval, tools, prompts, critics, scaffold, fine-tuning) with tight regression gates at each step to avoid trading one improvement for a hidden regression.

Next → Chapter 8 covers optimization — turning evaluation results into better prompts, trajectories, architectures, and production configurations through automated search.

7.9 Quick Reference Templates and Source Map

Copy-paste templates (metrics registry, gate policy matrix, synthetic data worksheet) and the full concept-to-source reference map for Chapter 7 have been extracted to Appendix F to keep this chapter focused on concepts and patterns.

📚 See: Appendix F — Evaluation Quick-Reference Templates and Source Map - Table F-1: Metrics registry template - Table F-2: Gate policy matrix template - Table F-3: Synthetic data worksheet (dimensions approach) - Section F.2 (Reference Map): Concept-to-source mapping for every section of Chapter 7