Skip to content

Chapter 8 — Optimization: From Evaluation to Improvement

TL;DR

  • Evaluation tells you what's wrong; optimization tells you how to improve — it is the outer loop that turns metrics, traces, and judgements into better prompts, scaffolds, skills, and production configurations.
  • Three optimization modes serve different needs: single-task search (solve one hard problem), multi-task search (batch with cross-transfer), and generalization (learn skills that transfer to unseen inputs via train/validation splits).
  • Actionable Side Information (ASI) — error traces, intermediate outputs, structured diagnostics — transforms blind score-based search into diagnostic-driven improvement, acting as the "gradient equivalent" for text optimization.
  • Agents are optimizable at every layer: prompts and system instructions, tool descriptions, multi-step trajectories, entire scaffold architectures, transferable skills, and production system policies.
  • Pareto-efficient search handles the competing objectives inherent in agents (accuracy vs latency, safety vs helpfulness, cost vs quality) by preserving candidates that excel on any dimension rather than optimizing a single blended score.

Chapters 6 and 7 gave you the measurement stack: metrics, judges, datasets, and gates. You can now answer "how good is this agent?" with precision.

This chapter answers the next question: "how do I make it better?"

Manual prompt engineering — rewriting instructions, adjusting few-shot examples, tweaking tool descriptions — works for early prototypes. But as agent complexity grows, manual iteration hits diminishing returns. You cannot efficiently explore the space of possible system prompts, tool schemas, routing policies, and scaffold configurations by hand.

Optimization treats improvement as a search problem. If an artifact is text and its quality is measurable, it can be optimized. The core insight: the same evaluation infrastructure you built in Chapters 5–7 — harnesses, golden sets, judges, metrics — becomes the fitness function for automated improvement.

This chapter covers what can be optimized (§8.1), the feedback signals that make search efficient (§8.2), the algorithms available (§8.3), and how to apply them to trajectories (§8.4), architectures (§8.5), skills (§8.6), and production systems (§8.7). We address multi-objective trade-offs (§8.8), infrastructure requirements (§8.9), walk through a complete HR Leave Agent optimization (§8.10), and close with anti-patterns to avoid (§8.11).

8.1 The Optimization Landscape

8.1.1 What Can Be Optimized

Any text artifact whose quality you can measure is a candidate for optimization:

  • Prompts and system instructions. The most common starting point. System prompts, role definitions, output format instructions, and policy summaries.
  • Tool descriptions and schemas. The natural-language descriptions that tell the model what a tool does, when to use it, and what parameters it expects.
  • Agent architectures (scaffolds). The control flow itself — routing logic, retry strategies, validation loops, sub-agent orchestration.
  • Skills and transferable capabilities. Repository-specific instructions, domain knowledge, best practices encoded as text that can transfer across tasks.
  • Production system configurations. Scheduling policies, routing algorithms, resource allocation strategies — any decision logic expressible as code or configuration.

8.1.2 Manual vs Automated Optimization

Manual prompt engineering follows a familiar loop: run the agent, read the output, identify a failure, rewrite the prompt, repeat. This works when:

  • The prompt is short and the failure mode is obvious.
  • You have fewer than a dozen test cases.
  • The agent has a single objective.

Reach for automated optimization when:

  • You have a golden set with 20+ examples and a reliable evaluator.
  • Manual iteration has plateaued — you fix one failure but introduce another.
  • The artifact is long or structurally complex (multi-section system prompts, multi-tool schemas).
  • You need to balance competing objectives (accuracy vs latency, safety vs helpfulness).
  • You want to discover solutions outside your own design intuition.

The cost-benefit is straightforward: manual iteration costs human time with diminishing returns; automated optimization costs compute with compounding returns as your evaluation suite grows.

8.1.3 The Three Optimization Modes

Mode Dataset Valset Outcome Example
Single-Task Search — — Solve one hard problem Circle packing, blackbox optimization
Multi-Task Search ✓ — Batch with cross-transfer CUDA kernels, API adapters
Generalization ✓ ✓ Skills that transfer Agent prompts, scheduling policies

Single-task search treats optimization as solving one problem. There is no dataset — just a seed candidate and an evaluator. The optimizer mutates the candidate, evaluates it, and keeps improvements. This mode suits hard algorithmic problems where the goal is a single best solution.

Multi-task search operates over a dataset of related problems. The optimizer works on each problem but transfers insights across them — a mutation that helps on problem A may be tried on problem B. This mode suits batches of structurally similar tasks.

Generalization adds a held-out validation set. The optimizer trains on the dataset but measures success on unseen examples. This is the mode you want for agent prompts: you need instructions that work on future user requests, not just the golden set you trained on.

Cross-reference: Golden set management and train/validation splits are covered in Chapter 7, §7.4. The evaluation infrastructure from Chapter 5 provides the harness that makes all three modes possible.

8.2 Actionable Side Information (ASI)

8.2.1 Beyond Scalar Rewards

A score tells you how much a candidate failed. ASI tells you why and how it failed.

Consider the difference:

  • Score only: "Policy accuracy: 0.6" — the optimizer must blindly mutate and hope.
  • Score + ASI: "Policy accuracy: 0.6. Missed citations: parental leave §3.2, bereavement §5.1. Tool sequence error: called submit_leave before check_balance. Latency: 4200ms (budget: 3000ms)." — the optimizer can target specific failures.

ASI is the "gradient equivalent" for text optimization. In numerical optimization, gradients tell you which direction to move. In text optimization, ASI tells the optimizer which aspects of the candidate to change and what kind of change is needed.

8.2.2 Designing Rich Evaluator Feedback

Effective ASI combines multiple signal types:

  • Error traces and stack traces. When a tool call fails, include the full error — not just "tool failed" but the specific validation error, timeout, or permission denial.
  • Intermediate outputs and reasoning chains. The model's chain-of-thought, retrieved documents, and intermediate decisions. These reveal where in the reasoning the failure originated.
  • Structured diagnostics. Cost breakdowns, latency profiles, token counts per step, cache hit rates. These enable cost-aware and latency-aware optimization.
  • Missing evidence. What the model should have retrieved or cited but did not. This requires reference answers in your golden set.

8.2.3 ASI in Practice: HR Leave Agent

For the HR Leave Agent, an evaluator that returns rich ASI might look like this:

def evaluate_leave_agent(candidate_prompt: str, example: dict) -> tuple[float, dict]:
    """Evaluate a candidate system prompt on one leave request example.

    Returns:
        score: float between 0.0 and 1.0
        asi: dict with structured diagnostic information
    """
    result = run_agent(system_prompt=candidate_prompt, user_request=example["request"])

    # Compute sub-scores
    policy_score = check_policy_citations(result.response, example["expected_citations"])
    tool_score = check_tool_sequence(result.tool_calls, example["expected_tools"])
    safety_score = check_no_cross_person_leakage(result, example)

    overall = 0.4 * policy_score + 0.3 * tool_score + 0.3 * safety_score

    asi = {
        "policy_citations_found": result.citations,
        "policy_citations_missing": example["expected_citations"] - result.citations,
        "tool_sequence_actual": [t.name for t in result.tool_calls],
        "tool_sequence_expected": example["expected_tools"],
        "safety_violations": result.safety_flags,
        "latency_ms": result.latency_ms,
        "token_count": result.total_tokens,
        "error": result.error_message,
    }

    return overall, asi

The optimizer receives both the score and the ASI dictionary. When reflecting on failures, it can identify that the candidate missed specific policy sections, called tools in the wrong order, or exceeded latency budgets — and generate targeted mutations.

Cross-reference: Evaluator types that produce rich feedback are covered in Chapter 6, §6.4. The judge calibration techniques in Chapter 7, §7.1 ensure that ASI is reliable enough to guide optimization.

8.3 Optimization Algorithms

8.3.1 Program-Level Optimization (DSPy)

DSPy treats LLM programs as compositions of typed modules (signatures) and optimizes them holistically. Its MIPROv2 optimizer:

  1. Proposes instruction candidates for each module using the training set.
  2. Selects few-shot examples from a labeled dataset.
  3. Evaluates full program pipelines (not isolated modules).
  4. Uses Bayesian optimization to allocate evaluation budget efficiently.

When to use DSPy: Your agent is structured as a pipeline of discrete modules with clear input/output signatures. DSPy excels at optimizing few-shot examples and module-level instructions within a fixed program structure.

Limitations: DSPy optimizes within a fixed program topology. It does not discover new architectures, add modules, or restructure control flow.

8.3.2 Reflective Prompt Evolution (GEPA)

GEPA (Generalized Evolutionary Prompt Architect) takes a different approach: it treats the optimizer itself as an LLM that reflects on failures.

The core loop:

  1. Evaluate the current candidate on a minibatch of examples, collecting scores and ASI.
  2. Reflect using an LLM: given the candidate, the failures, and the ASI, generate a natural-language analysis of what went wrong and what to change.
  3. Mutate by applying the reflection to produce a new candidate.
  4. Select using Pareto-efficient comparison across all objectives and tasks.

GEPA's key innovations:

  • Minibatch reflection. Each iteration focuses on 2–3 examples, not the full dataset. This produces focused, actionable reflections rather than vague summaries.
  • Pareto frontier maintenance. Instead of a single "best" candidate, GEPA maintains a frontier of candidates that are best at something — best accuracy, best latency, best on a specific task category.
  • Generalization mode. When a validation set is provided, GEPA evaluates candidates on held-out examples and uses validation performance for selection, preventing overfitting to the training set.

When to use GEPA: Complex system prompts with competing objectives. Prompts that need to generalize to unseen inputs. Situations where you want the optimizer to explain its changes in natural language.

Cross-reference: GEPA's approach to system-level optimization complements the evaluation playbooks in Chapter 7, §7.6.

8.3.3 LLM-Evolution Frameworks

A family of frameworks applies evolutionary search to code and algorithms:

  • AlphaEvolve (Google DeepMind): Uses Gemini to evolve code solutions, with an "evolutionary controller" that maintains a population of programs and selects parents for crossover and mutation.
  • OpenEvolve and ShinkaEvolve: Open-source implementations of similar evolutionary approaches, enabling community experimentation.

These frameworks excel at single-task search — finding novel algorithms for hard mathematical or computational problems. They have produced state-of-the-art results in circle packing, matrix multiplication, and scheduling.

When to use: You have a hard optimization problem with a clear, automated evaluator. The search space is code (Python, CUDA). You have significant compute budget and the problem justifies it.

8.3.4 optimize_anything: Unified API

The optimize_anything API provides a declarative interface that selects the right optimization mode based on what you provide:

import gepa

result = gepa.optimize_anything(
    seed="You are an HR leave assistant. Help employees...",  # Initial candidate
    evaluator=evaluate_leave_agent,                           # Score + ASI function
    dataset=train_examples,                                   # Optional: enables multi-task
    valset=val_examples,                                      # Optional: enables generalization
    objective="Optimize system prompt for policy accuracy, "
              "tool selection, and safety compliance",
    n_iterations=50,
)

print(result.best_candidate)       # The optimized prompt
print(result.best_score)           # Validation score
print(result.pareto_frontier)      # All Pareto-optimal candidates
print(result.changelog)            # Natural-language summary of changes
Inputs Provided Mode Selected Behaviour
seed + evaluator Single-task search Optimize one candidate for one evaluator
seed + evaluator + dataset Multi-task search Optimize across examples, transfer insights
seed + evaluator + dataset + valset Generalization Train on dataset, select on valset

8.4 Trajectory Optimization

8.4.1 Beyond Final Outcomes

Standard evaluation asks: "Did the agent produce the right answer?" Trajectory optimization asks: "Did the agent take the right path to get there?"

An agent that reaches the correct leave calculation by calling tools in a random order, retrying three times, and ignoring a policy document is fragile — it may produce the right answer on this input but fail on the next. Optimizing trajectories produces agents that are correct and robust.

8.4.2 Trajectory Accuracy Measurement

Trajectory evaluation compares the actual sequence of actions to an expected sequence. Several matching strategies exist:

Strategy Description When to Use
Strict match Exact sequence equality Safety-critical: order matters absolutely
Unordered match Same actions, any order Independent sub-tasks
Subset match Expected actions present (extras allowed) Agent may add helpful steps
Superset match Agent actions are subset of expected Agent should not take extra actions

For approximate matching, distance metrics from time-series analysis apply:

  • Edit Distance on Real sequences (EDR): Counts insertions, deletions, and substitutions needed to transform actual into expected.
  • Longest Common Subsequence (LCSS): Measures the longest shared subsequence, tolerating insertions.
  • LLM-as-judge: For complex trajectories where structural matching is insufficient, an LLM judge can assess whether the trajectory is "reasonable" given the task — accounting for valid alternative paths.

8.4.3 Optimizing Multi-Tool Sequences

Trajectory optimization feeds trajectory scores and ASI back into the optimization loop. The evaluator reports not just "wrong answer" but "called submit_leave before check_balance" or "skipped lookup_policy entirely." The optimizer uses this to adjust the system prompt's instructions about tool ordering, add explicit sequencing constraints, or restructure the scaffold.

8.4.4 Connection to Observability

Production traces are a goldmine for trajectory optimization. Every agent run in production generates a trace with tool calls, timings, and outcomes. By comparing production traces to golden trajectories, you can:

  1. Identify systematic trajectory deviations (the agent consistently skips a step).
  2. Add deviating traces to the optimization dataset.
  3. Re-optimize with the expanded dataset.

Cross-reference: Trace infrastructure and span-level observability are covered in Chapter 11, §11.4. Production replay for debugging is in Chapter 11, §11.4.5.

8.5 Agent Architecture Discovery

8.5.1 Evolving Complete Agent Systems

The most ambitious form of optimization: evolving the agent's architecture itself.

The ARC-AGI case demonstrates the potential. Starting from a 10-line Python stub that scored near zero, an evolutionary optimizer discovered a 300-line multi-stage pipeline — with input parsing, pattern recognition sub-routines, validation loops, and output formatting — that achieved competitive scores on abstract reasoning tasks. No human designed the pipeline; the optimizer discovered it through mutation, evaluation, and selection.

8.5.2 What Can Be Discovered

Architecture discovery can evolve:

  • Sub-agent orchestration patterns. Manager-worker topologies, routing strategies, specialization boundaries.
  • Retry and fallback strategies. When to retry, when to fall back to a simpler model, when to escalate.
  • Validation and self-correction loops. Critic steps, output verification, confidence thresholds for re-generation.
  • Skill specialization and routing. Which sub-tasks benefit from specialized prompts vs a general-purpose prompt.

8.5.3 Practical Constraints

Architecture discovery is powerful but expensive:

  • Compute budget. Each candidate architecture requires full evaluation across the dataset. Budget 10–100x the cost of prompt-only optimization.
  • Interpretability. Evolved architectures may be opaque. Require that evolved code includes comments and follows coding standards (enforce this in the optimizer's mutation prompt).
  • Overfitting risk. Complex architectures overfit more easily than simple prompts. Always use generalization mode with a held-out validation set.

8.5.4 When to Evolve vs When to Design

Factor Favour Evolution Favour Design
Problem novelty High — unknown optimal patterns Low — established solutions exist
Eval signal quality Strong, fully automated Weak, requires human judgement
Compute budget Generous Constrained
Interpretability requirement Flexible Strict (regulated domain)
Time to ship Weeks Days
Team expertise Deep eval infrastructure Deep domain knowledge

Rule of thumb: Design first, evolve second. Start with a hand-crafted scaffold (Chapter 4). Optimize the prompts and tool descriptions (§8.3). Only reach for architecture discovery when prompt-level optimization plateaus and you have the eval infrastructure and compute budget to support it.

Cross-reference: Scaffold types that serve as starting points for architecture discovery are covered in Chapter 4, §4.2.

8.6 Skill Optimization

8.6.1 Skills as Text Artifacts

Skills — repository-specific instructions, domain best practices, coding conventions, policy summaries — are text artifacts that can be optimized just like prompts.

The difference: skills are designed to transfer. A prompt is optimized for a specific agent on a specific task. A skill is optimized to improve performance across a family of related tasks, potentially across different models.

8.6.2 Cross-Model Transfer

Skills optimized on a smaller, faster model often transfer to larger models with minimal degradation — and sometimes improvement. This enables a practical workflow:

  1. Optimize skills on a fast, cheap model (e.g., Haiku) with many iterations.
  2. Validate on the target model (e.g., Sonnet or Opus) with fewer, more expensive evaluations.
  3. Deploy the skill with the target model.

This works because skills encode domain knowledge (what the leave policy says, how the API behaves, what coding conventions to follow) rather than model-specific formatting tricks.

8.6.3 The Coding Agent Case Study

In a published case study, skill optimization for a coding agent improved pass rates from 79% to 100% on a benchmark suite, with 47% faster resolution times. The process:

  1. Seed skills: Hand-written instructions ("always run tests before submitting," "check for edge cases in input validation").
  2. Evaluation: Run the agent on a suite of coding tasks, measuring correctness and resolution time.
  3. Optimization: GEPA evolved the skills over 30 iterations, discovering non-obvious instructions like specific debugging strategies and error-recovery patterns.
  4. Validation: Tested on held-out tasks to confirm generalization.

The optimized skills were interpretable — engineers could read them and understand why they helped. Several were adopted as team coding standards.

8.7 Production Systems Optimization

8.7.1 Scheduling Policies

Cloud workload scheduling involves deciding which instances to run on spot (cheap, interruptible) vs on-demand (expensive, reliable). A scheduling policy encoded as a Python function can be optimized:

  • Seed: A simple heuristic ("use spot for jobs under 30 minutes, on-demand for the rest").
  • Evaluator: Run simulated workloads, measure total cost and deadline violations.
  • Optimization: Evolve the policy function, discovering nuanced rules about job priority, time-of-day pricing, and interruption probability.

The "Can't Be Late" case study demonstrated 15% cost reduction with zero increase in deadline violations by optimizing a scheduling policy with GEPA in generalization mode.

8.7.2 Routing Algorithms

Multi-cloud data transfer requires routing decisions: which path to use, when to split traffic, how to handle congestion. The CloudCast case optimized broadcast routing strategies:

  • Seed: Shortest-path routing with equal traffic splits.
  • Dataset: Historical traffic patterns across 12 data centre configurations.
  • Evaluator: Network simulator measuring throughput, latency, and cost.
  • Result: Evolved routing strategy outperformed the hand-tuned baseline by 22% on throughput while reducing cost.

8.7.3 Generalization Across Configurations

Production systems must generalize: you train on configurations A, B, and C but deploy on configurations A through Z. This is why generalization mode matters — optimizing a scheduling policy that only works on the training configurations is useless.

The pattern: split your configurations into training and validation sets. Optimize on training. Select the best candidate based on validation. Deploy with confidence that the policy generalizes.

8.8 Multi-Objective and Pareto Optimization

8.8.1 Competing Objectives in Agents

Agents rarely have a single objective. Real-world optimization must balance:

  • Accuracy vs latency. A more thorough agent takes longer.
  • Safety vs helpfulness. Stricter guardrails reduce false positives but increase false negatives (refusing valid requests).
  • Cost vs quality. More capable models and longer reasoning chains cost more.
  • Precision vs recall. In retrieval: missing a relevant document vs including irrelevant ones.

Blending these into a single weighted score loses information. A candidate that is fastest but least accurate disappears if you weight accuracy heavily — but that candidate might reveal a useful architectural insight.

8.8.2 Pareto Frontier Maintenance

A Pareto frontier is the set of candidates where no candidate is strictly better than another on all objectives. GEPA maintains this frontier throughout optimization:

  1. Each new candidate is compared against the frontier.
  2. If it dominates any frontier member (better on all objectives), the dominated member is removed.
  3. If it is dominated by any frontier member, it is discarded.
  4. If it is non-dominated (better on some objectives, worse on others), it is added to the frontier.

At the end of optimization, you choose from the frontier based on your deployment priorities.

8.8.3 Minibatch Reflection

GEPA's minibatch strategy is crucial for multi-objective optimization. Each iteration samples 2–3 examples and reflects on failures specific to those examples. This produces focused mutations: "improve policy citation for parental leave cases" rather than "improve everything."

Over many iterations, the focused mutations accumulate into broad improvement — without the dilution that comes from reflecting on dozens of failures simultaneously.

8.8.4 Selection Strategies

Two strategies for selecting from the Pareto frontier:

  • Exploit: Choose the candidate closest to your target trade-off point. Deploy the prompt that gives 95% accuracy within the 3-second latency budget.
  • Explore: Choose the candidate in the least-explored region of the frontier. Use this during early optimization when you are still mapping the trade-off landscape.

8.9 Optimization Infrastructure

8.9.1 Evaluation Caching

The most expensive part of optimization is evaluation. A single optimization run may evaluate hundreds of (candidate, example) pairs. Many evaluations are redundant — the same candidate evaluated on the same example produces the same result.

An evaluation cache, keyed on (candidate_hash, example_hash), eliminates redundant evaluations. This is especially valuable when:

  • The optimizer revisits candidates from the Pareto frontier.
  • Multiple optimization runs share the same dataset.
  • You resume a checkpointed run.

8.9.2 Parallelization

Evaluations across examples are independent and can run concurrently. Evaluations across candidates within the same generation are also independent. A well-structured optimization infrastructure parallelizes both:

  • Within-candidate parallelism: Evaluate one candidate on N examples simultaneously.
  • Cross-candidate parallelism: Evaluate M candidates concurrently (when the algorithm supports it — evolutionary approaches naturally do).

8.9.3 Checkpointing and Resumption

Optimization runs can take hours. Infrastructure must support:

  • Periodic checkpoints saving the current population, Pareto frontier, evaluation cache, and iteration count.
  • Resumption from any checkpoint, continuing with full state restored.
  • Early stopping when improvement plateaus — measured as no Pareto frontier change for N consecutive iterations.

8.9.4 Experiment Tracking

Each optimization run produces artifacts that must be logged:

  • Candidates: Every candidate text, with parent lineage.
  • Scores: Per-example and aggregate scores for each candidate.
  • ASI: Diagnostic information for each evaluation.
  • Reflections: The optimizer's natural-language reasoning for each mutation.
  • Frontier snapshots: The Pareto frontier at each iteration.

8.9.5 CI/CD Integration

Optimization fits naturally into a CI/CD pipeline:

┌──────────────┐    ┌──────────────┐    ┌──────────────┐    ┌──────────────┐
│   Golden     │───▶│   Optimize   │───▶│   Validate   │───▶│   Deploy     │
│   Set        │    │   (GEPA)     │    │   (Holdout)  │    │   (Prod)     │
└──────────────┘    └──────────────┘    └──────────────┘    └──────────────┘
       │                   │                   │                   │
       ▼                   ▼                   ▼                   ▼
   Curated             Candidates          Gate check          Canary +
   examples            + ASI logs          + regression        rollout

The pipeline: curate a golden set → run optimization → validate on a held-out set → gate on regression thresholds → deploy via canary rollout. This is the same evaluate-then-ship pattern from Chapters 6–7, with optimization inserted as the improvement step.

Cross-reference: Deployment gates and canary rollout patterns are covered in Chapter 11, §11.3.

8.10 Optimization Playbook: HR Leave Agent

This section walks through a complete optimization of the HR Leave Agent, demonstrating each technique on the handbook's running example.

8.10.1 Prompt Optimization

Goal: Improve policy citation accuracy from 72% to 90%+.

Seed prompt: The initial system prompt from Chapter 4, instructing the agent to help employees with leave requests by consulting policy documents.

Evaluator: Checks whether the agent cites the correct policy sections for each leave type. Returns ASI including missing citations, incorrect citations, and the retrieved policy chunks.

Optimization run (GEPA, generalization mode):

  • Dataset: 40 leave request scenarios across 6 leave types (annual, sick, parental, bereavement, unpaid, study).
  • Validation set: 15 held-out scenarios, including edge cases (overlapping leave types, policy exceptions).
  • Iterations: 30.
  • Result: Policy citation accuracy improved from 72% to 94% on validation.

What the optimizer discovered: The seed prompt said "consult the leave policy." The optimized prompt specified how to consult it: "For each leave type mentioned in the request, retrieve the specific policy section by leave-type code. Quote the section number in your response. If multiple leave types apply, address each separately."

8.10.2 Trajectory Optimization

Goal: Reduce unnecessary tool calls from an average of 8 per request to 5 or fewer.

Approach: Added trajectory scoring to the evaluator. Each evaluation returns the actual tool sequence and compares it to the expected minimal sequence using EDR (Edit Distance on Real sequences).

Result: Average tool calls dropped from 8.1 to 4.7. The optimizer added explicit instructions: "Do not call check_balance until you have confirmed the leave type. Do not call submit_leave until the employee has approved the draft."

8.10.3 Multi-Tenant Skill Learning

Goal: Learn tenant-specific instructions that generalize to new employees within each tenant.

Approach: Each tenant has different leave policies (public sector vs private sector, different countries). Rather than maintaining separate prompts per tenant, optimize a skill — a set of instructions that gets injected alongside the base prompt based on the tenant.

  • Dataset: 20 scenarios per tenant, 4 tenants.
  • Validation set: 5 new scenarios per tenant.
  • Result: Tenant-specific skills improved accuracy by 12 percentage points on average over the generic prompt, and generalized to new employees within each tenant.

8.10.4 Production-Aware Optimization

Goal: Maintain 90%+ accuracy while meeting the 3-second latency SLO and staying under $0.02 per request.

Approach: Multi-objective optimization with three objectives: accuracy, latency, and cost. The evaluator returns all three as separate scores. GEPA maintains a Pareto frontier across all three dimensions.

Before optimization:

Metric Value
Policy accuracy 72%
Avg latency 4,200ms
Avg cost/request $0.028

After optimization (selected from Pareto frontier):

Metric Value
Policy accuracy 91%
Avg latency 2,800ms
Avg cost/request $0.019

The optimizer achieved this by tightening the prompt (reducing unnecessary reasoning tokens), reordering tool calls (eliminating redundant lookups), and adding early-exit conditions (skip balance check if leave type is invalid).

Cross-reference: The complete HR Leave Agent reference, including the artifact set used in this optimization, is in Appendix B. The production walkthrough is in Chapter 11, §11.A.

8.11 Anti-Patterns and Pitfalls

8.11.1 Overfitting to the Golden Set

Symptom: Optimized prompt scores 98% on your golden set but drops to 75% on real traffic.

Cause: The golden set is too small or too uniform. The optimizer learned to handle your specific examples rather than the general task.

Fix: Always use generalization mode with a held-out validation set. Periodically refresh the golden set with production examples. Monitor production metrics after deploying optimized artifacts.

8.11.2 Reward Hacking

Symptom: The optimized prompt achieves high scores by exploiting evaluator weaknesses rather than genuinely improving.

Cause: The evaluator has blind spots. For example, a citation checker that only verifies section numbers (not content) can be gamed by an optimized prompt that cites every section.

Fix: Use multi-dimensional evaluation. Combine deterministic checks with LLM-as-judge evaluation. Review optimized candidates manually before deployment. Track evaluator-score vs human-assessment correlation.

8.11.3 Catastrophic Forgetting

Symptom: Optimization for one capability (policy citation) regresses another (tone and empathy).

Cause: The optimizer focuses on the measured objective and neglects unmeasured qualities.

Fix: Include all critical dimensions in the evaluator. Use Pareto optimization so that improvements on one dimension do not sacrifice others. Add regression tests for previously-solved capabilities to the golden set.

8.11.4 Compute Spirals

Symptom: Each optimization run costs more than the last as you add more examples, more objectives, and more iterations.

Cause: Unbounded optimization scope.

Fix: Set compute budgets upfront. Use evaluation caching aggressively. Start with small datasets and fast models for initial exploration; scale up only for final optimization. Track cost-per-improvement-point and stop when marginal returns diminish.

8.12 Summary

Optimization is the bridge between knowing what's wrong (evaluation) and making it better (improvement). The key ideas:

  1. Three modes — single-task search, multi-task search, and generalization — serve different needs. For agent prompts, generalization is almost always the right choice.
  2. Actionable Side Information transforms blind search into diagnostic-driven improvement. Design your evaluators to return rich feedback, not just scores.
  3. Everything is optimizable — prompts, tool schemas, trajectories, architectures, skills, and production configurations. Start with prompts; graduate to trajectories and architectures as your eval infrastructure matures.
  4. Pareto-efficient search handles competing objectives without collapsing them into a single score. Choose from the frontier based on deployment priorities.
  5. Infrastructure matters — evaluation caching, parallelization, checkpointing, and experiment tracking make the difference between optimization as a one-off experiment and optimization as a continuous capability.
  6. Guard against anti-patterns — overfitting, reward hacking, catastrophic forgetting, and compute spirals. The fix for all of them is better evaluation and held-out validation.

Quick-Reference Tables

When to Use Each Mode

Situation Mode Example
One hard problem Single-Task Circle packing, blackbox optimization
Batch of related problems Multi-Task CUDA kernels, API adapters
Must generalize to unseen Generalization Agent prompts, scheduling policies

Optimization Ladder

Level Approach When to Use
1 Manual prompt iteration Starting point, low volume
2 Few-shot example optimization Structured programs, DSPy
3 Prompt optimization System instructions, GEPA
4 Trajectory optimization Tool selection, sequencing
5 Architecture discovery Novel agent designs, high compute budget

Algorithm Selection

Algorithm Best For Compute Cost Generalization
DSPy MIPROv2 Few-shot examples, module pipelines Low Yes
GEPA Complex prompts, competing objectives Medium Yes
optimize_anything Any text artifact Medium–High Configurable
AlphaEvolve Code and algorithms High No (single-task)

Annotated References

Topic Sources
GEPA algorithm arXiv:2507.19457
optimize_anything gepa-ai.github.io/gepa
DSPy optimizers dspy.ai/learn/optimization
AgentEvals (trajectory) github.com/langchain-ai/agentevals
Phoenix trajectory evals arize.com/phoenix
AlphaEvolve deepmind.google/discover/blog/alphaevolve

Key Takeaways

  1. Evaluation is the foundation, optimization is the flywheel. You cannot optimize what you cannot measure. The evaluation infrastructure from Chapters 6–7 becomes the fitness function that powers continuous improvement.
  2. ASI is the difference between blind search and guided search. Invest in evaluators that return structured diagnostics — error traces, missing citations, tool sequence deviations — not just scalar scores.
  3. Match the mode to the need. Single-task for hard one-off problems, multi-task for batches, generalization for anything that must work on unseen inputs. Agent prompts almost always need generalization.
  4. Start with prompts, graduate to architectures. Prompt optimization is cheap and high-impact. Trajectory optimization catches process failures. Architecture discovery is powerful but expensive — earn it through evidence that simpler approaches have plateaued.
  5. Pareto beats blending. When objectives compete, maintain a frontier of trade-off candidates rather than collapsing to a single weighted score. Let deployment context drive final selection.
  6. Guard the validation set. Overfitting to the golden set is the most common optimization failure. Always hold out data. Refresh with production examples. Monitor post-deployment.
  7. Optimization is a pipeline stage, not a one-off. Wire it into CI/CD: golden set → optimize → validate → gate → deploy. Every incident that adds examples to the golden set triggers a re-optimization opportunity.

Next → Chapter 9 provides the builder's catalogue of reusable design patterns for prompts, tools, context, memory, and orchestration — many of which serve as the building blocks that optimization improves.