Appendix I: Glossary
Version: 1.0 Last updated: February 2026
Navigation: ← Appendix H | Appendix Index
Key terms used throughout this handbook.
A2A (Agent-to-Agent): Communication protocol between agents. One agent delegates work to another and integrates the result.
AgentBundle: The versioned, deployable unit. It contains model config, prompts, tool contracts, policy configs, and memory rules.
Autonomy level: The degree of independent action granted to an agent. Ranges from advisory (suggest only) through execute-with-approval to fully autonomous.
Composite judge: An evaluation pipeline that combines multiple individual judges (e.g., eligibility, safety, citation, quality) into a weighted overall score, with non-negotiable gates that override the aggregate.
Context engineering: The practice of assembling the right information for each agent step. Includes retrieval, filtering, and budget management.
Critic prompt: A verification prompt that an agent uses to self-check its own output before committing an action. Catches errors before they reach the user.
Evaluation harness: A test framework that runs scenarios against an agent. It captures traces and applies graders to assess outcomes.
Evidence bundle: The proof package that justifies shipping an AgentBundle. Includes harness results, red-team outcomes, and approvals.
Golden set: A curated, labeled dataset used as the primary benchmark for agent evaluation.
Grader/Judge: A component that scores agent outputs. Can be deterministic rules, LLM-as-judge, or human reviewers.
Idempotency key: A unique identifier attached to a write operation so that retrying the same operation produces the same result rather than duplicating it. Essential for safe tool calls.
HITL (Human-in-the-loop): The highest oversight level. A human approves every consequential agent action before execution.
HOTL (Human-on-the-loop): Medium oversight level. Humans review samples and exceptions. Routine actions proceed automatically.
HOOTL (Human-out-of-the-loop): Lowest oversight level. Agents operate autonomously. Humans monitor dashboards and intervene only on alerts.
Inner loop: The step-by-step execution within a single agent run. This is scaffold behavior.
MCP (Model Context Protocol): Anthropic's open standard for tool and resource integration with LLMs.
Non-negotiable: A threshold that must be met for an agent to ship. Violations block release regardless of other metrics.
Outer loop: The improvement cycle across runs. Includes evaluation, learning, and deployment.
Orchestrator: The deterministic workflow controller. It manages long-running processes, approvals, and state transitions.
Preview→Commit: A UX pattern where the agent shows proposed changes for approval before executing them.
Regression suite: A collection of test cases that must continue to pass as the agent evolves. Includes cases added from past failures.
Rubric: A structured scoring guide that defines evaluation criteria, scales, and pass/fail thresholds. Used by LLM judges and human reviewers to assess agent outputs consistently.
Runtime: The execution layer. It provides tool sandboxing, auth, state management, and observability.
Scaffold: The reusable inner-loop behavior pattern. Includes prompts, tool routing, branching logic, and stopping conditions.
Synthetic dataset: A programmatically generated collection of test cases designed to cover specific dimensions (leave types, edge cases, adversarial scenarios) that may be rare in production data.
Skill: A reusable capability with explicit input/output contracts, tool allowlists, and policy scope.
Tolerance: A metric threshold that is tracked but does not block release. Used for quality dimensions that can degrade gracefully.
Trace: The complete record of an agent run. Includes inputs, steps, tool calls, outputs, and decisions.
Navigation: ← Appendix H | Appendix Index