Chapter 2 — Core Concepts of Agentic AI

TL;DR

An agent is an LLM wrapped in an observe-plan-act-reflect loop with tools, memory, and a scaffold that structures each step — not a chatbot that "thinks."
Tools are contracts: narrow, schema-validated functions that define the boundary of what an agent is allowed to do, and where safety enforcement (RBAC, audit, rate limits) lives.
Memory stores state; context engineering selects what the model actually sees — assembling the right retrieved docs, history, policies, and scratchpad notes into a token-budgeted prompt is a first-class design problem.
Autonomy should be earned in levels (0-4), matched with the right human oversight mode (HITL / HOTL / HOOTL), and validated by inner-loop traces and outer-loop evaluation harnesses before you increase it.
Every failure mode — hallucination-with-action, infinite loops, prompt injection, tool misuse — has concrete mitigations rooted in the same building blocks: schemas, guardrails, step caps, and evaluation as a continuous practice.

This chapter gives you a shared mental model for agentic AI systems. Think of them not as "LLMs that think," but as software systems with explicit control flow, state, interfaces, safety boundaries, and feedback loops.

If you can explain the ideas here clearly, you'll be able to design agents that are easier to build, debug, evaluate, and operate.

2.0 From LLMs to Agentic Systems

Most people meet large language models (LLMs) as a chatbot. You type a prompt. It replies. Done.

For builders, that's the starting point, not the destination.

In production, you almost never ship a raw model. You ship an augmented LLM system:

LLM + tools + data + memory + agent runtime + scaffold

Here's what each piece does. The LLM is a stochastic reasoning engine. It's great at language and fuzzy reasoning, but not great at guarantees. Tools are external functions the system can call, like APIs, database queries, or code runners. Data means your knowledge sources: documents, warehouses, and feature stores. Memory holds state across steps or sessions. The agent runtime is the execution layer. It handles tool sandboxing, auth/RBAC, state management, context compaction, and retries. The scaffold is the reusable inner-loop pattern that defines behaviour logic, prompts, and tool routing decisions.

An agentic AI application is one where this augmented system can form sub-goals, take a sequence of actions over time, observe the effects, and adapt the plan based on what happened.

Visual: Evolution from LLM app to RAG to agentic system

flowchart LR
  subgraph A["Plain LLM App"]
    direction LR
    A_U[User] --> A_P[Prompt Template] --> A_L[LLM] --> A_R[Response]
  end

  subgraph B["RAG App"]
    direction LR
    B_U[User] --> B_Q[Query] --> B_RET[Retriever] --> B_C[Prompt + Retrieved Docs] --> B_L[LLM] --> B_R[Response]
  end

  subgraph C["Agentic System"]
    direction LR

    C_U[User] --> C_AGENT[Agent]

    subgraph C_AGENT_BOX["Agent (internal)"]
      direction TB
      C_L3[LLM]
      C_M[Memory]
      C_SCAF[Scaffold]
      C_TOOLS[Tools]
    end

    C_AGENT <--> C_ENV["Environment<br/>(APIs, DBs, UIs)"]
    C_AGENT --> C_OUT[Actions & Responses]
  end

  A_R --> B_U
  B_R --> C_U

Figure 2.1 — Evolution from plain LLM app to RAG app to full agentic system.

2.1 Agent vs App vs Workflow (and A2A)

People use these words interchangeably. Don't. They're different layers.

2.1.1 LLM App

A simple LLM app follows this pattern:

User → prompt template → LLM → response

Sometimes it includes retrieval:

User → retrieve docs → prompt + docs → LLM → response

This can be very useful. But there's no explicit notion of time, tasks, or actions beyond a single response.

2.1.2 Agent

An agent is an augmented LLM that has a goal, operates in an environment, takes actions through tool calls, receives observations from tool outputs and errors, and updates state until it's done or escalates.

A simple agent loop looks like this:

state = init_state(goal, context)

while not done(state):
    observation = get_observation(state)
    thought, action = LLM.decide(state, observation)
    result = environment.execute(action)
    state = update_state(state, thought, action, result)

2.1.3 Workflow

A workflow is the deterministic structure that surrounds the agent. It includes orchestration through state machines, BPMN, Step Functions, or Airflow/Argo. It also includes business rules, retry policies, SLAs and timeouts, plus approvals and routing.

The agent decides what to do next. The workflow decides what's allowed and how execution proceeds.

Orchestrator / Controller (one-liner): The orchestrator (or controller) is the logic that advances the workflow at each step, deciding which agent or tool to call next, when to wait for humans, and when to stop.

When you say "orchestrator," be explicit about which layer you mean. The workflow orchestrator schedules steps, approvals, SLAs, and retries. The scaffold controller handles inner-loop step logic—which tool or model call happens next within the scaffold.

flowchart TB
    subgraph WF[Workflow]
        direction TB
        subgraph AG[Agent]
            direction TB
            LLM[LLM]
        end
    end

Figure 2.2 — Workflow (deterministic shell), Agent (goal-driven control), LLM (reasoning core).

2.1.4 Agent-to-Agent (A2A): Multi-agent collaboration

As systems grow, one agent often shouldn't do everything. You'll break work into specialist agents like a Policy Interpreter, RAG Researcher, or Action Executor. A supervisor coordinates them.

A2A (one-liner): A2A (agent-to-agent) is communication and delegation between agents—where one agent requests work from another agent and integrates the result—often with explicit contracts and shared context.

Why use A2A? First, it enables separation of concerns. You can keep "thinking" separate from "acting." Second, it allows different policies and permissions. A "Write Agent" can be tightly locked down. Third, you can use different models. Use a stronger model for planning and a cheaper one for extraction. Fourth, it enables scalability. You can parallelise work across research, summarisation, and verification.

Visual: Supervisor delegating to specialist agents (A2A)

flowchart TB
    U[User] --> SA[Supervisor Agent]
    SA --> PA[Policy Agent<br/>interprets rules]
    SA --> RA[RAG Agent<br/>retrieves & cites]
    SA --> EA[Executor Agent<br/>performs actions]
    PA --> SA
    RA --> SA
    EA --> SA
    SA --> OUT[Final Answer / Action Outcome]

Figure 2.3 — A2A pattern: a supervisor agent delegates to specialist agents and composes the result.

2.2 Environment and Tools (and MCP)

An agent becomes useful when it can act on the world.

2.2.1 What is the Environment?

For most software and data teams, the environment includes APIs and services like HR, ticketing, CRM, and ERP. It includes databases and warehouses. It includes knowledge stores such as document repositories and vector databases. It includes external services like email and Slack/Teams. Sometimes it even includes legacy UIs accessed via RPA or browser automation.

Visual: Agent in its environment

flowchart TB
  AG["Agent<br/>(LLM + Memory + Control)"]
  DB[("Database")]
  KB[("Knowledge Store / Vector DB")]
  API["Business APIs<br/>(HR / ERP / CRM)"]
  MSG["Messaging<br/>(Email / Slack)"]
  UI["Legacy UI / RPA"]

  AG -->|"tool calls"| DB
  DB -->|"results"| AG

  AG -->|"retrieve"| KB
  KB -->|"snippets"| AG

  AG -->|"API calls"| API
  API -->|"responses"| AG

  AG -->|"send/notify"| MSG
  MSG -->|"delivery status"| AG

  AG -->|"automation"| UI
  UI -->|"observations"| AG

Figure 2.4 — The agent interacts with the environment via tool calls and observations.

2.2.2 Tools as Contracts

A tool is any external function the agent can call:

def get_leave_balance(employee_id: str) -> LeaveBalance: ...
def submit_purchase_request(payload: PurchaseRequest) -> RequestId: ...
def search_knowledge_base(query: str) -> list[Document]: ...

When you expose tools to an agent, you're defining the boundary of power:

"Here are the things you are allowed to do. Do not improvise beyond this."

Keep these builder instincts in mind. Tools should be narrow and well-specified. Tools should validate inputs and handle errors robustly. Tools are where RBAC, audit logging, and rate limits belong.

Schema (one-liner): Schemas (for example JSON Schema) give a machine-readable shape to tool inputs and LLM outputs so you can validate them before they reach downstream systems.

Tool boundary = safety boundary

If you do nothing else, validate tool inputs and constrain tool side-effects. This is your strongest line of defense against "hallucination with action".

2.2.3 MCP (Model Context Protocol): Standardising tool and context integration

As your toolset grows, the hard part becomes integration sprawl. Every agent framework and app needs connectors, auth handling, schemas, and upgrades.

MCP (one-liner): MCP (Model Context Protocol) is a standard way to expose tools and context sources via an MCP server so agent runtimes can discover and call them consistently.

Think of MCP as tool servers with a standard interface. It creates a clean boundary between agent logic and enterprise integrations.

MCP brings several benefits for builders. You get reusable connectors across many agents and apps. You get centralised security including auth, allow-lists, auditing, and rate limits. You get faster iteration because you can swap integrations without rewriting agent code. You get better ops through versioning and consistent tool schemas.

Visual: Agent runtime talking to MCP servers

flowchart LR
  subgraph RT["Agent Runtime"]
    RT_AG["Agent<br/>(LLM + Scaffold + Context Builder)"]
    RT_ORCH[Orchestrator]
    RT_AG <--> RT_ORCH
  end

  subgraph MCP1["MCP Server: Knowledge"]
    MCP1_KB["Search / Retrieve / Docs"]
  end

  subgraph MCP2["MCP Server: Collaboration"]
    MCP2_SL["Slack / Teams / Email"]
  end

  subgraph MCP3["MCP Server: Enterprise Systems"]
    MCP3_ERP["HR / ERP / Ticketing"]
  end

  RT_ORCH -->|discover/call tools| MCP1_KB
  RT_ORCH -->|discover/call tools| MCP2_SL
  RT_ORCH -->|discover/call tools| MCP3_ERP

Figure 2.5 — MCP servers provide a standardised interface for tools and context sources.

2.2.4 Agent runtime: Executing scaffolds safely

If the scaffold is the reusable inner-loop behaviour pattern, the agent runtime is the execution layer that runs that scaffold safely and consistently.

Agent runtime (one-liner):
The agent runtime is the execution layer that safely runs scaffolds. It manages tool sandboxing, auth/RBAC, state, context construction and compaction, retries, and observability.

The runtime typically handles tool sandboxing and isolation. It handles auth/RBAC and credential management. It provides a state store for session state and checkpoints. It handles context building and compaction through summaries and retrieval limits. It provides retries, backoff, and idempotency helpers. It emits traces, spans, and telemetry.

Table: Integration choices — direct tools vs MCP vs A2A

Mechanism	What it is	Best for	Trade-offs
Direct tool integration	Your code calls APIs/SDKs directly as tools	Small systems, fastest iteration	Connector sprawl, duplicated auth/logging
MCP	Tools/context exposed via MCP servers	Standardised connectors, enterprise reuse	Requires MCP server ops/versioning discipline
A2A	Delegation to specialist agents	Complex tasks, separation of concerns	Needs agent contracts, traceability, coordination

Table 2.1 — Three ways to scale an agent's capabilities.

2.3 Goals, Tasks, and Policies

To make an agent feel less mystical, define these clearly.

2.3.1 Goals

A goal is the high-level outcome. Examples include "Process this leave request end-to-end," "Summarise and email the meeting notes," and "Analyse this tender and produce a compliance checklist."

2.3.2 Tasks

A task is a concrete step. For leave processing, the tasks might be:

Parse request.
Check leave balance.
Check policy constraints.
Decide approve/reject/escalate.
Update HR system.
Notify stakeholders.

Tasks might be fixed in workflow code, planned dynamically by the model, or a hybrid where the LLM proposes and the workflow validates.

2.3.3 Policy

A policy is the system's rules. This includes safety policies, privacy rules, business rules, and approval rules.

Guardrails (one-liner): Guardrails are the concrete mechanisms—filters, schema checks, allow/deny lists and safety rules—that enforce your policies on real inputs, actions and outputs.

Table: Goal vs Task vs Policy

Concept	Question it answers	Scope	Represented as	Example (leave agent)
Goal	What outcome do we want?	Whole run / episode	Text, workflow config	"Process this leave request end-to-end."
Task	What concrete step now?	Single step	State machine step, plan item	"Check blackout dates for requested range."
Policy	How do we choose next action?	All steps / all runs	Prompts + orchestration + rules	"If overlaps blackout dates → escalate."

Table 2.2 — Distinguishing goal, task, and policy.

2.4 The Core Agent Loop: Observe, Plan, Act, Reflect (and Scaffolds)

Almost every agent boils down to five steps: observe, think/plan, act with tools, reflect, and repeat until done or escalated.

Visual: Observe → Think → Act → Reflect

flowchart LR
    O[Observe<br/>Inputs, history, tool outputs, memory]
    T[Think / Plan<br/>Sub-goal, next step, tool choice]
    A[Act<br/>Call tools, APIs, DBs]
    R[Reflect<br/>Check success, errors, stop?]
    O --> T --> A --> R --> O

Figure 2.6 — The core agent loop.

2.4.1 Observe

Observation includes the latest user input, relevant conversation history, tool results and errors, and current state and memory.

2.4.2 Think / Plan

The system decides on the sub-goal, the next step, which tool (if any) to call, and whether it needs more info or human input.

Planner (one-liner): A planner is the module—often implemented with an LLM—that takes a goal and breaks it into ordered steps or sub-tasks the agent can execute.

2.4.3 Act (Tool Calls)

The agent executes actions. It might call an API, query or update a database, retrieve documents, send a message, or invoke another agent through A2A delegation.

2.4.4 Reflect

Reflection answers key questions. Did it work? Did we violate any rules? Should we retry, backtrack, ask the user, or escalate? Are we done?

Reflection needs a stopping condition

Without explicit stop rules (max steps, max retries, "done" criteria), agents can loop forever.

2.4.5 Scaffolds: Shaping How the Agent Thinks and Acts

A scaffold is the reusable inner-loop pattern. It includes behaviour logic, prompts, and tool routing decisions that wrap the LLM so it solves a task more reliably than a single prompt.

Common scaffold families include ReAct, which repeats "think → tool → observe." Plan-and-execute plans first, then executes steps. Spec-then-generate produces a spec or outline first, then generates final output. Critic-and-revise drafts, critiques, and revises.

Why scaffolds matter

Scaffolds improve reliability and debuggability by making intermediate steps visible and controllable.

2.4.6 Skills: Packaging Reusable Capabilities

As you build more agents, you'll notice the same chunks of work repeating. Things like summarise a document, extract entities, check policy compliance, draft a customer reply, or create a ticket update.

A skill is a named, reusable capability. It's often implemented as a prompt plus tools plus a mini-scaffold. You can plug it into different agents and workflows.

Skill (one-liner):
A skill is a reusable capability packaged as a small bundle (instructions + tool access + light orchestration) that agents can invoke consistently across use cases.

Think of the hierarchy like this. A tool is one function call, like "get_leave_balance." A skill is a small procedure, like "interpret leave policy for request X." An agent is a goal-driven controller that chooses skills and tools over time. A workflow is the deterministic shell that governs execution, approvals, and retries.

Table: Tool vs Skill vs Agent

Building block	What it is	Typical size	Example (leave agent)	How you test it
Tool	One external function	1 call	`get_leave_balance(employee_id)`	Unit + contract tests, schema validation
Skill	Reusable mini-procedure	2–10 steps	"Policy check for requested dates"	Scenario tests + judge checks
Agent	Goal-driven decision maker	10–100+ steps	"Process leave request end-to-end"	Harness runs across journeys
Workflow	Deterministic orchestration	Whole system	"Approvals + SLAs + escalation path"	Integration + reliability tests

Design each skill to be well-bounded so it does one job. Make it idempotent when possible so it's safe to retry. Give it a contract with schemas for inputs and outputs. Evaluate it with a small, repeatable test set.

Over time, a "skill library" becomes your real acceleration lever. New agents get built faster because you assemble proven skills rather than invent new behaviour every time.

flowchart TB
  WF[Workflow] --> AG[Agent]
  AG -->|invokes| SK1[Skill: Policy Check]
  AG -->|invokes| SK2[Skill: Draft Response]
  SK1 --> T1[Tool: Retrieve Policy]
  SK1 --> T2[Tool: Validate Dates]
  SK2 --> T3[Tool: Send Notification]

Figure 2.7 — Tools are primitives; skills are reusable bundles; agents choose; workflows govern.

2.5 Memory and Context (and Context Engineering)

LLMs have limited context windows. Agents run over many steps and sessions. Memory fills the gap.

2.5.1 Working Memory (Scratchpad)

Working memory is ephemeral memory within a single run. It holds intermediate reasoning notes, partial plans, and tool outputs cached for the next step.

2.5.2 Session / Episodic Memory

Session memory holds state for a conversation or workflow instance. It tracks steps completed, decisions made, errors encountered, and user clarifications.

{
  "user_id": "123",
  "current_goal": "process_leave_request",
  "history": [
    {"step": "parse_request", "status": "done"},
    {"step": "check_balance", "status": "done"},
    {"step": "check_policy", "status": "pending"}
  ]
}

2.5.3 Long-term Knowledge & Semantic Memory

Long-term knowledge often shows up as RAG: documents plus embeddings plus retrieval.

Semantic memory (one-liner): Semantic memory is the distilled layer of long-term memory that stores stable facts, inferred preferences and domain rules abstracted from many individual episodes.

Rerankers

Many production RAG systems add a reranker after retrieval for better relevance ordering. Chapter 9 covers practical design patterns including retrieval and context injection.

2.5.4 Memory is also a Risk

Persisted memory can store sensitive information like PII, secrets, and regulated data.

Design memory like a database. Define schemas. Set access controls. Configure TTL and retention policies. Support deletion on request.

2.5.5 Context Engineering: Designing What the Model Sees

Memory is what you could store. Context is what you actually pass to the model for a given decision.

Context engineering is assembling the right context, in the right structure, at the right time.

The model is (mostly) stateless. Its "world" at each call is whatever you put into the prompt.

Context may come from state and memory, retrieved documents, MCP-provided tools and resources, conversation history, and policy instructions.

2.5.5.1 Layers of context

A single LLM call might include system and policy instructions at the top, followed by the current task and goal, then relevant conversation history, then scratchpad state, then retrieved knowledge snippets, and finally tool descriptions and environment state.

flowchart TB
    SYS[System / Policy Instructions]
    TASK[Task & Goal Description]
    HIST[Relevant History]
    SCR[Scratchpad / Notes]
    DOCS[Retrieved Knowledge Snippets]
    TOOLS[Tool Descriptions & Env State]
    SYS --> TASK --> HIST --> SCR --> DOCS --> TOOLS

Figure 2.8 — Layers of context assembled into a single LLM call.

2.5.5.2 Budgeting and trade-offs

Context has budgets: tokens, latency, and cost.

You have several practical knobs to manage these. Summarise older history. Retrieve fewer but higher-quality chunks. Prefer structured state like JSON over long prose. Keep policy rules in a stable "sticky" section.

2.5.5.3 Context builders as first-class components

Treat context assembly as its own module:

context = build_context(
    goal=goal,
    state=agent_state,
    history=session_history,
    retrieved_docs=docs,
    tools=available_tools,
    policies=policy_specs,
)
response = call_llm(context)

This makes context recipes testable and optimisable.

Table: Memory types vs implementation and risk

Memory type	Lifetime	Typical implementation	What it stores	Risk considerations
Working / scratchpad	Single run	In-memory state, prompt text	Intermediate results, partial plans	Avoid secrets in logs
Session / episodic	Conversation/workflow	Redis/session DB	Steps, decisions, user prefs	Access control, TTL, deletion
Long-term knowledge	Months/years	Vector DB, KB, warehouse	Docs, policies, cases	PII, retention, freshness/correctness

Table 2.3 — Practical view of memory in agentic systems.

2.6 Levels of Autonomy and Human Oversight

Not every agent should act freely. Autonomy must match risk.

2.6.1 Autonomy levels (0–4)

Level 0 is insight only. The agent provides information but takes no actions. Level 1 is draft assistant. The agent drafts; the human executes. Level 2 is copilot with guardrails. Low-risk actions are allowed. Level 3 is semi-autonomous operator. The agent acts within limits and is monitored. Level 4 is fully autonomous in a narrow domain. It operates with tight constraints and heavy evaluation.

flowchart TB
  L0["Level 0 - Insight only"]
  L1["Level 1 - Draft assistant"]
  L2["Level 2 - Copilot with guardrails"]
  L3["Level 3 - Semi-autonomous operator"]
  L4["Level 4 - Fully autonomous (narrow)"]
  L0 --> L1 --> L2 --> L3 --> L4

Figure 2.9 — Autonomy ladder.

2.6.2 Human Oversight Modes: HITL, HOTL, HOOTL

Autonomy says "how much can it do." Oversight says "how do humans supervise."

In Human-in-the-loop (HITL), the agent proposes and the human approves before execution. In Human-on-the-loop (HOTL), the agent acts within limits while humans monitor and can intervene. In Human-out-of-the-loop (HOOTL), the agent acts autonomously in a tightly constrained, low-risk domain.

Choose oversight per action type

Read-only actions may be HOOTL. Write actions are often HOTL. High-impact actions involving money, legal, or compliance are usually HITL.

Table: Autonomy vs oversight

Autonomy level	Typical description	Example	Recommended oversight
0	Insight only	Analytics assistant	HOOTL
1	Draft assistant	Email/report drafting	HITL
2	Copilot w/ guardrails	Ticket tagging/routing	HOTL for writes; HOOTL for read-only
3	Semi-autonomous operator	Small refunds within limits	HOTL + sampled HITL
4	Fully autonomous (narrow)	Low-risk prioritisation	HOOTL + strong offline evaluation

Table 2.4 — A practical mapping from autonomy to oversight.

2.7 Inner Loops and Outer Loops (Evaluation + Evaluation Harnesses)

The inner loop is what happens per request or episode:

observe → think/plan → act → reflect → repeat → finish

It must be fast, reliable, and traceable.

Trace (one-liner): A trace is the structured log of a single run through a scaffold, capturing prompts, tool calls, intermediate state, outputs and timings.

Span (helpful observability add-on): A span is one step within a trace (e.g., "retrieve docs", "call LLM", "run safety check"), often nested to show hierarchy.

2.7.1 Outer loop (learning / ops)

The outer loop improves the system over time. You analyse traces, tune prompts and scaffolds, update tools and policies, refine retrieval, and redeploy safely.

Telemetry / Metrics (one-liner): Telemetry and metrics are the quantitative signals—latency, cost, success rate and policy-violation rate—that you monitor continuously to understand how agents behave in production.

2.7.2 Evaluation: Glue Between Inner and Outer Loops

Evaluation turns traces into judgement. It answers whether outputs are correct and grounded, whether actions are safe and policy-compliant, and whether behaviour improves over versions.

Evaluator / Judge / Critic (one-liner): An evaluator (or judge/critic) is a model or module that scores an agent's behaviour or outputs for qualities like correctness, faithfulness, style and safety.

Benchmark / Task suite (one-liner): A benchmark or task suite is a curated collection of test tasks you rerun over time to compare different models, scaffolds and configurations on the same ground.

Terminology: harness

Some literature (e.g., Anthropic) uses "agent harness" to mean what we call a scaffold plus runtime wrapper. In this handbook, we reserve "harness" for evaluation to avoid ambiguity.

Terminology: LLMOps vs AgentOps

LLMOps covers the model layer (prompt versioning, inference, token costs). AgentOps extends it to multi-step, tool-using systems (trace collection, tool monitoring, workflow state, action auditing, journey evaluation). AgentOps includes LLMOps. This handbook uses "ops" to mean the full AgentOps scope.

2.7.3 Evaluation Harnesses: Safe Places to Exercise Your Agents

If scaffolds shape the inner loop, harnesses shape the outer loop.

An evaluation harness is a test rig that feeds tasks to an agent, records what it does as traces, and measures performance, safety, and robustness.

Evaluation harnesses typically include task suites and scenarios, graders and judges, resettable environments or sandboxes, and aggregation and regression reporting.

Scenario / Journey (one-liner): A scenario or journey is a multi-step, end-to-end interaction that you replay in the evaluation harness to test agents under realistic conditions.

Sandbox / Simulator (practical note): Many evaluation harnesses run inside a sandbox with mock or test systems, or a simulator, to avoid real-world side effects.

Visual: Inner vs outer loop

flowchart LR
  subgraph INNER_LOOP["Inner Loop (per request)"]
    O[Observe] --> T["Think / Plan"] --> A[Act] --> R[Reflect] --> O
  end

  subgraph OUTER_LOOP["Outer Loop (over many requests)"]
    COL[Collect traces] --> EVAL[Evaluate] --> IMP["Improve prompts / tools / policies"] --> DEP[Deploy] --> COL
  end

  O -->|"Traces & metrics"| COL
  DEP -->|"Updated configs"| O

Figure 2.10 — Nested inner and outer loops with evaluation as the bridge.

Table: Inner vs outer loop responsibilities

Loop	Time scale	Owned by	Typical activities	Key artifacts
Inner	per request	runtime system	decide, call tools, recover from errors	traces, spans, per-run metrics
Outer	days–weeks	product/ML team	evaluate, red-team, tune, release	task suites, dashboards, versioned prompts/policies

Table 2.5 — Who does what, where.

2.8 Common Failure Modes in Agentic Systems

Agents fail differently from chatbots because they can act.

2.8.1 Hallucination with action

The agent invents facts and then uses them to act. It might update the wrong record or send the wrong email.

Mitigation: Validate at tool boundaries. Require approvals for high-impact actions.

2.8.2 Tool misuse and overuse

The agent uses the wrong tool, wrong parameters, or makes excessive calls. This causes cost blowups and rate limit hits.

Router (one-liner): A router is the decision logic that sends each request to the most appropriate model, tool or agent based on its content and context.

Mitigation: Use strong schemas, call budgets, routing policies, and error-aware retries.

2.8.3 Infinite or unbounded loops

The agent never decides it is done. It keeps replanning or retrying forever.

Mitigation: Set step caps, explicit stopping criteria, and "escalate after N failures" rules.

2.8.4 Prompt injection & data exfiltration

Untrusted content tries to override instructions or extract secrets.

Mitigation: Isolate data from instructions. Allow-list tools. Use content filters. Apply strict permissioning.

2.8.5 Misaligned objectives

The agent optimises the wrong thing. It might favour speed over correctness, or "looks confident" over truth.

Mitigation: Use better metrics, better test suites, and periodic human review.

Table: Failure modes and mitigations

Failure mode	What happens	Primary layer affected	Example	Mitigations
Hallucination with action	Makes up facts then acts	tools/environment	wrong account updated	schema validation, HITL for high-risk actions
Tool misuse/overuse	wrong/excessive tool calls	tools/cost	API hammered in a loop	budgets, router, retries with limits
Infinite loops	never stops	control loop	replans forever	max steps, stop rules, escalation
Injection/exfiltration	untrusted content hijacks	context	"ignore previous rules"	instruction/data separation, allow-lists
Misaligned objectives	optimises wrong target	outer loop	speed over correctness	better metrics, eval suites, review

Table 2.6 — Turn scary failure modes into concrete mitigations.

2.9 Governance Context

Governance in agentic systems is deeply technical. The same features that help you debug agents—tool contracts, traces, evaluation harnesses, and telemetry—also help your organisation operate safely and responsibly. Chapter 12 covers governance in detail.

2.10 A Builder's Checklist for Core Concepts

Do you know what is agent, workflow, scaffold, and agent runtime in your system?
Can you state the goal, enumerate tasks, and identify policies?
Can you draw the observe → think → act → reflect loop?
What scaffold structures your inner loop?
What memory do you use: scratchpad, episodic, long-term knowledge, semantic memory?
What is your context recipe (and how do you test it)?
Are you integrating tools directly, via MCP servers, or both?
Are you using A2A and does each agent have clear contracts and permissions?
What autonomy level are you at, and where are humans in/on/out of the loop?
What do you log as traces, and what telemetry do you track?
What evaluation harness do you use (task suites, judges, resettable environments, aggregation)?

2.11 Key Takeaways

Agents are software systems, not magic — An agent is an augmented LLM (model + tools + memory + scaffold + runtime) that loops through observe-plan-act-reflect to pursue a goal, and every piece of that stack is something you design and test.
Tools define the boundary of power — Each tool is a contract with a schema, validation, and access controls; if you get tool design right, you have already solved most of your safety surface area.
Context engineering is the real prompt engineering — What matters is not just the prompt text but the entire context recipe (policies, retrieved docs, history, scratchpad, tool descriptions) assembled under a token budget for each LLM call.
Scaffolds make agents reliable and debuggable — Patterns like ReAct, plan-and-execute, and critic-and-revise structure the inner loop so intermediate steps are visible, testable, and controllable rather than opaque.
Autonomy must be earned, not assumed — Start at Level 0 (insight only), prove safety and value with evaluation data, then graduate to higher autonomy with the matching oversight mode (HITL, HOTL, or HOOTL).
Evaluation is the bridge between building and operating — Inner-loop traces feed outer-loop evaluation harnesses (task suites, judges, sandboxes) that catch regressions, measure safety, and justify increased autonomy.
Design for failure modes from day one — Hallucination-with-action, infinite loops, prompt injection, and misaligned objectives are predictable risks with known mitigations: schema validation, step caps, instruction/data separation, and continuous eval.

Next → Chapter 3 turns these concepts into engineering principles: how to pick problems, earn autonomy, design for oversight, and make evaluation a first-class feature.

2.12 Reference Map by Chapter Concept

Concept	Recommended reputed sources	Why it's a good reference
What is an agent (LLM + tools + loop)	Anthropic: Building Effective AI Agents (Anthropic); OpenAI: New tools for building agents (OpenAI); Chip Huyen: Agents (Chip Huyen)	Concrete, builder-facing descriptions of tool use, loops, and real engineering lessons.
Tools / function calling as capability boundary	OpenAI: Function calling guide (OpenAI Platform); Anthropic: Building Effective AI Agents (Anthropic)	Shows "tools as contracts" and why tool definitions matter for reliability.
Structured outputs + schemas (JSON Schema)	OpenAI: Structured Outputs guide (OpenAI Platform); OpenAI: Introducing Structured Outputs (OpenAI)	Canonical references for "schema-constrained outputs" and why they improve reliability.
MCP (Model Context Protocol)	Anthropic: Introducing MCP (Anthropic); MCP official site (Model Context Protocol); MCP spec repo (GitHub); OpenAI: MCP concept docs (OpenAI Developers)	Covers what MCP is, the client/server model, and the official spec.
A2A / multi-agent coordination	Anthropic: How we built our multi-agent research system (Anthropic); Chip Huyen: Agents (Chip Huyen)	Strong real-world grounding for "supervisor + specialists", delegation, and coordination failures.
Skills (reusable capabilities)	Anthropic docs: Agent Skills (Claude Docs); Anthropic: Claude Agent SDK (Anthropic)	Uses "skills" explicitly and shows how teams package reusable agent behaviors.
RAG + retrieval components (retriever/reranker/query rewrite)	Chip Huyen: Building a Generative AI Platform (Chip Huyen); Jay Alammar: Illustrated Retrieval Transformer (RETRO) (Jay Alammar); Jay Alammar (InfoQ): RAG patterns & best practices (InfoQ)	Explains "context construction" clearly and gives practical RAG design patterns.
Context engineering / prompting for reliability	OpenAI: GPT-4.1 prompting guide (OpenAI Cookbook); OpenAI: GPT-5 prompting guide (OpenAI Cookbook)	Practical prompting guidance, including planning and agentic/tool flows.
Tracing, observability, and "debuggable agents"	OpenAI: Trace grading (OpenAI Platform); OpenAI: New tools for building agents (OpenAI)	Directly supports your "trace/span" and "inner vs outer loop" framing.
Evaluation, harnesses, and judge-based grading	OpenAI: Working with evals (OpenAI Platform); OpenAI: Evaluation best practices (OpenAI Platform); OpenAI: Agent evals (OpenAI Platform); OpenAI Evals (OSS) (GitHub); Anthropic system cards (agentic harnesses) (Anthropic); Demystifying evals for AI agents (Anthropic)	High-quality sources for "evaluation is a feature", harness thinking, and methods (reference-guided, pairwise, etc.).
Safety: prompt injection & tool risk	OpenAI: Understanding prompt injections (OpenAI); OpenAI: Agent builder safety (OpenAI Platform); Anthropic: Prompt injection defenses (Anthropic); Chip Huyen: security + prompt injection in GenAI platforms (Chip Huyen)	Authoritative guidance on the most common agentic threat vector (untrusted text → misaligned actions).
Sandboxing / safe execution boundaries	Anthropic: Claude Code sandboxing (Anthropic)	Great reference for "permissioning + isolation" as practical guardrails.
Governance / oversight for agentic systems	OpenAI paper: Practices for Governing Agentic AI Systems (OpenAI); OpenAI paper: Monitoring Monitorability (OpenAI)	Strong "oversight-by-design" framing to support your HITL/HOTL + eval/ops sections.
AGENTS.md (repo-level agent guidance) + AAIF origin/current state	OpenAI Codex docs: Custom instructions with AGENTS.md (OpenAI Developers); Linux Foundation AAIF announcement (AGENTS.md + MCP contributions) (Linux Foundation); OpenAI on AAIF + AGENTS.md donation (OpenAI); Anthropic on donating MCP to AAIF (Anthropic); AGENTS.md site (Agents)	Establishes both (a) what AGENTS.md is in practice and (b) its status as a foundational open standard under AAIF.