Chapter 4 — Scaffolds: Structuring Agent Behaviour (Inner Loop in Practice)

TL;DR

A scaffold is the reusable inner-loop behaviour pattern (prompts, tool routing, branching, and stopping conditions) that wraps an LLM so it behaves like an engineered workflow rather than an improvising chatbot.
The chapter covers five canonical patterns — Structured Prompt (single-step), ReAct (reason-act loops), Plan-Execute (upfront plan then stepwise execution), State Machine / Workflow (approval-driven flows), and Multi-Agent (manager-worker, router, critic-builder) — each matched to a different level of task complexity.
Every pattern is illustrated through the HR Leave Agent running example, showing how an employee's leave request moves from intake through policy checks, balance verification, approvals, and submission.
Scaffolds must be harness-ready from day one: structured traces, outcome labels, prompt versioning, and deterministic replay hooks are what make agent behaviour testable, comparable, and improvable in production.
Choosing the right scaffold is a design decision driven by task complexity, oversight requirements, and auditability needs — the chapter provides a decision tree and warning signs for when you have chosen wrong.

Chapter 2 introduced the core concepts: agents, tools, memory, and the split between inner loops and outer loops. Chapter 3 turned those concepts into builder principles such as Context-first design, Oversight by Design, and Evaluation is a Feature.

This chapter shows how those ideas become concrete in the inner loop. We use one running example throughout: the HR Leave AI Agent. This agent helps employees interpret leave policy, check their balance, draft a leave application, and (when permitted) submit it into an HR system.

Scaffold, in simple terms (HR leave analogy)

A scaffold is the reusable inner-loop behaviour pattern that wraps an LLM so it behaves reliably. It specifies the behaviour logic, prompts, tool routing decisions, and stopping conditions used step by step.

Use the HR leave process as an analogy. When an employee requests leave, the organisation does not rely on improvisation. The request is handled like a case with a repeatable playbook. The team collects the required details. They check policy and eligibility. They verify leave balance. They check constraints like public holidays and blackout periods. They obtain approvals when required. Finally, they submit in the HR system and confirm the outcome.

The scaffold is the agent's version of that playbook. It is not the policy itself and not the HR system. It is the step-by-step inner-loop logic that decides what to do next and which tools to use.

Mapping the HR leave process to an agent scaffold

Consider how each part of the leave process maps to scaffold behaviour.

Case intake fields include dates, leave type, and reason. The scaffold asks for missing information before proceeding.

Policy interpretation covers eligibility, carry-forward rules, and blackout periods. The scaffold retrieves policy clauses and prompts the model to interpret them in context.

Balance verification checks annual leave remaining and childcare leave entitlement. The scaffold calls the HRIS tool, validates the response, and updates task state.

Approval handling manages manager approval for certain cases. The scaffold inserts a checkpoint. It requests approval, waits, then continues or revises.

Submission and confirmation produces the HRIS transaction ID and notification. The scaffold triggers the final action or produces a draft. Then it returns evidence of the outcome.

Why this matters

Without a scaffold, the agent behaves like an untrained case handler. It may respond quickly, skip checks, or guess when unsure.

With a scaffold, the agent behaves like a well-run leave processing flow. It follows steps, enforces checks, routes for approval, and completes only when conditions are satisfied and outcomes are confirmed.

In simple terms: a scaffold is the agent's leave-case playbook, the steps and decision points that make behaviour reliable.

Scaffolds through a software engineering lens

Most readers already know how to build reliable systems without LLMs. A useful trick is to map agent scaffolds to software structures you already trust. Then translate the "agent-y" parts into familiar engineering concerns: control flow, state, contracts, invariants, observability, and tests.

A scaffold is not magic. It is the application control logic that turns a probabilistic model into a system that behaves like an engineered workflow.

Table 4.1 — Software analogies for scaffold, runtime, orchestrator, and harness

Familiar SE concept	Closest mapping	Why it helps	Where it breaks (important caveat)
Model–View–Controller (MVC)	Controller ≈ scaffold, Model ≈ task state + domain data, View ≈ agent response UI	Shows that the scaffold is the "controller" for the agent, it sequences actions, validates inputs, and decides what happens next	MVC is UI-centric, it does not naturally express tool safety, step budgets, or replayable traces
Request handler + service orchestration (classic backend)	Handler/service layer ≈ scaffold, platform concerns ≈ runtime	The scaffold looks like the business logic that coordinates calls to policy DB, HRIS, calendar, messaging	Handlers assume deterministic functions, LLM decisions are probabilistic, you need explicit guardrails and evaluation
Control loop (observe → decide → act → check)	Scaffold = the loop, runtime = execution substrate	Matches the agent loop directly, helps readers reason about stop conditions, retries, and fallbacks	In agents, "decide" is often model-driven, you must bound decisions with allowed actions
State machine / workflow (BPMN, saga, case management)	Orchestrator = deterministic shell, scaffold runs inside states	Explains approvals, waits, SLAs, and allowed transitions cleanly	A scaffold can contain a state machine, but long-running business progression usually belongs outside the scaffold
Pipeline (ETL stages, middleware chain)	Scaffold = staged pipeline, each stage has inputs/outputs	Makes Plan–Execute feel like a familiar "stages with contracts" pattern	Pipelines assume stable stage boundaries, LLM steps may need repair loops and dynamic routing
Command pattern	Tool calls = commands, schema-validated and auditable	Encourages "tools as contracts", typed I/O, idempotency, audit logs	Commands need permissioning and sandboxing, which belongs in the runtime, not in the scaffold
Test harness (unit/integration rig)	Evaluation harness = test rig, scenarios, replays, graders	Reinforces "evaluation is a feature" and "harness-ready" design	Unlike classic tests, pass/fail is often graded, and variance requires thresholds and regression suites

MVC as a bridge, and its limits

If you want an analogy that immediately clicks for many software builders, MVC is a good entry point.

The Controller maps to the scaffold. It routes intents, asks for missing fields, chooses tools, and enforces stop conditions.

The Model maps to domain and task state. This includes the leave case object, employee profile, leave balance, approvals, and policy snippets.

The View maps to presentation. This is the final answer, a draft application, a confirmation screen, or an escalation message.

Figure 4.1 — MVC-inspired mental model for the HR Leave agent

flowchart TB
  U[Employee request] --> V1[View<br/>Chat UI or form UI]
  V1 --> C[Controller<br/>Scaffold: route, validate, decide next step]
  C --> M[Model<br/>Leave case state + employee data]
  C --> RT[Runtime<br/>auth, retries, sandboxing, tracing]
  RT --> T[(Tools<br/>Policy search, HRIS API, Calendar, Messaging)]
  T --> M
  C --> V2[View<br/>Answer, draft, confirm, or escalate]

Note

MVC helps readers see scaffolds as "controller logic". The key difference is that the controller's decisions can be model-driven. You must bound them with allowed actions, budgets, validation, and tracing.

"Control loop / orchestrator" in classic systems

Many production systems already have a control loop even if they do not call it that.

In the Observe phase, the system parses the request, loads context, and fetches data. In the Decide phase, it chooses a code path and selects downstream calls. In the Act phase, it calls services or writes to systems. In the Check phase, it validates results, applies retries, and returns a response.

A scaffold is the same idea, except "Decide" often involves an LLM.

HR Leave mapping

In the HR Leave scaffold, Observe gathers user intent, dates, leave type, and policy clause retrieval. Decide determines whether to ask a clarifying question, call HRIS, check calendar, or stop. Act executes a tool call such as policy search, get balance, or create approval task. Check validates tool outputs, detects risky conditions, and produces a decision summary.

What this analogy teaches

Scaffolds should look like well-engineered request handling code. They need clear branching, bounded retries, validated contracts, and explicit outcomes. It is normal to separate responsibilities. The scaffold decides what to do. The runtime ensures it is safe, secure, and observable.

State machine / workflow as the "outer shell"

When readers are familiar with BPMN, state machines, sagas, or case-management systems, this mapping is often the clearest.

The workflow orchestrator is the deterministic shell that owns approvals and waits, SLA timers and escalation, and durable state transitions.

The scaffold runs inside a workflow step to do local reasoning. It interprets policy clauses, drafts messages, and proposes next actions.

Figure 4.2 — Orchestrator controls case progression, scaffold runs within states

stateDiagram-v2
    [*] --> COLLECT_INPUT
    COLLECT_INPUT --> CHECK_POLICY
    CHECK_POLICY --> CHECK_BALANCE
    CHECK_BALANCE --> DRAFT_REQUEST
    DRAFT_REQUEST --> AWAIT_MANAGER_APPROVAL: if requires_approval
    AWAIT_MANAGER_APPROVAL --> SUBMIT_TO_HRIS: if approved
    AWAIT_MANAGER_APPROVAL --> REVISION: if rejected
    REVISION --> DRAFT_REQUEST
    SUBMIT_TO_HRIS --> NOTIFY_EMPLOYEE
    NOTIFY_EMPLOYEE --> [*]

    note right of CHECK_POLICY
      Scaffold runs here:
      retrieve clauses, interpret policy,
      produce decision summary
    end note

What this analogy teaches

If the process is long-running and approval-heavy, do not force the scaffold to "sleep" or manage days-long waits. Let the orchestrator handle durable progression. Let the scaffold handle reasoning within a step.

Two additional analogies that often land well

Pipeline (Plan–Execute as staged processing)

Plan–Execute resembles an ETL or build pipeline. The plan stage outputs a structured plan like a DAG or job list. The executor runs steps in order. Each step emits outputs, validations, and trace labels.

This helps readers insist on per-step contracts, step-level logging, and failure handling that is local to a stage (retry, repair, escalate).

Command pattern (tools as auditable commands)

Tool calls map cleanly to commands. They have schema-defined inputs, validated execution, idempotency, audit logs, and explicit permission checks.

This helps readers see why "free-form tool calling" is a design smell. Tools should behave like a well-defined API surface.

A practical exercise for the reader

When you design a scaffold, try this translation:

What is the request handler? (entry/exit conditions, stop criteria)
What is the controller logic? (branches, loops, allowed actions)
What is the state model? (case object, progress markers, risk flags)
What are the commands and contracts? (tool schemas, validation, idempotency)
Where is the workflow shell? (approvals, waits, SLAs)
What will the harness assert? (trace labels, outcome flags, replayability)

If readers can answer these in familiar SE terms, they are already "thinking in scaffolds".

4.0 Learning Objectives

By the end of this chapter, you should be able to:

Select an appropriate scaffold type for a leave workflow.
Convert a vague "leave chatbot" idea into an explicit inner-loop design (steps, tools, stopping conditions).
Embed oversight, context, and explainability into the scaffold.
Make the scaffold harness-ready so it can be tested and improved in later chapters.

Running scenario for this chapter

An employee asks:
"I want to take 3 days of annual leave next week. Can you help me apply?"
The agent must interpret policy, validate eligibility, check balance, propose dates, draft the application, and request approval if required.

4.1 Where Scaffolds Fit in the Agent Architecture

In production, an agent is not only an LLM. It is an augmented system where different layers play different roles. Three terms are frequently mixed, so this section differentiates them clearly.

The agent scaffold is the inner-loop behaviour pattern. It defines how the agent reasons and acts step by step.

The agent runtime is the safe execution layer. It defines how tool calls run reliably, securely, and observably.

The workflow orchestrator is the deterministic process controller. It defines how a business case progresses over time with approvals and handoffs.

The evaluation harness sits outside these layers. It runs test cases, replays traces, and scores outcomes over time.

4.1.1 Definitions, by responsibility

Agent scaffold (inner-loop behaviour pattern)

The scaffold structures the agent's step-by-step behaviour. It handles prompts, tool routing, branching, and stopping conditions.

For HR Leave, the scaffold decides several things. Does it have enough information, or should it ask a clarifying question? Should it call policy search, HRIS balance, or calendar next? Should it stop at a draft, wait for confirmation, or proceed to submission?

Agent runtime (safe execution layer)

The runtime executes the scaffold consistently in production. It handles concerns the scaffold should not re-implement repeatedly. These include identity, auth, RBAC, and credential handling. They also include tool sandboxing and isolation, context assembly and compaction, retries, timeouts, rate limits, and circuit breakers. The runtime also handles observability through traces, logs, and metrics.

For HR Leave, when the scaffold calls get_leave_balance(user_id), the runtime enforces permissions, injects credentials safely, applies retries/timeouts, and emits trace spans.

Workflow orchestrator (deterministic business process control)

The workflow orchestrator controls the end-to-end business process. This often spans long-running steps lasting minutes to days. It handles step sequencing and allowed transitions using state machines or BPMN. It manages approvals, waits, SLAs, and escalations. It coordinates human-in-the-loop tasks and handoffs. It integrates with ticketing or case management.

For HR Leave, the orchestrator might define this sequence: Collect request → Validate policy → Manager approval → Submit → Notify employee → Close case. It invokes the agent scaffold within specific steps for tasks like interpreting policy or drafting a message.

Table 4.2 — Quick comparison: scaffold vs runtime vs orchestrator

Layer	Primary role	Time scale	Owns which decisions	HR Leave example
Agent scaffold	How the agent thinks and acts step by step	Seconds to minutes	Tool routing, branching, stop conditions, local state	Ask for missing dates, retrieve policy, check balance, draft application
Agent runtime	How execution is safe and operable	Per step and per call	Auth/RBAC, sandboxing, retries/timeouts, trace emission	Enforce permissions for HRIS read/write and log tool call spans
Workflow orchestrator	How the business case progresses	Minutes to days	SOP sequencing, approvals, waits, SLAs, escalation	Pause at manager approval, resume later, then submit and notify

Figure 4.3 — How the layers work together (HR Leave)

flowchart TB
  U[Employee] --> W_NODE

  subgraph W["Workflow orchestrator<br/>case progression, approvals, SLAs"]
    W_NODE[Orchestrator]
    WS1[Step: intake and validation] --> WS2[Step: manager approval wait] --> WS3[Step: submit and notify]
  end

  W_NODE --> RT_NODE

  subgraph RT["Agent runtime<br/>auth, sandboxing, retries, observability"]
    RT_NODE[Runtime]
    CTX[Context builder] --> EXEC[Execution controller] --> OBS[Traces and telemetry]
  end

  RT_NODE --> S_NODE

  subgraph SCAF["Agent scaffold<br/>inner-loop behaviour"]
    S_NODE[Scaffold]
    LOOP[Observe, Think/Plan, Act, Reflect] --> ROUTE[Tool routing and branching] --> STOP[Stopping conditions]
  end

  S_NODE --> LLM[LLM]
  S_NODE --> MEM["Memory<br/>scratchpad and task state"]
  S_NODE --> TOOLS[(Tools<br/>Policy search, HRIS API, Calendar, Ticketing, Messaging)]
  TOOLS --> S_NODE

  H["Evaluation harness<br/>scenarios, replay, grading, regression"] -. "runs and scores" .-> W_NODE
  H -. "runs and scores" .-> RT_NODE

Note

A useful rule of thumb. If it is about how the agent reasons and chooses actions, it is the scaffold. If it is about security, reliability, and observability of execution, it is the runtime. If it is about end-to-end process sequencing over time, approvals, and handoffs, it is the workflow orchestrator.

⚠️ Prerequisites: Review Chapter 2, §2.1 for agent/workflow/orchestrator definitions.

📚 Related sections: - Orchestration patterns → Chapter 9, §9.5 - Production layer separation → Chapter 11, §11.1 - Layer ownership in teams → Chapter 13, §13.2

4.1.2 Agent loop and inner/outer loops

The agent loop (Observe → Think/Plan → Act → Reflect → repeat or stop) was defined in Chapter 2, §2.4. The scaffold makes these transitions explicit. This chapter focuses on the inner loop (per-request scaffold execution); Chapters 5–7 cover the outer loop (harnesses, evaluation, and improvement across many runs).

4.2 From Principles to Scaffold Shape (HR Leave)

Chapter 3 principles translate into design decisions. Table 4.3 makes the mapping explicit.

Table 4.3 — Principles mapped to scaffold design decisions (HR Leave)

Principle (Ch3)	Scaffold design question	Typical implementation choice	HR Leave example
Select the right problems	How complex does the inner loop need to be?	Single-step vs ReAct vs Plan–Execute vs state machine	FAQ about carry-forward uses single-step, applying leave uses a multi-step scaffold
Clear, specific goals	When does the scaffold start and stop?	Explicit entry and exit conditions	Done when application draft is ready, or submitted and confirmed
Context-first design	What does the model see at each step?	Retrieval and context assembly before THINK/ACT	Retrieve relevant policy clause plus employee leave balance before drafting
Oversight by Design	Where do humans intervene?	Approval states, review branches, escalation	Manager approval required if leave exceeds threshold or overlaps critical period
Evaluation is a Feature	What evidence is emitted per step?	Structured traces, step tags, counters	Log tool calls, approvals requested, fallbacks used
Design for explainability	How are decisions inspectable?	Visible decision summaries and trace	"Reason: insufficient balance, action: propose alternative dates"

4.2.1 Select the right problems, choose scaffold complexity

Not every HR question needs an agentic scaffold.

Simple questions like "How many days of annual leave do I get?" are often solved by a single LLM call over policy snippets plus validation.

Complex requests like "Help me apply for leave next week" require multi-step reasoning, tool calls, and oversight points.

Tip

If you can answer with one instruction and verify easily, use a simple scaffold. If you need policy lookup plus HRIS checks plus calendar constraints plus approvals, use a richer scaffold.

4.2.2 Clear goals, define entry and exit conditions

A scaffold needs operational definitions.

Entry conditions (examples): - user intent is "apply leave" - user is authenticated - channel is supported (HR portal, Teams, Slack)

Exit conditions (examples): - application draft produced and user confirms details - submission completed and HRIS returns confirmation ID - escalation created and user notified

Exit conditions in practice

The agent should not stop after explaining policy. It should stop only when it has either drafted the leave request with dates and leave type ready for user confirmation, or submitted the request and displayed the confirmation.

4.2.3 Context-first design, control what the model sees

In HR Leave, context is not just policy text. It includes employee leave balances, employment type and tenure rules, blackout periods from business policy, public holiday calendars, manager details and approval rules, and locale and jurisdiction.

A context-first scaffold makes two choices. First, when to retrieve or refresh context. For example, re-check balance before submission. Second, how to structure the context using policy section, employee data section, task state section, and instructions.

4.2.4 Oversight by Design, place checkpoints in the flow

Oversight is implemented as control flow. Manager approval is required if leave length exceeds a threshold. HR review is required for special leave types. The system asks the user to confirm dates before submission.

Oversight as an explicit branch

IF action_type == "submit_leave_request" THEN
    IF requires_manager_approval THEN
        request_manager_approval()
        WAIT for approval result
        IF approved THEN submit_to_HRIS()
        ELSE ask_user_to_adjust_request()
    ELSE
        submit_to_HRIS()
END IF

4.2.5 Evaluation is a Feature, design instrumentation early

A harness can only evaluate what the scaffold makes observable.

For HR Leave, instrumentation should capture which policy snippets were used, which HRIS fields were read and written, when approvals were requested and their outcomes, whether the agent asked clarifying questions, and whether it fell back to escalation.

4.2.6 Design for explainability, produce inspectable reasons

Explainability in HR workflows is often about accountability. Users need to know why a leave request was rejected or routed for approval. They need to understand why the agent asked for more information. They need to see why it recommended alternative dates.

The scaffold should maintain a short, user-friendly decision summary and a fuller trace for auditors.

📚 Related sections: - Orchestration patterns → Chapter 9, §9.5 - Model routing by scaffold step → Chapter 9, §9.6A - Harness compatibility by scaffold type → Chapter 5, §5.2

4.3 Anatomy of a Scaffold (Applied to HR Leave)

4.3.1 Core building blocks

A typical HR Leave scaffold uses several step types and state elements.

Step types:

THINK: decide next action (policy lookup, balance check, clarification)
ACT: call tools (policy search, HRIS balance query, calendar query)
OBSERVE: interpret tool results into state
REFLECT: check completeness and risks
DECIDE: stop, continue, escalate, or request approval

State elements:

task state: leave type, dates, duration, reason
employee state: balance, role, tenure, manager
risk flags: low balance, blackout period, missing data
audit state: policy references used, tool calls made

4.3.2 Control-flow primitives

HR Leave scaffolds commonly use four control-flow patterns.

Sequence moves through authenticate → gather info → check balance → draft → confirm → submit.

Loops ask clarifying questions until required fields are filled.

Branches separate "approval required" from "not required" paths.

Fallback triggers escalation when tools fail.

4.3.3 Interaction with context engineering

Context engineering shows up at multiple points in the scaffold.

Before THINK, the scaffold retrieves policy snippets for the leave type. Before ACT, it validates tool arguments like dates and leave type code. Before REFLECT, it adds tool errors and risk flags into the prompt for critique.

4.3.4 Guardrails inside the scaffold

Guardrails should separate "model suggestion" from "system action".

On input to model, redact PII when not needed and enforce policy reminders. On model to action, validate date ranges, enforce role-based permissions, and block write actions without approval.

4.3.5 Instrumentation and logging

A minimal trace should include task_id, user_id (or anonymized id), and session_id. For each step, log step_id, step_type, prompt_id, tool_name, and timestamps. Record tool results including success, error_code, and latency. Track policy references with document id and clause ids. Include outcome flags like submitted, draft-only, or escalated.

📚 Related sections: - Approval gates in governance → Chapter 12, §12.2 - Preview→Commit in UX → Chapter 10, §10.4 - HR Leave complete example → Chapter 14, §14.2

4.4 Canonical Single-Agent Scaffold Patterns (HR Leave)

Table 4.4 — Scaffold pattern comparison (HR Leave examples)

Pattern	Complexity	When to use	Strengths	Limitations	HR Leave example
Structured Prompt	Low	Single-step questions with clear format	Fast, cheap, easy to evaluate	Limited multi-step control	"Summarize carry-forward rule for annual leave"
ReAct	Medium	Iterative lookup and reasoning	Flexible, transparent trace	Can be slow, loop risk	"Find policy clause, then apply to employee's tenure"
Plan–Execute	Medium–High	Multi-step leave application preparation	Inspectable plan, stepwise progress	Plan quality matters	"Prepare leave application draft with checks and confirmations"
State machine	High	Approval-driven or regulated flows	Strong control, auditable	More design effort	"Leave request requires manager approval and HR review"

4.4.1 Structured Prompt scaffold (single-step)

Use case: policy explanation or rule summarization.

Example: "Can I carry forward unused annual leave?"

Structure:

Retrieve relevant policy clause.
One LLM call: summarize policy, include caveats.
Validate: includes key constraints (cap, deadlines, eligibility).
Return answer.

Common pitfall: Trying to encode the entire application workflow in one prompt. That belongs in a richer scaffold.

4.4.2 ReAct scaffold (Reason + Act loops)

Use case: interpret a request that needs multiple checks.

Example: "Can I take childcare leave next month and carry forward unused days?"

Structure: Loop through Thought → Tool → Observation → Next Thought until ready to answer.

Figure 4.4 — ReAct scaffold flow (HR Leave)

flowchart TD
  Q[User query + initial context] --> L1[LLM: decide next check]
  L1 -->|Action: policy search or HRIS query| T[Tool execution]
  T --> O[Observation: policy clause or balance or calendar]
  O --> L2[LLM: update reasoning and choose next action]
  L2 -->|Continue until stop or max steps| L1
  L2 -->|Final answer or next step| F[Output: answer, draft, or clarification]

Controls enforced by the scaffold: The scaffold enforces max steps, allowed tools, schema validation for tool args, and consistent logging per step.

4.4.3 Plan–Execute scaffold

Use case: produce a leave application draft with checks.

Example: "Apply 3 days annual leave next week."

Plan example:

Confirm dates and leave type.
Check leave balance and eligibility.
Check calendar constraints (public holidays, blackout period).
Draft application text and summary.
Ask user to confirm.
Submit or request approval.

Figure 4.5 — Plan–Execute flow (HR Leave)

flowchart LR
  G[Goal + context] --> P[Planner LLM]
  P --> PL[Plan: steps 1..N]

  PL --> EX[Executor loop]
  EX --> S1[Step execution: tool calls and prompts]
  S1 --> SN[Repeat for remaining steps]
  SN --> R[Aggregate results]
  R --> FA[Output: draft or submission request]

Where oversight fits: HR ops reviews the plan for specific leave types. Users confirm before submission.

4.4.4 State machine or workflow-driven scaffold

Use case: regulated flows where approvals are mandatory.

Example: leave request above threshold requires manager approval.

State machines often belong to the workflow orchestrator

A state machine can be implemented inside the scaffold controller. But conceptually it often belongs to the workflow orchestrator—the deterministic shell that governs approvals, waits, SLAs, and allowed transitions. In that setup, the scaffold focuses on local inner-loop behaviour within a state (retrieve policy, call HRIS, draft message). The orchestrator decides when to advance the case and when to wait for humans.

Figure 4.6 — HR Leave state machine example

stateDiagram-v2
    [*] --> COLLECT_INPUT
    COLLECT_INPUT --> POLICY_LOOKUP
    POLICY_LOOKUP --> CHECK_BALANCE
    CHECK_BALANCE --> CHECK_CALENDAR
    CHECK_CALENDAR --> DRAFT_REQUEST

    DRAFT_REQUEST --> MANAGER_APPROVAL: if requires_approval
    DRAFT_REQUEST --> USER_CONFIRMATION: if no_approval_required

    MANAGER_APPROVAL --> USER_CONFIRMATION: if approved
    MANAGER_APPROVAL --> REVISION: if rejected

    REVISION --> DRAFT_REQUEST

    USER_CONFIRMATION --> SUBMIT_TO_HRIS
    SUBMIT_TO_HRIS --> [*]

Why this helps: Every step is a state transition, which provides auditability. Only allowed transitions can occur, which provides safety.

📋 Complete Example: The HR Leave Agent implements a six-state scaffold (INTAKE → RETRIEVE → ASSESS → RESPOND → PREVIEW → SUBMIT). See Appendix E, §E.2 for the full implementation with state definitions, transition logic, and state-specific tool restrictions.

4.4.5 Scaffold Selection Decision Tree

Use this flowchart to select the appropriate scaffold pattern for your agent.

flowchart TD
    START([Start: Define your agent task]) --> Q1{Does the task have<br/>a fixed sequence of steps?}

    Q1 -->|Yes, always same order| Q2{Are there approval<br/>gates or checkpoints?}
    Q1 -->|No, steps vary by input| Q3{Is the step order<br/>predictable from input?}

    Q2 -->|Yes| SM[State Machine Scaffold]
    Q2 -->|No| PIPE[Pipeline Scaffold]

    Q3 -->|Yes, can route upfront| PLAN[Plan-Execute Scaffold]
    Q3 -->|No, must decide step-by-step| Q4{How many tools<br/>are involved?}

    Q4 -->|Few, 2-4 tools| REACT[ReAct Scaffold]
    Q4 -->|Many, 5+ tools| Q5{Can tools be grouped<br/>into skills?}

    Q5 -->|Yes| SKILL[Skill-Based Scaffold]
    Q5 -->|No, flat tool space| Q6{Is task decomposable<br/>into specialist subtasks?}

    Q6 -->|Yes, clear boundaries| MULTI[Multi-Agent Scaffold]
    Q6 -->|No| REACT2[ReAct with tool scoping]

    SM --> CHECK1{High-stakes writes?}
    PIPE --> CHECK1
    PLAN --> CHECK1
    REACT --> CHECK1
    REACT2 --> CHECK1
    SKILL --> CHECK1
    MULTI --> CHECK1

    CHECK1 -->|Yes| ADD1[Add: Preview→Commit pattern]
    CHECK1 -->|No| CHECK2{Need audit trail?}
    ADD1 --> CHECK2

    CHECK2 -->|Yes| ADD2[Add: Checkpoint logging]
    CHECK2 -->|No| DONE([Scaffold selected])
    ADD2 --> DONE

Quick reference

If your task...	Consider...	Example
Has fixed steps with approvals	State Machine	HR Leave, Invoice approval
Has fixed steps, no approvals	Pipeline	Document processing
Can plan all steps upfront	Plan-Execute	Research tasks
Needs dynamic step-by-step decisions	ReAct	Q&A with tool use
Has reusable capability bundles	Skill-Based	Multi-domain assistant
Has clear specialist subtasks	Multi-Agent	Complex workflows

Warning signs you chose wrong

State machine with >10 states → consider Plan-Execute
ReAct with >20 average steps → add structure or checkpoints
Multi-agent with >50% coordination overhead → simplify to single agent
Pipeline with frequent exceptions → consider State Machine

4.5 Multi-Agent and Orchestration Scaffolds (HR Leave)

Multi-agent scaffolds are optional in HR Leave. They are useful when responsibilities are distinct.

4.5.1 Manager–Worker scaffold

Example decomposition: Worker A interprets policy. Worker B checks HRIS data. Worker C drafts user-facing messages and application text.

Figure 4.7 — Manager–Worker coordination (HR Leave)

flowchart TB
  M[Manager agent: decompose and integrate]

  subgraph Workers[Worker agents]
    W1[Policy worker]
    W2[HRIS worker]
    W3[Drafting worker]
  end

  M -->|Task| W1
  M -->|Task| W2
  M -->|Task| W3

  W1 -->|Result| M
  W2 -->|Result| M
  W3 -->|Result| M

  M --> F[Final: answer, draft, or submission request]

4.5.2 Router scaffold

A router scaffold routes requests by intent. "Leave policy" goes to a structured prompt or ReAct. "Apply leave" goes to Plan–Execute or state machine. "Claims" goes to a different agent.

A router can be rule-based first. Then it can be upgraded to a model-based classifier.

4.5.3 Critic–builder scaffold (optional)

For high-stakes communications, a critic reviews the draft before submission. The builder drafts the application and rationale. The critic checks policy grounding, tone, and missing details. They iterate within limits.

4.6 Embedding Oversight and Safety into HR Leave Scaffolds

Oversight is implemented as control flow. Safety is implemented as enforcement.

Table 4.5 — Oversight points by scaffold pattern (HR Leave)

Scaffold pattern	Typical oversight points	HR Leave example
Structured Prompt	Sampled review	HR audits 5% of policy answers
ReAct	Risky tool calls, step-limit triggers	Manual review if repeated failed HRIS calls or sensitive data exposure risk
Plan–Execute	Plan and final review	HR ops reviews plan for special leave types
State machine	Approval states embedded	Manager approval required before submission for certain conditions
Manager–Worker	Handoff inspection	Manager agent's task decomposition reviewed in high-risk deployments

4.6.1 Policy checks and guardrails

Apply checks at three points. On inputs, detect prompt injection attempts and restrict sensitive content. On tool actions, enforce role-based access, validate date ranges, and validate leave type codes. On outputs, ensure the answer includes policy caveats when required.

4.6.2 Sandboxing side-effectful actions

Start with draft-only mode. Generate an application draft, show confirmation, but do not submit.

Graduate to submission mode. Submit only after user confirmation. Submit only after approvals if needed. Submit only via whitelisted, typed tool calls.

4.6.3 Fail-safe behaviours

On failure, the scaffold has several options. Ask for missing info like dates or leave type. Propose alternatives when balance is insufficient. Escalate to HR ticketing with evidence when tools fail or policy is ambiguous.

4.7 Instrumentation and Hooks for Harnesses (HR Leave)

This section bridges inner-loop scaffolds to later chapters on harnesses and evaluation.

Who emits which telemetry

Instrumentation is shared across layers. Separating responsibilities keeps traces consistent.

Scaffold telemetry includes step labels (THINK, TOOL_CALL, OBSERVATION, REFLECTION, FINAL), routing decisions (which tool, which branch), and short decision summaries (why it chose an action or escalation).

Runtime telemetry includes execution spans for each model and tool call (latency, retries, timeouts, error codes), resource signals (rate limits, circuit breakers), and security signals (auth checks, permission denials).

Workflow orchestrator telemetry includes case-level status transitions (opened, awaiting approval, approved, submitted, closed), SLA and timer events (wait durations, overdue approvals), and handoff events (assigned to manager, escalated to HR ticketing).

A harness can then score outcomes using scaffold decisions. It can diagnose performance using runtime spans. It can monitor operational health using orchestrator SLAs.

4.7.1 What to log at each step

A harness-ready scaffold logs step type, tool calls, and tool outcomes. It logs policy references used. It logs approvals requested and their outcomes. It logs fallbacks and escalations.

Table 4.6 — Step type to logging requirements

Step type	Mandatory log fields	Optional fields	Why it matters
THINK	step_id, task_id, prompt_id, prompt template	confidence, tokens_used	Debug reasoning and compare prompt variants
TOOL_CALL	tool_name, args (sanitized), timestamp	latency_ms, retries	Diagnose tool issues and detect overuse
OBSERVATION	tool_name, result summary	full result (if allowed), error_code	Understand tool impact and find flaky tools
REFLECTION	reflection text, referenced step_ids	auto-score, risk_flags	See self-correction and detect hallucination patterns
FINAL_ANSWER	final output, success flag	quality_score, user feedback	Outcome tracking

Figure 4.8 — Example trace view (HR Leave)

Step #	Type	Description	Tool	Status
1	THINK	User wants to apply for annual leave	—	✓
2	TOOL_CALL	Retrieve policy clauses for annual leave	policy_search	✓
3	OBSERVATION	Found carry-forward cap and blackout rules	—	✓
4	TOOL_CALL	Check employee leave balance	hris_balance	✓
5	OBSERVATION	12 days remaining	—	✓
6	THINK	Sufficient balance, check calendar	—	✓
7	TOOL_CALL	Check public holidays	calendar	✓
8	OBSERVATION	No conflicts detected	—	✓
9	REFLECTION	Ready to draft application	—	✓
10	FINAL_ANSWER	Draft produced, awaiting user confirmation	—	✓

4.7.2 Evaluation hooks

Typical hooks include emitting a final outcome label (success, partial, escalation, failure), logging which golden scenario this run most resembles, and flagging any rule violation for immediate review.

4.8 Skill and Capability Modularity (HR Leave)

Skills are reusable sub-scaffolds that can be composed into richer agents.

4.8.1 Skills as encapsulated capabilities

Examples for HR Leave:

AssessLeaveEligibility: given employee profile and leave request, determine eligibility and required approvals.
GetLeaveBalance: call HRIS, return balance summary.
DraftLeaveRequest: given confirmed inputs, produce a user-facing request draft.
SubmitLeaveRequest: submit to HRIS, return confirmation or error.
EscalateToHR: create a ticketing request with evidence bundle.

4.8.2 Skill interface patterns

A skill exposes an input schema (typed, documented), an output schema (typed, documented), and metadata including latency profile, cost tier, and side-effect flags.

4.8.3 Skill composition in a router

A router scaffold selects skills by intent. "What is my leave balance?" uses GetLeaveBalance only. "Apply for leave" uses AssessLeaveEligibility → GetLeaveBalance → DraftLeaveRequest → (SubmitLeaveRequest or EscalateToHR).

4.9 Scaffold Design Practices Summary

4.9.1 Harness compatibility features

Building an agentic system that is reliably testable requires designing the scaffold with a harness in mind.

4.9.1.1 Trace Schema Definition

Define a structured trace format that captures all important run data. Include step-level actions, LLM inputs and outputs, tool calls and results, and timing information. A well-defined trace schema lets harnesses replay and analyze runs consistently.

4.9.1.2 Outcome Flags and Labels

Every run should end with clear outcome tags. Harnesses use these to categorize results at a glance.

submitted: agent completed the task successfully
escalated: agent handed off to a human or backup
incomplete: agent could not reach a solution

Clear outcome tags let harnesses automatically measure success rates and identify when and why the agent fails or defers.

4.9.1.3 Prompt & Model Versioning

Track the versions of prompts, models, and tools used by the scaffold for each run. Treat prompt templates as versioned artifacts, just like code. Record which model and tool versions were invoked. This makes it possible to reproduce behaviors later and compare runs when prompts or models change.

4.9.1.4 Deterministic Replay

Whenever possible, design the scaffold to produce repeatable results under the same conditions. Use fixed random seeds. Avoid nondeterministic operations in the agent loop so that test harnesses can replay scenarios reliably.

Deterministic scaffolds let you rerun identical trials to debug issues. They ensure that changes in outputs are due to intentional updates rather than randomness.

4.9.1.5 Why this matters

By extending the scaffold with these harness compatibility features, you make it much easier to observe, test, and trust your agent. A well-instrumented scaffold paired with a robust harness means you can trace every decision, pinpoint failure modes, and continually improve the agent's performance in a controlled way.

4.9.1.6 Scaffold design checklist (HR Leave)

Is the goal concrete (draft, confirm, submit, or escalate)?
Are entry and exit conditions explicit?
Which scaffold type fits the risk and workflow?
What context is required at each step (policy, balance, calendar)?
Where are approvals and human checkpoints?
What trace fields are emitted per step?
What are the safe fallbacks for failures?

📜 Coming up: - Harness design → Chapter 5 - Trace structure for replay → Chapter 5, §5.3 - Evaluation playbook → Chapter 7

4.10 Key Takeaways

A scaffold is application control logic, not magic — it is the explicit inner-loop pattern (prompts, tool routing, branching, stop conditions) that turns a probabilistic LLM into an engineered, repeatable workflow.
Match scaffold complexity to task complexity — use a Structured Prompt for single-step lookups, ReAct for iterative reasoning, Plan-Execute for multi-step preparation, and a State Machine for approval-heavy regulated flows.
Separate scaffold, runtime, and orchestrator concerns — the scaffold owns reasoning and tool selection, the runtime owns security and observability, and the workflow orchestrator owns long-running case progression, approvals, and SLAs.
Oversight is control flow, not an afterthought — embed human checkpoints (manager approvals, user confirmations, HR reviews) directly into scaffold transitions so they are mandatory, not optional.
Context-first design determines agent quality — what the model sees at each step (policy snippets, balances, risk flags) matters more than the model's raw capability; retrieve and structure context deliberately before every THINK or ACT.
Design for harness compatibility from the start — structured trace schemas, outcome labels, prompt versioning, and deterministic replay hooks are what make scaffold behaviour testable, comparable, and debuggable across iterations.
Use the decision tree, then watch for warning signs — select a scaffold pattern using task characteristics (fixed steps, approvals, tool count, decomposability), and revisit the choice if you see runaway state counts, excessive loop steps, or coordination overhead dominating execution time.

Next --> Chapter 5 introduces harnesses — the testing and observability rigs that make scaffold behaviour replayable, comparable, and debuggable.

4.11 Reference map

Concept	What it covers (engineering angle)	Primary sources (clickable)
Scaffold (inner loop)	The explicit "run loop" that structures Observe → Decide → Act → Check, including stop conditions and allowed actions	Anthropic — Building effective agents (Dec 2024) Chip Huyen — Agents (Jan 2025) Simon Willison — "agent" definition (Sep 2025)
Scaffolds vs prompt-only	Why prompt-only systems break down for multi-step work, and why control logic belongs in code/graphs	Anthropic — Building effective agents (Dec 2024) Chip Huyen — Agents (Jan 2025)
Agent scaffold vs agent runtime vs workflow orchestrator	Separation of concerns: behaviour logic (scaffold) vs execution layer (runtime: tool sandboxing, auth/RBAC, retries, state) vs durable workflow management (orchestrator: long-running state, SLAs, timers)	Anthropic — Building agents with the Claude Agent SDK (Sep 2025) OpenAI — Agents resources (Docs) Temporal — Observability (Docs)
Structured prompt scaffolds	Single-step scaffolds with strict schemas, validation, retries, and predictable outputs	OpenAI — Structured outputs (Docs) OpenAI — Function calling (Docs)
ReAct scaffold	Iterative Thought → Tool → Observation loops, step limits, tool allowlists, and scratchpad state	ReAct paper (arXiv, 2022) Google Research — ReAct (Oct 2022) Anthropic — Building effective agents (Dec 2024)
Plan–Execute scaffold	Explicit planning phase, then stepwise execution with per-step checks, plan repair, and human review points	OpenAI — Orchestrating Agents: Routines and Handoffs (Oct 2024) OpenAI — A practical guide to building agents (PDF)
Workflow-driven or state-machine scaffolds	Bounded transitions, mandatory approval states, auditability, "judgment within guardrails"	OpenAI — A practical guide to building agents (PDF) Temporal — Observability (Docs)
Multi-agent orchestration	Routines, delegation, handoffs, manager–worker coordination, and composition of specialists	OpenAI — Orchestrating Agents: Routines and Handoffs (Oct 2024) Anthropic — How we built our multi-agent research system (Jun 2025)
Tools as contracts	Typed tool schemas, validation, safe defaults, and making tool calls auditable and testable	Anthropic — Writing effective tools for agents (Sep 2025) OpenAI — Function calling (Docs) OpenAI — Structured outputs (Docs)
Context engineering in scaffolds	Step-specific context builders, context budgets, delimiting, and retrieval strategies that reduce "good LLM, bad retriever" failures	Anthropic — Effective context engineering for AI agents (Sep 2025) Anthropic — Introducing Contextual Retrieval (Sep 2024) Jay Alammar — RAG patterns & best practices (InfoQ, May 2024)
Oversight, permissions, sandboxing	Approval gates, least privilege, sandboxed tool execution, and kill switches for side-effectful actions	Anthropic — Beyond permission prompts: making Claude Code more secure and autonomous (Oct 2025) OpenAI — Safety best practices (Docs)
Instrumentation and telemetry ownership	What to log at each layer: scaffold (step labels, decision summaries), runtime (spans, tool metrics), orchestrator (case status, SLA events), plus trace-based debugging	OpenAI — Tracing and debugging agents (Docs) OpenAI — Trace grading (Docs) OpenTelemetry — Specification (Traces, Metrics)
Harness compatibility	Making scaffolds replayable and measurable: trace schemas, outcome flags, regression datasets, and repeatable eval runs	Anthropic — Demystifying evals for AI agents (Jan 2026) OpenAI — Working with evals (Docs) OpenAI — Evaluation best practices (Docs) Anthropic — Effective harnesses for long-running agents (Nov 2025)
Scaffold selection decision tree	Flowchart and quick reference for choosing the right scaffold pattern based on task characteristics	This handbook