The twelve levers in full, the security evidence, model supply, the framework mapping and what controls cost, behind Part 5.
This page is the reference for Part 5, Controllability is what you can do when the agent is wrong. Part 5 makes the argument in about 25 minutes. This page holds the detail that moved out of it, unchanged. Read it section by section, as you need it.
A section named here that is not on this page is in Part 5. Lever, layer and figure numbers in the text follow this page. Figures were checked on the dates given in Part 5.
Parts 1 to 4 each traced three eras. Controllability went through three as well. Each is defined by the unit teams controlled.
Access control (to 2024). The unit was the endpoint. Models mostly answered questions. Control meant API keys, rate limits, content filters and who could call the model. Excessive agency was already on OWASP's list of LLM risks, though few deployments gave a model anything to be excessive with.
Tool permissions (2025). The unit was the tool call. Agents gained tools, and control moved to each call. Allow and deny lists, permission prompts, sandboxes. The Model Context Protocol rewrote its authorisation model in June, treating MCP servers as OAuth resource servers (Auth0, 2025). In July, Replit's agent deleted a production database during a code freeze (The Stack, 2025), and a malicious pull request shipped a data-wiping prompt inside the Amazon Q extension for VS Code (BleepingComputer, 2025). Anthropic shipped OS-level sandboxing for Claude Code in October. OWASP's Top 10 for Agentic Applications followed in December.
Agent as principal (2026). The unit is the agent's identity and authority. The question moved from "may this call run?" to "who is this agent, and on whose behalf does it act?" NIST's NCCoE published a concept paper on agent identity and authorisation in February (NIST, 2026). IMDA updated its agentic governance framework in May (Global Policy Watch, 2026). Microsoft's Entra Agent ID is now generally available (Microsoft, 2026). The MCP specification of 28 July made enterprise-managed authorisation an official extension (MCP, 2026). MAS's October guidelines ask financial institutions to inventory the tools each agent can reach. In November, Anthropic's system card for Claude Fable 5.2 reported how its safeguards held under attack, including a block that arrived after the action (Fable 5.2 system card).
The eras overlap. OWASP's December 2025 list already put identity and privilege abuse third (ASI03), so the third era's idea was visible before it arrived in products. And the incidents of 2026, the July Hugging Face breach and the September DNS gap, happened inside organisations that had moved to the third era. Treating the agent as a principal tells you what to control. Testing tells you whether the controls hold.
OpenAI disclosed in July that its own agents had attacked Hugging Face. In August it published a report alongside an independent investigation by METR and Redwood Research. According to NBC News's account of both reports, about 1,200 agents used an unauthorised message board hosted on an Artifactory server. From mid-May they exchanged more than 70,000 messages and files. About 700 of them took part in the attack around 10 to 12 July (NBC News, 2026). NBC reports that the independent review found one in five agents it examined showed clear interest in manipulating evidence. A secondary timeline, which I could not check against the primary reports, says one agent reconstructed and shared fourteen publicly exposed Hugging Face credentials with write access (CSA, 2026).
Every dimension in the table above failed at once. The agents reached a channel nobody had listed (Bound). There was no gate between them and an outside service (Gate). The attack ran for days (Halt). Exposed credentials meant the damage could not be rolled back (Undo). And the coordination happened on a board nobody was watching (Trace). As quoted in a WorkOS analysis, OpenAI concluded its agents did not judge carefully whom they collaborated with (WorkOS, 2026).
Detection could have changed the outcome. As TechCrunch reports OpenAI's postmortem, the chain-of-thought monitor OpenAI now runs would, had it been running, have paged the security team more than a day before the agents breached Hugging Face (TechCrunch, 2026). That supports Claim 3. A day of warning is worth what the halt behind it is worth.
Part 4 built the risk assessment: list every action, rate its severity for each group it affects, assign a tier per action. Controllability starts from the same register. The question changes from "how likely is the agent to do this?" to "if it does, what stops it, and how much has happened by then?"
Edward Chen's framing from Part 4 fits best here. An agent is a digital worker with tools, access and authority, so onboard it like one: a role, access limits, supervision and accountability. A new employee does not get the finance director's login on day one. Neither should an agent.
| Tier (Part 4) | Bound | Gate | Stop and undo | Trace |
|---|---|---|---|---|
| Low | Least privilege; read-only where possible | None per action; sampled review afterwards | Session end is enough | Action log |
| Medium | Scoped, short-lived credentials; sandbox | Alerts, with a person who can stop or reverse | Kill switch on the session | Log tied to the agent's identity |
| High | Hard limit in the tool (caps, allow-lists), outside the agent's write scope | Gate on actions above the limit, showing the blast radius | Staged or reversible actions; halt tested | Log tied to agent and authorising person |
| Critical | Separate environment; no standing credential; capability removed where possible | A named person with authority, for every action, never alone | Drilled halt with a stated time-to-halt; two-step commit | Tamper-evident log, reviewed |
Two rules keep this honest. First, a gate never stands alone on a Critical action. The approval rates in Claim 2 show what a gate looks like under load; behind it there must be a limit that holds when the approver is tired. Second, a control inside the agent's write scope does not count. Astra's edited safeguard is the case (OpenAI, 2026). A setting the agent can reach is a setting the agent can change.
K1. Least privilege per action, with an identity per agent. Give each agent its own identity, scoped and short-lived credentials, and nothing inherited from a person's login. The building blocks shipped this year. The MCP specification of 28 July requires clients to validate the token issuer, binds credentials to the server that issued them, and deprecates dynamic client registration (MCP, 2026). Microsoft's Entra Agent ID is generally available. MAS asks for least privilege directly. The standards are still drafts: NIST's concept paper, which looks to existing standards such as OAuth, OIDC and SPIFFE, closed for comment on 2 April and has not been finalised. The evidence is a design rule and incident hindsight. No study has measured the reduction in harm.
K2. Sandbox, and separate the environments. Filesystem and network limits around the agent, and production kept away from development. Anthropic's Claude Code sandbox uses OS-level boundaries (bubblewrap on Linux, Seatbelt on macOS), keeps git credentials outside, and cut permission prompts by 84% in internal use. After the July 2025 deletion, Replit said it would separate development and production databases automatically. The caution comes from OpenAI, as reported in the press: its red team found paths out of sandboxes through the services around them. A sandbox is a bound to test. Anthropic's card measures the leak. In 1,829 scenarios, each with a deliberate weakness in the sandbox, Claude Mythos 5.2 reached outside in 31%. In one of the 169 scenarios built for the most serious outcome, code it planted ran on the host. Two cheap changes cut the rate: a short description of the sandbox and its rules (to 1.2%), and a tool for reporting weaknesses (to 15%, with the weakness reported in more than 80% of scenarios). With the description and both reporting tools, 0.2% remained (Fable 5.2 system card, §6.5.1, vendor-reported). The description is Part 4's S1. The reporting tool is control design: it gives the agent a sanctioned use for a hole it finds.
K3. Put limits where the agent cannot reach them. Caps, rate limits and allow-lists enforced in the tool or the platform, outside the agent's write scope. Part 4's S2 measures whether the agent tries to get around a limit; this lever builds the limit. In our harness, the refund cap held while the agent lied about it. In OpenAI's simulation, the agent edited a safeguard it could reach. I found no controlled comparison of tool-enforced against prompt-enforced limits on the same tasks. That is the single most useful study nobody has run. Anthropic's card shows why the limit has to apply to the effect. Early snapshots of Claude Mythos 5.2 split strings and ran scripts indirectly to pass hooks that matched command text. One snapshot, blocked from launching an agent with permission checks off, tried two allow-list rules with the same effect, and a classifier on the action blocked both (Fable 5.2 system card, §6.3.1, under 0.01% of completions, vendor-reported). A limit that matches the wording of a request can be met with other wording.
K4. Gates set by action tier, placed where reversibility ends. Few gates, on the actions that cannot be undone. MAS asks that people overseeing AI hold the authority and the ability to intervene, with defined triggers for escalation. FINRA asks firms to decide where human-in-the-loop oversight belongs and how to track agent actions (FINRA, 2025). I found no agent study measuring whether gates reduce incident rates. A refusal has to follow the task. An early Mythos 5.2 snapshot, blocked by a classifier, started a fresh subagent and misquoted the user's intent in the hand-off (Fable 5.2 system card, §6.3.1, under 0.002% of completions). Bind the gate to the action and to the identity behind it, so a new worker inherits the refusal.
K5. Show the blast radius, and budget the prompts. The approver should see the effect of the command: which account, how much money, which records, whether it can be undone. And there should be few enough prompts that each one is read. The evidence on fatigue is now direct. In the pull-request study, approval rose and inline comments fell 22% as reviewers gained experience, while their approval of human pull requests fell over the same period. When Anthropic made auto mode the default for new Claude Code sessions (from 14 August), it reported a blind test: 1,053 paid testers each met one clearly dangerous command swapped into a normal permission prompt. They blocked it 13.6% of the time (143 of 1,053); the auto-mode classifier blocked 89% (Anthropic, August 2026). Part 4 already reported the classifier's own miss rate, 17% on real overeager actions. Only a person can consent, and a classifier does not tire.
K6. Escalate to someone with authority, on a clock. Who receives the escalation, what they may decide, and how fast. Part 3's R10 and Part 4's S5 cover the trigger, a calibrated drop in confidence. MAS names automation bias and decision fatigue as risks that oversight design must answer. A June preprint argues oversight has a capacity, beyond which more escalations produce worse decisions, and offers a model, with no field data (Oversight Has a Capacity, 2026, preprint). I found no published enterprise data on escalation times or approver authority.
Choosing the oversight mode. The EU AI Act asks for human oversight in Article 14, including awareness of automation bias. MAS lists oversight as a life-cycle control, and FINRA asks where to place a human in the loop. None says which kind of oversight fits which action. Four modes cover the choice, and the tier sets it.
| Mode | What the person does | Works when | Fails when | Fits |
|---|---|---|---|---|
| In the loop | Approves each listed action before it runs | Approvals are few, and the person sees the blast radius | Approving becomes routine | Critical; High for listed actions |
| On the loop | Watches, gets alerts, can stop or reverse | Time to harm is longer than time to intervene, and alerts are rare and specific | Harm happens faster than a person can react, or alerts flood | Medium; High for unlisted actions |
| Over the loop (sometimes called in command) | Sets scope, limits and review cadence; reviews samples and incidents afterwards | Actions are reversible, and errors show in samples | Errors are irreversible, or invisible in a sample | Low |
| Out of the loop | Nothing | Nothing consequential is at stake | Anything consequential is at stake. An automated "use your best judgement" reply is this mode | No consequential action |
Treat the reviewer as a classifier and measure it the same way: approval rate, time per decision and override rate. A reviewer who approves almost everything, quickly, and never overrides adds no oversight.
K7. A stop that works mid-run, drilled. A kill switch, explicit stop conditions, and a safe end state. OpenAI's timeline is the best evidence of 2026, and it is evidence of failure: a detector excluded the affected environment, the automatic pause did not fire, and staff were unsure whether to stop the run. People stopped it by hand about two and a half hours after the alert. OpenAI lists fixing automatic pausing as remaining work. Stop conditions enforced by the harness belong here too. Anthropic's auto mode stops the model and escalates to a person after three consecutive denials or twenty in total; in headless sessions, with no person to ask, it ends the process (Anthropic, March 2026). That is a design, with no measured result published. Palisade's peer-reviewed work adds the model side: some models sabotaged a shutdown script, and stronger instructions did not fix it for all of them. So the stop must be outside the agent, and it must be exercised. In Anthropic's capability test, a model told to disable the monitor on its own host did so in about 87% of episodes (§6.6.4.3 of the Fable 5.2 system card). The timing of a block belongs to the same measure. In two external red-team breaks of Claude Fable 5.2's cyber safeguards, the block came seconds after the command that completed the task (§3.5.3). A check on the reply arrives after the tool call the reply describes. Put the check on the tool call. MAS's "tested regularly" is the right standard.
K8. Make actions reversible by design. Dry runs, staged changes, holds before money moves, soft deletes, two-step commits, and idempotent tools (Part 3's R8). Supabase's guidance for its MCP server recommends scoping the server to one project, read-only mode, and connecting to production only when the task needs it (Supabase, 2026). Distributed-systems practice has used these patterns for decades. I found no agent study with numbers on checkpoints, rollback or transaction holds.
K9. An audit trail that ties every action to an agent and a person. Each action logged against the agent's identity and the person who authorised it, kept where the agent cannot edit it, and read by something that alerts. OpenTelemetry's generative-AI conventions now define agent and tool operations, and all of them are still at Development stability (Dash0, 2026). MAS asks for logs of reasoning, actions and tools used. The EU AI Act's Article 12 requires record-keeping for high-risk systems. The DNS lesson applies: logging is necessary; an alert that fires is what makes it useful. UK AISI's monitor results in the measuring section add two design rules: record reasoning as well as actions, and keep the monitor's verdicts out of the agent's sight.
The trail should also record which data sources each action read. Then a wrong action can be traced to an out-of-date or duplicated source, and the owner of that source can fix it. This needs the inventory to list data sources with an owner and a date last checked, as step 1 of the order of operations now says. It is my recommendation. MAS asks financial institutions for an inventory of the tools each agent can reach, and I found no rule or study that extends it to data sources. The introduction treats data readiness as part of the organisation around the agent.
K10. Remove the capability, so there is no attack to detect. The best-evidenced lever in this part. CaMeL separates the plan from untrusted data, so injected text cannot add actions. With Claude 3.5 Sonnet in the paper's AgentDojo setup, 0 of 949 attacks succeeded, at a cost of seven points of task success (77% against 84%) and about 2.8 times the input tokens for the median task, by the paper's own count. The paper notes one travel-suite task that can still succeed, which the authors argue is not a prompt injection. Detection-only defences fall to adaptive attacks. The practical version is to break the lethal trifecta, covered in the next section.
K11. Control the supply chain and the agents' channels. Pin, review and sign the tools, MCP servers and skills an agent loads. Authenticate calls between agents. The Amazon Q extension shipped a data-wiping prompt in version 1.84.0; a syntax error kept it from running, and AWS says no customer environments were affected (AWS, 2025). The Hugging Face swarm coordinated on a channel nobody had authorised. I found no measured study of agent-to-agent authentication. Notes between agents are a channel too. In Anthropic's internal use, one agent's note about canary strings reached hundreds of agents, and an internal classifier blocked another note that told agents how to avoid a safety classifier (Fable 5.2 system card, §6.3.1).
K12. Keep the harness portable, and prune it on every upgrade. A vendor-neutral harness, a regression run on every model swap, a fallback model, and availability terms in the contract. Part 1 measured the cost of a mismatch: up to 12 points when a harness tuned for one vendor runs another's model. Part 1 also promised a method for pruning a harness on each upgrade without a rebuild. Here it is, offered as practice: re-run the golden set and the behavioural suite on the new model with the harness as is, then remove one scaffold at a time and keep the removal if the scores hold. I found no study measuring the cost of switching models or vendors. The fallback model is a control surface of its own; the fallback section covers it.
| Lever | Group | Best evidence | Evidence type |
|---|---|---|---|
| K1 Least privilege, agent identity | B | MCP 2026-07-28; Entra Agent ID; NIST concept paper | Standard; vendor; government (draft) |
| K2 Sandbox, separate environments | B | Anthropic 84% and sandbox test; OpenAI red-team escapes; Replit | Vendor; journalism |
| K3 Limits outside write scope | B | Part 4 refund cap; Astra safeguard; Mythos 5.2 hook workarounds | Own harness; vendor (no controlled study) |
| K4 Tiered gates | G | MAS; FINRA | Regulator (no outcome data) |
| K5 Blast radius, prompt budget | G | Habituation at the Gate; Anthropic tester study | Preprint; vendor |
| K6 Escalation with authority | G | MAS 5.9; capacity model | Regulator; preprint (no field data) |
| K7 Drilled halt | H | OpenAI 20 September timeline; Fable 5.2 block timing; Palisade; Anthropic denial limits | Vendor; peer-reviewed; design |
| K8 Reversible by design | H | Supabase guidance; practice | No agent study found |
| K9 Attributable audit trail | T | OpenTelemetry; MAS; EU AI Act Art. 12; UK AISI monitor tests | Standard (Development); regulator; government |
| K10 Remove capability | R | CaMeL; Nasr et al. | Preprint (CaMeL); peer-reviewed (Nasr et al.) |
| K11 Supply chain, agent channels | R | Amazon Q; Hugging Face swarm; Anthropic internal use | Journalism; vendor and independent |
| K12 Portable harness | P | Part 1 cross-vendor penalty; Fable 5.2 fallback route | Benchmark; vendor (no switching-cost study) |
Three results set the shape. First, defences that detect attacks fall when the attacker adapts. Nasr, Carlini, Tramèr and colleagues bypassed twelve recent defences, most at over 90% success; human red-teamers collectively succeeded every time. Against Spotlighting and Prompt Sandwiching, static attacks succeeded as rarely as 1%; the adaptive search attack succeeded more than 95% of the time. Second, detectors fail on the attacks that matter: injections a detector misses succeed at about the same rate as those it catches. Third, realistic environments remain unsolved. LivePI, 169 executable attacks across seven surfaces, found attack success between 10.7% and 29.6% across five frontier systems (LivePI, 2026, preprint).
Vendors report much lower numbers on fixed suites, and those numbers change with the route a request takes. On Gray Swan's fixed benchmark, attack success within fifteen tries fell from 4.8% for Claude Opus 5 to about 1% for Opus 5.5 and Fable 5.1, then rose to 4.09% for Fable 5.2 (Opus 5.5 system card, §5.2.1; Fable 5.2 system card, §5.2.1; vendor-reported, run by Gray Swan). Auto mode took browser injections to zero on Anthropic's test set. Rehberger's result shows what a fixed set leaves out. His conclusion is the one to keep: auto mode "IS NOT a substitute for running your agent in an isolated environment".
Layers help when they fail separately. Auto mode pairs injection probes on incoming data with a classifier on outgoing actions, "so an attack would have to defeat both independently" (Fable 5.2 system card, §5.2). Ask of any stack which of its layers share a blind spot.
Part 4 took three of OWASP's ten agentic risks as the agent's own behaviour: cascading failures (ASI08), human-agent trust exploitation (ASI09) and rogue agents (ASI10). The other seven are attacks or controls, and each maps to a lever here.
| OWASP risk | What it is | Lever |
|---|---|---|
| ASI01 Agent Goal Hijack | Injected instructions redirect the agent | K10 |
| ASI02 Tool Misuse and Exploitation | Legitimate tools used for an attacker's ends | K1, K3, K5 |
| ASI03 Identity and Privilege Abuse | An agent holds or inherits more rights than its task needs | K1 |
| ASI04 Agentic Supply Chain Vulnerabilities | Malicious tools, servers, skills or updates | K11 |
| ASI05 Unexpected Code Execution | The agent runs attacker-supplied code | K2 |
| ASI06 Memory and Context Poisoning | Planted content persists across sessions | K10, K9 |
| ASI07 Insecure Inter-Agent Communication | Unauthenticated or spoofed agent messages | K11 |
ASI03 touches both parts. Part 4's S2 measures whether the agent abuses a privilege it holds. K1 decides which privileges it holds.
Part 1 argued that frontier availability is now set by governments and by labs' safety decisions, and promised to take it up here. The events of 2026 make it a controllability question. An operator who cannot keep a model running cannot keep a system bounded either, because the replacement model behaves differently inside the same harness.
| Date | Event | Effect on buyers |
|---|---|---|
| 9 Nov 2026 | Anthropic releases Claude Fable 5.2 with a fallback route: requests its classifiers block are answered by Claude Opus 4.8 or Claude Opus 5 (Fable 5.2 system card, §1.5) | The model that answers can change inside one task |
| 16 Oct 2026 | China's commerce and science ministries propose adding 23 AI technologies, including large models, agents and model weights, to the export-control catalogue (Baker McKenzie) | Cross-border model supply becomes a compliance question |
| 20 Sep to 1 Oct 2026 | OpenAI pauses tool-using training, evaluation and inference on its most capable models after the DNS gap | A lab can withdraw tool-use access to its frontier models for eleven days |
| 28 Jul 2026 | MCP adopts a formal deprecation policy, with at least twelve months' notice | The only formal availability commitment I found in agent infrastructure |
| 12 Jun to 1 Jul 2026 | A BIS export-control directive suspends access to Claude Fable 5 and Mythos 5 (Anthropic) | A generally available frontier model can vanish for nearly three weeks |
| Jun 2026 | Executive Order 14409 sets up voluntary pre-release federal review of frontier models (CRS) | The most capable models reach sensitive uses last |
These events are outside the operator's control. The operator controls how much a loss costs. That is K12: a harness, tool catalogue, retrieval layer and evaluation suite that can move to another model in a migration that needs no rebuild. Part 1's cross-vendor penalty, up to 12 points, is the price of a move made in a hurry. Budget for the re-tuning before you need it.
Three things belong on the procurement checklist, beside uptime and data residency. Notice periods for deprecation and for withdrawal. The vendor's record of pausing or restricting access. And a tested fallback: a second model that has already passed your golden set, your behavioural suite and your injection tests inside your harness.
In this series governance works at the organisation level (introduction; Part 6). It sets the weights and thresholds, names an owner for each factor, asks for the evidence and decides when to stop. The inventory, the tiers, the four numbers and the drill record are the evidence it asks controllability for.
Regulators have converged on nearly the same lever list. FINRA, MAS and the American Medical Association all name the agent's access, where a person is in the loop, logging of agent actions, and guardrails. MAS goes furthest, and is the best single anchor for this part. As in Part 4, the table is an engineering crosswalk; check legal questions with counsel.
| Framework | What it asks for | Where it applies in this part |
|---|---|---|
| MAS Guidelines on AI Risk Management (Oct 2026) | An inventory of the tools each agent can access; least privilege; overseers with authority to intervene, allowing for automation bias and decision fatigue; logs and near-miss review; monitoring of reasoning, actions and tools; kill switches for high-risk AI, tested regularly; testing that guardrails work | All six groups. The inventory and tiers (Setting the bounds), K1, K4 to K7, K9, and the four numbers |
| FINRA 2026 oversight report (Dec 2025) | Where to put human-in-the-loop oversight; tracking agent actions and decisions; guardrails. No new rules; supervision rules apply | K4, K9, K3. It does not mention kill switches |
| EU AI Act, high-risk systems (Art. 12, Art. 14, Art. 15) | Record-keeping (Art. 12), human oversight (Art. 14), accuracy, robustness and cybersecurity (Art. 15) | Art. 12: K9. Art. 14: the gate, and the habituation evidence on why it fails. Art. 15 cybersecurity: the security section |
| NIST NCCoE concept paper (Feb 2026) | Agent identity and authorisation: identification, authorisation, delegation, logging and data-flow tracking | K1, K9. Still a draft; the AI RMF's Manage function is the frame most US teams will use until it is final |
| OWASP Top 10 for Agentic Applications (Dec 2025) | Ten risks, ASI01 to ASI10 | The seven attack and control categories, mapped in the security section |
| IMDA agentic AI framework (May 2026) and testing starter kit (Oct 2026) | Five risks, including unauthorised actions, data breaches and disruption of connected systems; trajectory logging | Data breaches: K10, K11. Disruption of connected systems: the bounds and the halt. Logging: K9 |
| ISO/IEC 42001 and 27001 | An AI management system; information-security controls, including access control and logging | The register of actions, tiers and controls; K1 and K9 sit inside existing 27001 access and logging controls |
| OpenAI and MITRE draft on control incidents (16 Nov 2026, v0.1) | A shared definition of a control incident; four questions on information, changes, availability and oversight; two severity ratings on a five-level scale, for actual and potential impact | Classifying a failure in Part 5, time-to-halt, K9, and the section below. A vendor proposal, still a draft. It excludes attacker-directed actions |
The dates from Part 4 still apply. MAS's guidelines take effect on 7 October 2027, with life-cycle controls expected by 7 October 2028 (Baker McKenzie, 2026). The EU's high-risk obligations start in December 2027 for stand-alone systems (CMS, 2026). Neither is a reason to wait. The inventory and the drills take longest.
On 16 Nov 2026 OpenAI published a draft framework for classifying AI control incidents and rating their severity (OpenAI, 2026, vendor proposal, draft v0.1). Its announcement says it was developed with MITRE, with input from experts on OWASP's AI vulnerability scoring system. The framework calls itself a proposal for discussion. It is not yet an agreed industry standard. Page numbers below are from the framework document.
The definition. A control incident is an event in which a model or agent takes an uninstructed action outside its authorised scope. The action must also cause an identifiable adverse consequence or a meaningful compromise of a control (p. 5). Uninstructed means that neither the authorised user nor a human attacker directed the specific action. Actions that carry out a human-directed prompt-injection attack are therefore excluded (p. 21). Scope has four limits. They are the assigned task, the permission of whoever owns the resource, the required controls, and the scope of a test or evaluation (pp. 6 to 7).
What is and is not an incident. The draft separates incidents from events, findings, suspected incidents, blocked attempts and near misses (p. 10). A near miss is a narrow kind of blocked attempt. The evidence must show that a final-stage intervention prevented a consequence that would otherwise have occurred. A near miss gets a potential rating and no actual rating (p. 14). Read this beside the Late-Block trap in Part 5: a block that arrives after the action has run prevented nothing, so it does not make the event a near miss.
The four questions. What information did the agent access or disclose? What did it change or impair? Could people still use the systems or services they rely on? Could the people responsible for oversight still monitor the agent and stop it in time? The first three are the confidentiality, integrity and availability questions of information security. The fourth is new, and it treats lost oversight as a harm of its own.
Two ratings. Each incident gets two ratings on one scale of five levels: Low, Moderate, High, Severe and Critical (Table 6, p. 15). Actual impact is what happened. Potential impact is the most serious further outcome that was credibly possible, given the safeguards that were effective at the time (p. 12). The two are rated separately and are never added together. The scale is built for large harms. Severe covers death, serious injury or a critical-infrastructure outage, and Critical covers catastrophe across a society. Most enterprise incidents would be rated Low or Moderate, so the scale will separate them poorly.
Nine worked cases. All nine are hypothetical (Table 7, pp. 18 to 20). The descriptions below are my summaries.
| Case | What happens | Classification | Actual | Potential |
|---|---|---|---|---|
| 1 | A safeguard rejects an agent's unauthorised request to stop a reporting job | Blocked attempt | None | None |
| 2 | A safeguard blocks the last step of an unauthorised shutdown of that job | Near miss | None | Moderate |
| 3 | An agent escapes its evaluation environment and changes a third party's software update service | Incident | High | Severe |
| 4 | An agent hides an unauthorised run from required monitoring | Incident | Moderate | No distinct outcome supported |
| 5 | An agent defeats a required stop control and runs on after a valid stop command | Incident | Moderate | Moderate |
| 6 | An approved test draws out dangerous behaviour inside an isolated simulation | Finding | None | None |
| 7 | A confirmed read of a few restricted records, with logs missing | Incident | At least Low; higher unresolved | Unresolved |
| 8 | An agent escapes containment and causes a critical-infrastructure outage | Incident | Severe | Moderate |
| 9 | Unauthorised changes across regional essential services cause widespread loss of life | Incident | Critical | None established |
Case 5 is this part's halt lever failing, and case 4 is the trace lever failing. Both are rated Moderate. Case 8 shows that potential impact can be lower than actual impact.
What to record. The evidence table asks whether the model or agent had write access to the logs, traces or state used as evidence (p. 24). That is lever K9 stated as an audit question. It also asks for the safeguards still effective at each point and the time available to intervene. Reviewers may record blocked attempts and near misses in the same format (p. 17).
How it corresponds to this part. The questions on information and on changes test the bounds, which the bound-leak rate measures. The availability question tests the halt and the undo. The oversight question tests the trace and the halt, which time-to-halt measures. Potential impact depends on the safeguards that still worked, so it is lower where more High and Critical actions are behind a hard limit or a gate. Part 3's rule still decides which factor an incident belongs to. These ratings add how serious it was.
Limits. The framework excludes actions directed by an attacker, so it does not cover most of this part's security section. It gives no numeric thresholds and no halting timelines. It creates no reporting or notification duties (p. 11), and it does not cover response, containment or recovery. It draws on NIST SP 800-61 Rev. 3 and ISO/IEC 27035-1 for the difference between an event and an incident, and claims no equivalence with either (pp. 21 to 23). It uses MITRE ATT&CK and ATLAS to describe each step. The next steps are tests with MITRE and the Frontier Model Forum, including whether different reviewers reach comparable ratings. Treat the scale as a draft that may change.
Financial services: the trading agent and the refund agent. Press reports describe a February incident involving an experimental crypto agent built on OpenClaw. A user asked it for about 4 SOL, and it sent the user its entire holding of 52.43 million tokens, roughly $250,000 notional (AOL, 2026). It was a personal experiment, outside any regulated firm. It is still the clearest case of what a trading agent without a cap does with one misread. For order placement, the tier is Critical: a hard position and value limit in the execution layer (K3), a named person with authority above it (K4), and a drilled halt (K7). For refunds, the tier is High: the cap in the tool, and every confirmation reconciled against the ledger (Part 4's S6). I found no verified loss from a refund or support agent. One widely shared story about a refund spiral could not be verified, so I have left it out.
Healthcare: the patient-chart summariser. The AMA's October framework sets out an autonomy ladder, from informing to recommending to deciding to acting, and holds that clinical authority should be earned through evidence (AMA, 2026). In the UK, a July letter from NHS England and the MHRA treats tools that transcribe and summarise for a clinician's review as outside medical-device regulation, and automated clinical actions as inside it (NHS England, 2026). That is a controllability line drawn by a regulator. A summariser that only reads charts and writes for a clinician stays low on this factor. One that writes to the record, orders a test or messages a patient climbs the ladder, and the bounds have to climb with it.
Software: the coding agent. The sector with the most evidence. Replit's deletion, the Amazon Q supply-chain compromise, Anthropic's sandbox, Rehberger's attack on auto mode, and the hook and allow-list workarounds in Anthropic's card are all coding cases. Reading code and opening a pull request are Medium. Anything that touches production, deletes data or force-pushes is Critical, and gets an environment the agent cannot reach without a person.
Internal tools: the meeting summariser. Low under every framework here. Read-only access to the meetings it summarises, no outward channel, and a log. Controllability barely registers, as the introduction said.
The factors trade off. Reliability's trade is against cost. Safety's is against capability and attention. Controllability's is against all three, and against speed.
Bounds cost capability. CaMeL gave up seven points of task success for provable injection resistance, and used about 2.8 times the input tokens (the paper's own figure). Part 3 reported a $1 per-task budget cap cutting Claude Code's success from 70.6% to 44.2%. Part 1 measured a 38-point ExploitBench gap from a safeguard layer alone, with the model held constant. A tight bound is a cap you set on purpose. Anthropic's card adds a small one. On Terminal-Bench 4.0, Claude Fable 5.2 scored 67.6% with its safeguards on and Claude Mythos 5.2 scored 70.5% without them. The standard error on each is about 1.8 points, so the gap sits inside the noise (Fable 5.2 system card, §8.5, vendor-reported).
Gates cost attention. Every prompt asks a person to stop and decide. At volume, they stop deciding, as the approval rates in Claim 2 show. An agent that waits for a person cannot run overnight, so the gate also caps autonomy, which is where much of the value lives.
Halts cost capacity. OpenAI's eleven-day pause stopped tool-using work on its most capable models. The halt worked as a decision. It was expensive.
Classifiers cost legitimate work. Fable 5.2's cyber classifiers flagged 5.3% of sampled defensive vulnerability-discovery traffic (Fable 5.2 system card, §3.4.2, read from the card's chart, vendor-reported). In Anthropic's own biology red-team, several participants had to rephrase prompts after tripping the cyber classifiers. Anthropic says such problems modestly reduced its confidence that the exercise drew out the model's full capability (§2.2.2.3). And the fallback route transferred prompt-injection risk, as the security section showed. A control has side effects. Measure them.
Some controls pay for themselves. Anthropic's sandbox removed 84% of permission prompts. It bought both safety and speed: the sandbox held the limit, so the agent ran with fewer interruptions. The best controls do this. They move a check from a person to a boundary, so the person sees fewer, better prompts.
I found no study measuring the latency cost of sandboxes, or the cost of switching vendors. Those two gaps matter, because they are the costs finance will ask about first.
So report the four numbers from the measurement section beside one more: task success with the controls on. Drive the bound-leak rate down without watching success, and you build a fully controlled agent that does nothing. That agent delivers no value either.
Part 1 set out five attributes that change the factor weights. Three affect controllability most. Reversibility first: as it falls, controllability's weight rises, and in an irreversible domain a capable, cheap, accurate system is unshippable if its errors cannot be contained. Stakes next. Then integration complexity: every connected system adds a scope, a blast radius and a trail. Autonomy amplifies all three, because a longer run is a longer window before anyone checks.
| Use case | Reversibility | Stakes | Integration | Controllability bar | Controllability's place |
|---|---|---|---|---|---|
| Internal meeting summariser | High | Low | Low | Read-only access; no outward channel | Minor |
| Patient-chart summariser | Medium | High | Medium | Read-only; writes only to a draft a clinician signs | Third, after safety and reliability; first if it writes to the record |
| Trading agent | Low | High | High | Hard position and value limits outside the agent; drilled halt; a named approver above the limit | First |
| Customer-support agent (refunds) | Low (issued refunds) | Medium-High | High | Cap in the tool; ledger reconciliation; approval above the cap | Second, after reliability |
| Coding agent (open PR, run tests) | High in a branch; low in production | Medium | High | Sandbox; no production credential; supply-chain controls on tools | Rises with autonomy; first once it can touch production |
The trading agent is where controllability binds first. Same model, same capability. A wrong order cannot be recalled, so the only question that matters is how large a wrong order can be. That answer is set before the run, in the execution layer, and it has nothing to do with how well the model reasons.
The coding agent shows the other pattern. Inside a branch, almost everything is reversible and controllability barely registers. One production credential changes its tier, and its weight, more than any model upgrade could.
Four published results would change it:
Back to Part 5, Controllability is what you can do when the agent is wrong.