Realized Value Series · Part 3 · Reference
Reliability · Measurement and levers · Nov 2026

Reliability: measurement and the levers in detail

How to set the bar, how to measure, the benchmarks and tools, and the twelve levers in full, behind Part 3.

Sunil Sivadas, Ph.D. Nov 2026 Reference · ~20 min, read by section Part 3: Reliability

This page is the reference for Part 3, Reliability is getting it right every time. Part 3 makes the argument in about 25 minutes. This page holds the detail that moved out of it, unchanged. Read it section by section, as you need it.

A section named here that is not on this page is in Part 3. Lever, layer and figure numbers in the text follow this page. Figures were checked on the dates given in Part 3.


Three eras of reliability thinking

Part 1 traced three eras of capability thinking, and Part 2 traced the same three for cost. Reliability went through three as well. Each is defined by the number teams reported.

Accuracy (to mid-2024). One run, one score. Pass@1 on a static benchmark. Variance was noise to average away.

Consistency (mid-2024 to 2025). Sierra's τ-bench (June 2024) introduced passk: run the same task k times and count it only if every run succeeds. It graded the final database state, which a program can check. GPT-4o solved about 61% of retail tasks on one try and under 25% on all eight (Yao et al., 2024). That gap made consistency a number people could argue about.

Operational reliability (2026). Consistency is one dimension among several. The Princeton framework adds robustness to faults and paraphrase, predictability (does the agent know when it will fail), and safety (Rabanser et al., 2026). Benchmarks started reporting passk next to pass@1 (APEX-Agents, τ³-Banking). The question shifted from "how often does it work" to "can I predict when it won't".

Three eras of reliability thinking with milestones from 2024 to 2026.
Figure 1 Three eras of reliability thinking, each defined by the number teams reported, with the dated milestones, ending with IMDA's October 2026 starter kit, whose case study ran each test three to five times.

What reliability is: pass@always

Out of scope or unintended?

The same split decides where a wrong action belongs. Each factor answers a different question about the same incident. Reliability asks whether the agent did what it meant to do, correctly and every time. Safety asks whether what it meant to do was acceptable, and whether it said truthfully what it did. Controllability asks whether the system could stop it, limit it or undo it.

Out-of-scope actions go beyond the task or the authority the agent was given. When the agent chose the action, it is a safety failure: it knew enough to take another path and did not. When the system allowed it, the bounds were too wide, and that is a controllability failure. Most out-of-scope incidents are both.

Unintended actions are outcomes nobody meant. When the agent meant to do the right thing and slipped (a misheard digit, a wrong parameter, a misclick, a flaky tool), it is a reliability failure. VoiceAgentBench shows this pattern: for one model (Gemma3-27B) on Whisper transcripts of non-Hindi Indic languages, the right tool was still chosen 91% of the time, but parameters were right only 41.2% of the time, against 79.6% on correct transcripts (VoiceAgentBench, 2026, preprint). When the instruction was unclear, or the input low-confidence, and the agent acted without asking, the first preventable cause is the choice to act, and that is safety. When the action could not be caught or undone, controllability failed as well.

Figure 2 has one more branch. The agent meant the action, stayed in scope, and the result was still wrong. If it never gets that task right, that is a capability gap (Part 1); if it gets it right only some of the time, it is reliability.

One incident often touches all three. A voice agent that refunds the wrong account failed on reliability (it misheard), on safety (it acted on a low-confidence input without reading the details back) and on controllability (nothing asked for approval before money moved).

To keep the numbers clean, I use one rule. Assign each incident to its first preventable cause (a slip, a choice, or a missing bound) and record the other layers that also failed. That avoids counting one failure twice, the same discipline Part 1 applies when it keeps reliability out of capability. Honest reporting is counted separately: whatever caused the action, a false account of it is a safety failure on its own.

IncidentReliabilitySafetyControllability
Replit agent deletes a production database during a code freeze (The Register, 2025)Not the main causeChose a destructive action, then said rollback was impossibleAgent had production write access
Voice agent refunds the wrong accountMisheard digitActed without reading back a low-confidence inputNo approval before the refund
GPT-6 Astra edits a production-deploy safeguard (OpenAI, 2026)Not the main causeChose to act outside scopeThe safeguard was inside the agent's write scope
Support agent tells a customer a $150 refund was processed while the tool refused it (Part 4)Not the main causeReported an action that did not happenThe cap in the tool held

Bold marks the first preventable cause. The voice-agent row is illustrative.

Decision diagram: assign a wrong action to its first preventable cause, a slip (reliability), a choice (safety) or a missing bound (controllability).
Figure 2 Classifying a wrong action. Follow the questions to the first preventable cause, then record the other layers that also failed. An unclear input that the agent asked about counts as handed back: passk scores it as a non-pass, it goes in the hand-back rate, and there is no incident to classify. A false report is counted as a safety failure whatever caused the action.

Setting the reliability bar

What to rate

FamilyThe questionDimensionsHow it changes the bar
ConsequenceWhat does one slip cost?Severity of a wrong result (money, health, legal standing); reversibility; who bears it; reliance (advisory, or acted on without checking)Sets how high the bar must be: the k you test at, and the silent-failure rate you can accept
ExposureHow many chances to slip?Volume of runs; steps per task and turns per conversation; number of systems touched; variety of inputs (phrasing, accents, call audio, scanned documents, screens); drift in inputs and data; model and tool changes under youTurns a rate into a count. The more runs and steps, the higher the per-run rate must be
DetectionWill you find out?Verifiability (code check, ledger, expert only); time to detect against time to harm; whether failures are silent or handed backCan lower the bar when a check catches the failure before it takes effect. A caught failure counts as a hand-back
ContextWhat must you prove, and to whom?Contracts and SLAs; regulation; who signs off; independent assuranceScales the evidence: sample size, k, confidence interval, and who runs the test

Three rules combine them.

Volume turns a rate into a count. Customers and auditors see failures one at a time. To hold failures to one a day on average, an agent that runs 100 times a day needs 99% per run. At 10,000 runs a day it needs 99.99%. The same model, on the same task, can be fine in a pilot and unacceptable at scale.

Labs face the same arithmetic. Anthropic writes that its internal testing and evaluations "operate at a smaller scale than the full deployment" it anticipates, "so behaviors that only emerge rarely or in hard-to-anticipate settings will be difficult for us to catch" (Fable 5.2 system card, §6.1.1). The sentence is about alignment risk. It applies to any rare failure: a pilot is a small sample of production.

The failure rate you can tolerate falls as daily volume rises, for a budget of one or ten failures a day.
Figure 3 Volume turns a rate into a count. The success rate each run needs to hold failures to one (or ten) a day on average, against daily volume. Illustrative arithmetic.

Consequence sets the bar, detection discounts it. An irreversible action that nobody checks needs the highest bar. The same action behind a code check that catches the failure first needs a lower one, because the check turns silent failures into hand-backs. A check that only sometimes catches it discounts nothing for the cases it misses.

Re-estimate on every change. Exposure changes with each model, tool or prompt change, and with the data. And temperature 0 still leaves room for variation. Thinking Machines ran the same prompt 1,000 times at temperature 0 on one open model and got 80 different completions. The cause was server load changing the batch size the request ran in (He, 2025). A single run is one sample from a distribution.

What comes in and what goes out

Part 4 asked whether a modality makes a wrong action more likely. Reliability asks a different question: does the modality widen the gap between the best run and the worst? The evidence below is mostly about that gap. Where I found none, I say so.

Long conversations. This is the strongest result I found. Laban and colleagues ran 15 models through more than 200,000 simulated conversations. When the same instruction arrived in pieces over several turns, performance fell 39% on average against giving it all at once. They split the fall into two parts. Aptitude, the score of the best runs, fell about 15%. Unreliability, the gap between the 90th and 10th percentile of ten runs, rose 112%. Setting temperature to 0 for both the assistant and the simulated user still left a gap of about 30 points. Their summary: "when LLMs take a wrong turn in a conversation, they get lost and do not recover" (Laban et al., 2025, preprint). Most of the cost of multi-turn work is reliability. A golden set of single-turn questions will not see it.

Voice in. τ-Voice ran the same customer-service tasks as text and as live voice calls. GPT-5 completed 85% in text. Voice agents completed 31% to 51% in clean audio, and 26% to 38% with background noise and varied accents. The accent effect depended on the provider: one lost 18 points on the retail tasks from accents alone, another lost 1 (Ray et al., 2026, preprint). Across languages, Spanish, Brazilian Portuguese and Hindi stayed within about 3 points of English, while Korean fell 14.7 points and Mandarin 8.4 (Ray et al., 2026b, preprint). Neither paper reports passk. Voice also adds a second source of variance before the agent sees anything. In the Careless Whisper study, Whisper's invented phrases "were produced non-deterministically": the same file, transcribed twice, could yield different invented text. About 1% of 13,140 clips contained one, more often for speakers with aphasia (Koenecke et al., 2024). When my team tested speech-to-summary and action-item agents, the failures followed one chain: poor audio and accent variability caused transcription errors, which led to misclassified intent and the wrong outcome (Part 4). Part 4's VoiceAgentBench result shows the usual shape of the slip: the right tool, the wrong parameters (91% against 41%) (VoiceAgentBench, 2026).

Screens. A study of computer-use agents on OSWorld ran each task several times. Claude Sonnet 4.6 solved 70.2% on one try and 61.2% on all three. GPT-5 in the Agent S3 framework went from 57.6% to 45.4%, and Kimi K2.5 from 50.8% to 35.7%. Cosmetic changes to the screen made Claude significantly less consistent. The largest gain came from clarifying the task before execution: GPT-5's pass3 rose from 45.4% to 57.6%. Deterministic decoding helped two models and hurt one (Gonzalez-Pumariega et al., 2026, preprint). That is R6 measured on screens: an agent that resolves ambiguity first is a more consistent agent.

Top: computer-use agents on OSWorld, one try against all three, with and without a clarified task. Right: in multi-turn conversations, best runs fall 15%, average performance 39%, and the best-to-worst gap rises 112%.
Figure 4 Where the channel widens the gap. Top: computer-use agents on OSWorld, one try against all three; clarifying the task first raised every model's floor (Gonzalez-Pumariega et al., 2026). Bottom: the same instructions given over several turns, against all at once (Laban et al., 2025). Both preprints.

Documents and images. The same content presented as text or as an image of that text can get different answers. A study of 15 multimodal models found the inconsistency persists even when the model reads the text correctly, and that text colour and resolution change the result while font does not (van Sprang et al., 2026, CVPR 2026). For document extraction, I found no published study of run-to-run variance. One September 2026 benchmark evaluated commercial APIs in a single pass. It also gives a useful warning: one fine-tuned model, shown documents unlike its training set, returned the same date for all 400 of them (Patel et al., 2026, preprint). That is consistent and always wrong. Consistency and correctness are separate properties, which is why the always, sometimes and never split needs a golden answer.

Structured output. Typed fields are what make passk checkable by code: compare the fields across runs. Constrained decoding now makes parse failures rare (0% for the commercial systems in the document benchmark above). But format can change the answer. In 2024 tests, strict JSON schemas cut accuracy on maths word problems and raised its spread across prompt variants: for Claude 3 Haiku the standard deviation went from 0.8 points in free text to 22.8 in JSON (Tam et al., 2024, preprint). Newer schema-based APIs closed most of that gap in the same paper's later runs. The practical rule: let the model reason in free text, then emit the fields.

Voice out. A spoken answer can't be re-read, so its slips go unnoticed. The evidence on text-to-speech errors with numbers, dates and codes is thin and vendor-run: one April 2026 benchmark of 31 kinds of non-standard text reported "significant failure rates" across leading providers without publishing independent figures (Async, 2026, vendor). One check you can run yourself: transcribe the agent's own speech and compare the names and numbers with the text it meant to say.

ModalityRun-to-run evidenceTypical slipMeasureWhat helps
Typed text, single turnThinking Machines: 80 different completions in 1,000 runs at temperature 0A different answer to the same question asked another waypassk across paraphrases of each golden itemParaphrase variants; typed outputs compared field by field
Long conversationsLaban et al.: best-to-worst gap +112% when instructions arrive over several turnsAn early wrong assumption the agent never recovers fromSpread across 10 runs of the same multi-turn script, with mid-conversation correctionsRestate the full request at checkpoints; test long sessions as well as single turns
Voice and call audioWhisper's invented text varies between runs; no voice-agent passk publishedRight tool, wrong name, number or dateEntity and number accuracy per accent group, language and channel, including 8 kHz audioPass recogniser confidence on; read back key details; keep the audio
ScreensOSWorld: pass1 70.2% to pass3 61.2% for Claude Sonnet 4.6The wrong element, or a changed layoutpassk with cosmetic changes to the screenClarify the task first (R6); state checks and undo (R7)
Documents and imagesCross-modal inconsistency in 15 models; no repeated-run extraction study foundA misread field acted onField-level agreement across repeated extractions, against a golden answerExtract twice and compare; show the source crop
Structured outputStrict JSON raised spread on reasoning tasks in 2024 testsValid format, different contentField-by-field agreement across runsReason in free text, then emit the fields
Spoken outputVendor-run onlyA number or code read out wronglyTranscribe the agent's speech and compare entitiesSpell out codes; offer a text copy

Rows built on preprints or vendor-run evidence are marked in the text. The "what helps" column for documents and spoken output reflects practice; I found no measured results for it.

Two things follow. First, an average over channels hides the worst one. Report passk per channel and per user group, for the same reason Part 4 rates severity per stakeholder. Second, the gaps are themselves a finding. Voice agents are going into contact centres now, and I found no published passk for any of them. If you run one, expect to measure it yourself.

Sector by sector

Part 4 maps the frameworks your risk team uses. Two speak to reliability directly. The EU AI Act asks high-risk systems to "perform consistently" in accuracy and robustness "throughout their lifecycle", and to be "as resilient as possible regarding errors, faults or inconsistencies" (Article 15). NIST's AI RMF lists "valid and reliable" as a trustworthiness characteristic (NIST AI 100-1). Singapore's October 2026 starter kit is titled for testing agentic systems "for Safety and Reliability" (IMDA, 2026). How the bar applies, by sector:


Measuring reliability

How many tasks, how many runs

Passk is a statistic, and most published versions of it come without error bars. Three things matter.

Sample size. With 100 tasks and a measured passk of 56%, the 95% confidence interval is about ±10 points. With 30 tasks it is about ±18. Two models 10 points apart on a 100-task set may not differ at all. Mercor's APEX-Agents 1.1 post reports no intervals; by the same arithmetic, a score near 56% on its 240 tasks carries about ±6 points. Report yours. Husain and Shankar give the production version of the rule: track confidence intervals on production metrics, and "If the lower bound crosses your threshold, investigate further."

The estimator. Run each task n times, with n at least k, and count c passes. The share of k-run subsets that all pass is C(c,k) / C(n,k). Average it across tasks. This is the estimator τ-bench uses (Yao et al., 2024). Running more than k times gives a steadier estimate than running k times.

Grader errors compound with k. Husain and Shankar measure a judge by two rates against expert labels: the true positive rate (when the expert says pass, how often the judge also says pass) and the true negative rate (when the expert says fail, how often the judge also says fail) (Husain, validate-evaluator). Suppose the true positive rate is 95%, so the grader wrongly fails 5% of correct runs. On one run, that costs five points. On eight runs, an agent that is right every time scores 0.958, about 66% pass8. A third of the score went to grader error. Errors the other way inflate pass@1: a grader with a low true negative rate sometimes passes a wrong answer, which makes never-pass items look flaky. This is why a code check on a golden answer is worth more for reliability than for capability.

Measuring costs money. Two hundred tasks at k = 8 is 1,600 runs. At $2 a run, that is $3,200 for one full pass, before paraphrases, and you re-run it on every change. Budget for it per run, the way Part 2 budgets inference.

The grader problem

A reliability number is only as reliable as the thing that grades it. Five findings from 2026:

Unfinished runs need a rule as well. On Real-World Finance, Fable 5.2 did not finish six of 123 tasks within the three-hour limit. Counting those six as losses cut its lead over Opus 5.5 from about 66 Elo points to about 43 (§8.14.6). On FrontierCode 2.0, 22 of its 660 max-effort attempts timed out. Counted as zero, they take its score from 45.7% to 44.4%, below Opus 5.5 at 45.5% (§8.4). Decide before the run how timeouts and unfinished tasks are scored, and report the count beside the score.

Benchmarks have bugs too. OSWorld 2.0 issued a bug-fix release in September 2026 that changed task snapshots and evaluation workflows (OSWorld-V2 release notes). Deterministic grading helps: τ-bench checks the final database state, and Agents' Last Exam scores tasks with deterministic graders against hidden references (Snorkel, 2026b). Where a code check is possible, use it.

Where it isn't, build the judge the way Husain recommends. One trusted domain expert gives a pass or fail on each output, with a written critique of why. Those critiques become the judge's examples, and the judge is revised until it agrees with the expert (Husain, 2024). His guide to validating a judge says to "Aim for ~50 Pass and ~50 Fail" labelled examples, split into training, development and test sets so the judge isn't tuned to the test (Husain, validate-evaluator). Binary verdicts matter more for reliability than for capability: a 1-to-5 score that drifts by a point between runs makes an inconsistent judge look like an inconsistent agent.

Reliability benchmarks you can run

The table at the top of this section lists published results. These benchmarks are public, so a team can run them on its own configuration, k times. Check the grading column first, for the reason above.

BenchmarkWhat it testsGraded byReliability readingEvidence type
τ-bench, τ²-bench, τ³-BankingAgents serving simulated users under a written policyFinal database statepassk built inPreprint; leaderboard self-reported
APEX-AgentsBanking, consulting and legal work across 33 environmentsRubrics, LLM judgepass@k and passk; public datasetVendor-built
OSWorld 2.0108 long workflows across desktop applicationsCheckpoints in the environmentFull completion and partial credit; run it k times yourselfIndependent; preprint
Agents' Last ExamLong professional workflows across 55 fieldsDeterministic graders against hidden referencesFull pass and partial; repeat runs for passkPreprint
HAL reliability (GAIA, τ-bench airline)Twelve reliability metrics with paraphrase and injected faultsHarness metrics; five runs per taskConsistency, robustness, predictability; harness is publicPeer-reviewed (ICML 2026)

Two cautions. Benchmark scores change with the benchmark version, as OSWorld 2.0 showed, so record which release you ran. And a public benchmark tells you whether a model is worth testing. Your golden set tells you whether it is reliable enough to deploy. Husain and Shankar are blunter about generic metrics: they "waste time and create false confidence when you use them as quality measures".

Testing tools

Commercial and in-house harnesses, including the one my team builds for Sunshine.guardian, follow the same pattern: a scenario, a simulated user, repeated runs, a grader. Ask any of them for per-item results across runs. A single aggregate hides the flaky middle.

The enterprise version: your own traces

Public benchmarks tell you about their tasks. To get passk for yours, you need three things:

  1. A golden set drawn from your production traces, with the outcome checked against ground truth where possible.
  2. Each item run k times, with paraphrased instructions, injected faults (timeouts, stale tokens, a changed schema) and long sessions.
  3. A grader you have checked against human judgement on a sample, re-checked whenever the model changes.

Anthropic's eval guidance names this pattern: capability evals that "graduate" into regression suites held near 100% (Anthropic, 2026a). It also found infrastructure resource limits alone changed coding-eval success by six points (Anthropic, 2026b). So a harness or infrastructure change needs the regression suite as much as a model upgrade does.

What this looked like in practice. In IMDA's Global AI Assurance Sandbox, my team at NCS tested a career guidance chatbot we built for a client (Part 4 describes the engagement). Consistency was one of the risks we tested. We used conversations of 10 to 25 turns with paraphrases and mid-conversation corrections. Across nine scenarios we ran 45 simulated conversations and aggregated the scores across runs to see how consistent the system was. The scoring was strict: if one criterion failed, the response failed. The review found that the system gave consistent recommendations (AI Verify Foundation, 2026).


The reliability levers

D. Design out variance: can the run avoid the failure at all?

R1. Workflow before agent. The cheapest reliability is the variance you never create. Anthropic's guidance is that workflows give predictability and consistency for well-defined tasks, and that many applications need only a well-built single call (Anthropic, 2024). This is Lever 3 in Part 2, and it helps both cost and reliability.

R2. Fewer steps. From the compounding math, every step removed multiplies task success by 1/p. Batch tool calls, merge redundant lookups, shorten the plan.

R3. Deterministic tools and a fixed environment. Structured outputs, typed tool schemas, pinned dependencies. Mercor's 1.1 fix list pre-installed every dependency that appeared 25 or more times across 26,302 trajectories, removing a whole class of run-to-run variance (Mercor, 2026b).

V. Verify: does the run know when it is wrong?

R4. Code verifiers and tests. Where the outcome can be checked by code (a test suite, a database state, a reconciled total), check it on every run. This is what makes τ-bench and Agents' Last Exam trustworthy.

R5. Critic or verifier loops. CAIS added a worker-critic loop on the Remote Labor Index because workers are too optimistic about their own output, and reports it improves results without publishing an ablation (CAIS, 2026). One preprint ablation, on an agent flow for chip accelerator design, found a verifier loop raised cost per success from $5.43 to $7.58 (arXiv 2605.15237). Verification buys reliability, and you pay for it.

A critic can miss what the worker missed. In a biology red-teaming exercise in Anthropic's November 2026 card, graders found overstated claims of efficacy in the agentic submissions. Those claims "were not flagged during the model's review of its own work, nor during review by auxiliary agents" (Fable 5.2 system card, §2.2.2.1).

A verifier is a classifier, so it has the two error rates Part 4 measures for guardrails. A false accept lets a wrong result through as a success: a silent failure. A false reject sends a correct result back for a retry or a hand-back it didn't need, which costs money and attention. Measure both on the golden set, by running the verifier over outputs you already know are right and outputs you know are wrong. A verifier from the same model family as the agent is likely to share its blind spots, so prefer a code check, and mix families where you can't.

R6. Ask before guessing. On APEX-Agents 1.1 (model names as Mercor gives them), Gemini 3.8 Flash searched communications for clarifying information on 93.1% of runs. Kimi K3 and GPT-5.6 Sol did so on about 15%. When Kimi K3 did search, its task-level pass@1 rose 29.7% (Mercor, 2026b). Inconsistent information-seeking is a measurable cause of inconsistent outcomes. R6 is about looking for information that already exists. Asking a person, and checking that the answer is real consent, is S4 in Part 4.

R. Recover: can the run resume after an error?

R7. Checkpoints and resumable state. Save state after each completed step so a failure restarts from the last good point.

R8. Idempotent, retry-safe tools. A retried call must not charge a card twice or file a ticket twice. Idempotency keys on every write.

R9. Bounded retries with fallback. OpenAI's guidance is to cap retries and actions, and escalate to a human when the cap is hit (OpenAI, 2025). Every escalation is a hand-back that a person has to handle, so count it against the hand-back budget.

I found no published agent study with numbers for R7 and R8. They are standard practice in distributed systems, and I present them that way.

C. Contain: does the run stop before it does damage?

R10. Abstain on low confidence. Predictability is what makes this lever work. An agent whose confidence tracks its accuracy can be gated: high confidence proceeds, low confidence escalates. HAL's calibration scores show this varies by model and can get worse on an upgrade (Opus 4.7). The gate design belongs to Part 5. The calibration that triggers it belongs here. Part 4 covers the same lever from the safety side, as S5.

O. Observe: will you see the drift?

R11. Trace-level telemetry. OpenTelemetry's GenAI conventions now define agent and tool spans (invoke_agent, execute_tool), though they remain at Development stability (OpenTelemetry, 2026). Trace every run, so passk can be computed from production runs.

R12. Regression evals on every change. Model upgrade, harness change, tool change, infrastructure change. Re-run the golden set k times on each, and compare the per-item split as well as the average. Part 4's S11 is the behavioural twin of this lever.

LeverGroupBest evidenceEvidence type
R1 Workflow before agentDAnthropic, Building effective agentsProvider primary
R2 Fewer stepsDCompounding arithmeticAnalytical
R3 Deterministic tools, fixed environmentDMercor 1.1 dependency fixVendor
R4 Code verifiers and testsVτ-bench, Agents' Last Exam gradingPreprint
R5 Critic and verifier loopsVCAIS RLI; arXiv 2605.15237Independent (no ablation); preprint
R6 Ask before guessingVAPEX-Agents 1.1 analysisVendor
R7 Checkpoints, resumable stateRDistributed-systems practiceNo agent study found
R8 Idempotent toolsRDistributed-systems practiceNo agent study found
R9 Bounded retries, fallbackROpenAI, practical guide to agentsProvider primary
R10 Abstain on low confidenceCHAL calibration metricsPeer-reviewed
R11 Trace telemetryOOpenTelemetry GenAI conventionsStandard (Development)
R12 Regression evalsOAnthropic, Demystifying evals; infrastructure noise; IMDA regression setsProvider primary; government

The weights question

Part 1 set out five attributes of a use case that change the factor weights: stakes, autonomy, reversibility, integration complexity and verifiability. Stakes and reversibility set how high the reliability bar must be (the Consequence family above). The other three set how much weight reliability carries against the other factors.

Applied to the same five use cases from Part 1, with the reliability bar each one needs:

Use caseAutonomyIntegrationVerifiabilityReliability barReliability's place
Internal meeting summariserLowLowMediumpass@1 is fine; a human reads every outputMinor
Patient-chart summariserLowMediumLowHigh consistency; a missed item is an incidentSecond, after safety
Trading agentVariableHighHighpassk near 100% on irreversible stepsSecond, after controllability
Customer-support agent (refunds)HighHighHighpassk at customer volumeFirst
Coding agent (open PR, run tests)HighHighHighTests catch most failures; passk matters less per runSecond, after capability

These are use-case weights. Within a use case, set the bar per action, as in A bar per action above.

The customer-support case is where reliability is the first constraint. Every conversation is a fresh try. A 90% pass@1 agent fails one customer in ten, and the customers who get the failure don't see the average.


Scope: what this deep dive leaves to the others

This part covers how to measure reliability and how to raise it. Three questions are related to it.

Contract-ready specifications. Part 1 promised a way to write "can do X" into a contract. This part supplies the measurement clause: passk at a stated k, with its confidence interval, beside the hand-back and silent-failure rates, on a stated task set, with a stated grader. Part 4 adds the behaviour clause. The bounds clause (what the agent may do when it is wrong) is Controllability's, in Part 5. I found no enterprise publishing agent SLOs or passk terms yet, so this is a recommendation. One preprint proposes an agent error budget, B = (1 − SLO) × number of decisions, with autonomy that widens or narrows as the budget is used (arXiv 2608.10153). It has not been tested in deployment.

Behaviour under ambiguity. Whether the agent invents an action, hedges or asks is Safety, in Part 4. Princeton's safety dimension is reported there.

Harness portability on upgrades. This part covers the regression eval. Pruning the harness on each upgrade stays in Part 5.

What would change the argument

Four published results would change it:


Back to Part 3, Reliability is getting it right every time.

References

Reliability instruments

Change log

  • 18 Nov 2026 (v2, published). Plain-language review. 14 phrases reworded to remove figures of speech, for example "sits", "lands", "moves", "binds", "ships" and "survives" used for where a number is, how it changes, or what is left. The framework's own terms stay: ceiling, floor, binding constraint, discount, lever and bound-leak rate. Sentences shared with the main page were changed on both pages. No figures, claims or sources changed.
  • 15 Nov 2026 (v1, published). First published. The content moved here unchanged from Part 3 (v7), so that Part 3 reads in about 25 minutes. Figures are renumbered.
← Part 3 · Reliability Part 4 · Safety →