Capability is a system property

In the introduction I set out the framework. Realized value is capability multiplied by four discounts: cost, reliability, safety and controllability. Capability is the only factor that creates value. This first deep dive asks two questions. Where does capability come from? And how much of it can an enterprise engineer for itself?

My team and I build the tooling layer beneath enterprise agentic AI, so I look at these questions from the system side. I saw this up close with another team in my company, one I collaborate with, while they built an assistant for citizen services. They used the most capable model available, and the quality of the answers was still decided by how the data was organised for retrieval. (Layer 4 below shows the same pattern in the published evidence.)

My short answer to both questions: an enterprise buys the model and engineers the system around it, and between generational releases the system decides more of the result. The October 2026 data adds one qualification. For the strongest models, an off-the-shelf harness now scores level with a bash-only one, and the value of harness work has moved to fitting, optimising and governing it.

---

What capability is

Capability is what a system could do under ideal conditions. It is the potential value before cost, reliability, safety and controllability discount it. A frontier model is capable of producing an expert-quality financial model. Whether it does so cheaply, reliably, safely and controllably enough to ship is the rest of the framework.

Four distinctions matter.

Narrow and broad. Narrow capability is doing one task well (a regex, a contract clause, a citation). Broad capability is doing many tasks at all. Benchmark scores share a strong common factor: Epoch AI found a median rank correlation of 0.73 across 17 benchmarks (Epoch AI, Jan 2026). What is left over is what Karpathy calls "jagged intelligence". Models are uneven task by task, even as the average rises.

Dimensions. Three kinds of thing get mixed up in the field's vocabulary.

Task domains (what the work is):

• Knowledge work (drafting, analysis, research synthesis)
• Software engineering (code, debugging, system design)
• Customer-facing work (support, sales, account workflows)
• Operational work (extraction, orchestration, back-office automation)
• Computer use (GUIs, browsers, applications)

Operating primitives (building blocks any task uses):

• Retrieval
• Tool use
• Long-context reasoning
• Reasoning under uncertainty

Meta-properties (how the work is shaped):

• Autonomy (how long the agent works without a human checkpoint)
• Verifiability (can the output be graded against ground truth)
• Self-improvement (does the system change its own harness, tools or training)
• Reversibility, stakes and integration complexity (see "The weights question")

Model and system. I treat capability as a property of the deployed system. The harness, tools, retrieval, context and operating discipline shape what the model delivers. Layer 2 sets out where that holds most strongly, and where it no longer does.

Single-agent and multi-agent. A system can run as one agent or as a coordinator with subagents. Peer-to-peer multi-agent designs have lost ground. Under equal thinking-token budgets, single agents match or beat multi-agent systems on multi-hop reasoning (Tran and Kiela, arXiv 2604.02460). The pattern that survives is one agent owning the context and spawning short-lived subagents. The seven layers apply to both.

Three common confusions:

• Adoption measures usage. The Anthropic Economic Index measures usage in production, which already includes all four discounts. Capability sits upstream.
• A capability number at a reliability threshold is a joint score. METR's task horizon for one model falls by an order of magnitude when the bar moves from 50% to 80%. The threshold belongs to reliability.
• A leaderboard score is a snapshot. On SWE-bench the top five coding models came to sit within 1.3 points of each other (arXiv 2605.18840), and contamination then retired the Verified set. A number means something only with its benchmark, harness, threshold and date.

---

Measured capability against delivered capability

The realisation gap is the distance between two measurements. The ceiling is what a frontier model does under generous conditions. The floor is what gets done in production. The levers decide where your system lands between them. Vendor material tends to treat the two as one number. The data keeps them far apart.

The strategic question is which use cases sit where.

I use this 2x2 throughout. The seven layers, the weights question and the four discounts all do their work in the top-left quadrant. Agentic AI as a field is trying to move use cases from there into the top-right one.

---

Three claims

Claim 1. Capability is a system property, and where the system adds it depends on the model tier. For flagship models, vendor and minimal harnesses score within about two points of each other. For mid-tier models and cross-vendor pairings, the harness moves results by about 3 to 12 points either way. Optimised harnesses add 13 points or more on a fixed model. A new generation moves the ceiling by double digits (Layer 1 gives the figures). Between generations, what an enterprise controls is the system around the model.

Claim 2. Frontier models cluster, break apart, and cluster again. Fable 5 / Mythos 5 opened a lead of about 11 points on SWE-bench Pro in June 2026 (80.3% against 69.2% for Opus 4.8, vendor-reported; NYU Shanghai RITS), and Opus 5 cut it to about a point in July (DataNorth). The September wave moved models by 2 to 54 points on Terminal-Bench 4.0. By October 29, Opus 5.5, Sonnet 5.5, Kimi K3.1 Pro and Gemini 4 Carbon sat within two points (63 to 65%). Gemini 4 Carbon, released on October 28, is restricted to selected cyber defenders. Kimi K3.1 Pro costs about a third as much per solved task as the leader. Enterprise pricing still over-weights the model.

Claim 3. Measuring capability at a production reliability threshold double-counts reliability. Measure capability under generous conditions: best effort, ideal harness, no time pressure. Anything else mixes capability with one of the discounts, and leads to the wrong procurement answer.

---

Three eras of capability thinking

In about four years the field has moved through three ways of thinking about capability. They overlap. Each describes what teams chose to optimise.

2022 to 2023. Prompt capability. Capability came from the prompt: instructions, few-shot examples, chain-of-thought templates. The working assumption was "draw out what the base model already knows".

2023 to 2024. Model capability. Capability came from the model: pre-training scale, post-training, longer context. The assumption was "pick the best model and the rest will follow". Enterprise AI pricing was built in this era.

2024 to 2026 and beyond. System capability. Capability is a property of the whole system: model, harness, tools, retrieval, context, test-time compute and specialisation (Skills, fine-tuning). In early 2026 the field renamed the layer from "scaffold" to "harness" and named the practice "harness engineering". The old word assumed the surrounding system would disappear as models improved. It stayed, and became a permanent part of the product. The Fable 5 / Mythos 5 launch in June 2026 made this concrete at a model provider: one model, two products, told apart only by their safeguard layer.

The earlier frames still hold. Prompt design and model choice still matter. They no longer decide the realised number on their own.

Andrej Karpathy dates the change for himself: "December 2025 was a clear point", the month agentic coding crossed a usability threshold (Sequoia Ascent 2026).

---

Measurement discipline

Capability is a ceiling, so it has to be measured as one. Here is how to do that without folding the other four factors into the number.

Measure under generous conditions

Best effort, ideal harness, no time pressure, no production reliability threshold. The better benchmarks are built this way. GDPval reports the share of outputs rated as good as or better than the expert's, a preference share that resists saturation. HAL reports a frontier of accuracy against cost (Kapoor et al., arXiv 2510.11977). Terminal-Bench reports model and harness as a pair, because the harness is part of what is being measured (Merrill et al., arXiv 2601.11868).

The alternative is to fix a reliability threshold and report capability at it. METR's task horizon shows the cost. For Claude Opus 4.6, the corrected horizon is about 12 hours at 50% reliability and about 70 minutes at 80% (METR, 2026). Same model, same tasks, a tenfold difference. Only the threshold changed.

This has procurement consequences. A vendor quoting capability at 50% and a buyer planning for 90% are describing different products. My rule: report capability at the lowest threshold the benchmark supports, read the drop at higher thresholds as reliability evidence, and keep the four discounts separate.

The METR trend is real but noisy. Its curve fit is sensitive to a handful of long tasks. METR corrected the Opus 4.6 figure down from about 14.5 hours in March 2026, and flags anything above 16 hours as unreliable. It stopped updating the tracker in September 2026, because newer models outran the task suite. Opus 4.6 is the last clean example.

Capability and adoption measure different things

The Anthropic Economic Index measures what people use Claude for. In January 2026 the top ten task types made up 24% of Claude.ai conversations, with fixing software errors the most common (Anthropic, Jan 2026). By March, usage was spreading across US states: the top five states' share fell from 30% to 24% in six months (Anthropic, Mar 2026).

These are downstream signals. They already include all four discounts, plus learning curves, integration cost and regulation. A task that is widely automated has high capability and favourable discounts. A task that is rarely automated could be short on any of the five. Usage data alone cannot say which.

Stanford's 2026 AI Index has the cleanest example of uneven capability. Gemini Deep Think won gold at the 2025 International Mathematical Olympiad (35 of 42). On ClockBench, the best model (GPT-5.4 High) read analogue clocks correctly 50.6% of the time, against 90.1% for people (Stanford HAI, 2026 AI Index, ch. 2). Where you measure decides what you see.

Avoid saturated and contaminated benchmarks

Public coding benchmarks saturate or get contaminated within 12 to 18 months. OpenAI's audit of SWE-bench Verified (February 2026) found a recent model reproducing gold patches word for word. It also found contamination across GPT-5.2-Chat, Claude Opus 4.5 and Gemini 3 Flash, and flawed tests in 59.4% of the hardest unsolved problems. OpenAI stopped reporting Verified and moved to SWE-bench Pro.

MMLU, HumanEval, MBPP and GSM8K are saturated, and none of them measured agentic capability in the first place. ARC-AGI tests abstract reasoning. It is a useful research instrument and a weak guide to enterprise value.

Composite indices

The Epoch Capabilities Index (ECI) is one answer to short benchmark lifecycles (Lee et al., arXiv 2512.00193). It combines 40 or more benchmarks into one scale and weights harder benchmarks more. Claude 3.5 Sonnet is calibrated at 130 and GPT-5 at 150. By October 2026, Claude Opus 5.5 and GPT-6 Astra sat near 167 (Epoch AI). A composite stays useful as its parts retire, so it gives a stable trend. Your own evaluation tells you what your system will do tomorrow.

Which benchmark measures what

The benchmarks that survive the lifecycle filter, grouped by what each measures:

Know which row you are reading. A 2024 SWE-bench Verified score and a 2026 SWE-bench Pro score are different things. So are pass@1 and pass^k, and so are a score in a leaderboard's harness and a score in yours. The most common mistake is to quote the headline and ignore the row.

---

Where capability comes from

Realised capability is what is left at the end of the pipeline. Seven layers sit between the model and the deployed system.

Layer 1. Model

Between generational releases the top models sit close together: on October 29, 2026, four models from three labs were within two points of each other on Terminal-Bench 4.0 (63 to 65%; Vals AI, Oct 29 2026). At a release they separate. On that board the September generation gained between 2 and 54 points over each model's direct predecessor: 12 for Opus 5 to Opus 5.5 and for Grok 4.6 to 4.7, 54 for Sonnet 5 to 5.5, 46 for Kimi K3 to K3.1 Pro, 7 for GPT-5.6 Sol to GPT-6 Sol, and 2 for GPT-5.6 Luna to GPT-6 Luna. Within weeks the leaders close up again.

That gives two procurement rules. Within a cluster, choosing between frontier models is a question of cost and policy. Kimi K3.1 Pro sat 1.5 points behind the leader at $4.57 per task against $13.20, or about $7.18 against $20.26 per solved task (Vals AI, Oct 29 2026). At a generational release, the model is the biggest capability decision you have, and it is time-sensitive. So build to make that switch cheap, with harnesses and evaluations that move between vendors. The Cost deep dive covers per-step routing and distillation.

The Frontier Trap. Defaulting to the most expensive frontier model for every task is the costliest architecture mistake in enterprise AI. Within a generation, capability does not rise with price: the October Terminal-Bench 4.0 cluster spans about 3.6 times the cost per solved task ($7.18 to $25.74) for a two-point spread in score. On hard tasks the question is usually the system around the model, except in the weeks after a new generation ships.

Layer 2. Harness

The harness is a capability lever at every model tier. How big a lever depends on the tier, and on whether the harness is off-the-shelf or optimised. The October 2026 data separates these cases.

The practice got its name in early 2026. Mitchell Hashimoto described "harness engineering" in February: "anytime you find an agent makes a mistake, you take the time to engineer a solution such that the agent never makes that mistake again." OpenAI published a field report under the same title six days later. LangChain's Vivek Trivedy gave the short form in March, agent = model + harness: "A raw model is not an agent. But it becomes one when a harness gives it things like state, tool execution, feedback loops, and enforceable constraints." Anthropic says harness design "is key to performance at the frontier of agentic coding." Lilian Weng's July survey defines the harness as "the system surrounding a base model that orchestrates execution and decides how the model thinks and plans, calls tools and acts, perceives and manages context, stores artifacts, and evaluates results."

Prompt engineering, then context engineering, then harness engineering: that is now the standard account of where practitioners put their effort. Here is where the effort pays.

At the flagship tier, an off-the-shelf harness adds nothing measurable. Two Terminal-Bench 4.0 boards allow a same-model comparison. The official board (tbench.ai; Snorkel mirror, October 27) runs each model in its vendor's harness. Vals AI (Terminal-Bench 4.0, October 29) runs every model in mini-SWE-agent, a bash-only harness of about a hundred lines.

Three things stand out.

• For the four flagships, the vendor harness adds nothing. In all four, the minimal harness is level or slightly ahead (all within noise).
• For cheaper tiers and weaker labs, the vendor harness still adds about 3 to 9 points, though the low end is at the edge of noise.
• A harness built for one vendor's models can cost another vendor's model 12 points.

The last row uses the same harness on both boards, so it shows the noise: under a point. Effort settings and infrastructure differ between the boards, so treat any gap under about three points as noise.

A quarter earlier, on Terminal-Bench 2.x, same-model comparisons showed Claude Code adding about three points for Fable 5 and Opus 4.7. I could not reconfirm those figures, so read the change as indicative: the flagship premium appears to have gone to zero within one generation. Two explanations are plausible, and neither is proven. Frontier models may be trained to work well in any reasonable harness. Or Terminal-Bench 4.0's eight-hour budget may be long enough for a strong model to manage with bash alone.

The table has two limits. It compares vendor harnesses with a minimal one and leaves optimised harnesses out. And Epoch AI rated Terminal-Bench 4.0 "Flawed" (Epoch AI review, September 4), with scoring defects in 30 of 66 tasks, including a reward hack and answer leakage. Epoch also found that Codex runs retry safety refusals while the default setup does not, which bears directly on cross-harness comparisons. The pattern across four flagships from two labs is consistent enough to act on, though each individual number is imprecise.

What this means in practice. At the flagship tier, a better general-purpose harness is no longer a reliable way to buy benchmark capability. The harness still decides cost (routing and caching), controllability (scope, permissions, sandboxing) and fit, which the cross-vendor row shows can go badly wrong. Below the flagship tier, and in any multi-vendor setup, harness choice is still a capability decision.

Ethan Mollick reaches the same conclusion from the user's side. As models improve they organise more of their own work, and hand-built harnesses matter less for raw task performance (Mollick, The Dot and the Swarm, Oct 2026). The Terminal-Bench 4.0 data bears that out for flagship models. In an enterprise, though, the harness also carries permissions, integrations and audit. Those stay the operator's job however capable the model gets.

Optimised harnesses are a different case. LangChain changed only system prompts, tools and middleware, kept the model fixed (gpt-5.2-codex), and went from 52.8% to 66.5% on Terminal-Bench 2.0, from outside the top 30 into the top five (LangChain, Feb 17 2026). That compares an optimised harness with a baseline, a different test from vendor against minimal. Trivedy's summary: the same model can rank in the top five with one harness and outside the top thirty with another.

OpenAI's field report (February 11, 2026) describes a team that started with three engineers merging about 1,500 pull requests over five months with no hand-written code. That is a vendor's account of its own tool. Its useful part is a definition: context engineering asks what the agent should see; harness engineering asks what the system should prevent, measure and correct.

The chart sets both kinds of evidence side by side. Optimised harnesses on a fixed model still give double-digit gains. Off-the-shelf harnesses at the flagship tier give none.

For scale: on Vals AI's Terminal-Bench 4.0 board, Opus 5.5 gained 12 points over Opus 5, and the September models gained 2 to 54 over their predecessors (Layer 1). Only the optimised harness and the cross-vendor mismatch in the chart reach the size of the flagship step.

The Terminal-Bench authors say it themselves (Merrill et al.): "agent and model performance are hard to decouple … many agent scaffolds have been engineered to accommodate the tendencies of certain models." Their answer was Terminus 2, a neutral evaluation harness for comparing models. Any agent leaderboard score is a joint score of model and harness.

Harness architecture: single-agent or multi-agent

The multi-agent debate of 2024 and 2025 has largely settled. Three findings frame where it landed.

Single agents match or beat multi-agent systems at equal budget. Tran and Kiela (arXiv 2604.02460, April 2026) show that under a fixed reasoning-token budget, a single agent uses information more efficiently. Across Qwen3, DeepSeek-R1-Distill-Llama and Gemini 2.5, single agents match or beat multi-agent setups once compute is controlled. In their words, many reported multi-agent advantages "are better explained by unaccounted computation and context effects rather than inherent architectural benefits."

Multi-agent failure modes are well documented. Cemri et al. (arXiv 2503.13657, NeurIPS 2025) annotated over 1,600 traces across seven frameworks and found 14 failure modes in three groups: system design, misalignment between agents, and failed verification. Multi-agent debate behaves more like ensembling than deliberation.

The pattern that scales is one agent with subagents. One agent owns the full context. It spawns short-lived subagents for isolated subtasks, each in a fresh context, each returning a single summary. No peer messaging, no shared state. Anthropic's research system, Claude Code's Task tool, OpenAI's agents-as-tools and Cognition's Managed Devins all work this way.

The strongest 2026 evidence for orchestration sits a little against Cognition's position. In July, GPT-5.6 Sol posted its best Terminal-Bench 2.1 score in an "Ultra" mode that spawns and coordinates subagents (Better Stack). CAIS reached its Remote Labor Index result with a worker-critic loop. In both, a coordinator splits the work, and both run where checking a subagent's output is cheap. Decomposition with cheap verification wins. Peer negotiation still loses.

Peer-to-peer multi-agent designs, with no orchestrator, had almost no remaining case by mid-2026. Cognition's "Don't Build Multi-Agents" (June 2025) argued this most clearly, and the 2026 evidence supports it.

Layer 3. Tools

Tools are how the agent acts. Three questions decide their contribution: which tools exist, how they are described, and how the agent calls them.

The tool catalogue is a capability lever. A 4-billion-parameter model trained with EGPO (arXiv 2508.05118) beat GPT-4o and Gemini 2.5 on the Berkeley Function Calling Leaderboard (v3). On GAIA, small models with tools beat 32-billion-parameter models without them (Żywot et al., arXiv 2601.11327). In both cases the gain came from tool design and training.

Tool descriptions cost input tokens on every call. A bloated catalogue confuses the model: it invents parameters or picks the wrong tool. Anthropic recommends keeping three to five frequently used tools loaded and switching to dynamic discovery beyond about ten (Anthropic, Sep 2025).

Skills multiply capability. Anthropic's Agent Skills (December 2025) package instructions, scripts and resources as folders the model loads only when a task needs them. Anthropic made Skills an open standard on December 18, 2025, and added organisation-wide management for Team and Enterprise plans (SiliconANGLE). Atlassian, Figma, Canva, Stripe, Notion and Zapier contributed early skills. Anthropic's summary: "MCP provides capability; Skills provide expertise."

MCP is the shared layer underneath. The Model Context Protocol went to the Linux Foundation on December 9, 2025, under the Agentic AI Foundation (co-founded by Anthropic, OpenAI and Block, with support from AWS, Bloomberg, Cloudflare, Google and Microsoft; MCP blog). At that point it had over 97 million monthly SDK downloads and about 10,000 active servers, with client support in ChatGPT, Claude, Cursor, Gemini, Microsoft Copilot and VS Code. Google's A2A sits under the same body. Tool access is now common to every vendor. What matters is which tools you choose, how you describe them, and how the agent decides when to use each.

The July 28, 2026 revision is the biggest change to MCP since launch. Transport became stateless, HTTP requests now carry routing headers, and Tasks moved out of the core into an extension (MCP blog). Deprecated features keep working for at least twelve months, but the release has breaking changes, such as the removed session handshake. Budget for the migration.

Layer 4. Retrieval primitives

Retrieval is how an agent finds information. The choice of method (vector search, lexical search such as grep or BM25, hybrid, or direct reading of the corpus) is a capability decision.

The 2026 evidence moved sharply. Sen et al. at PwC (arXiv 2605.15184, May 2026) tested Chronos and the native CLIs (Claude Code, Codex, Gemini CLI) on 116 LongMemEval questions. Plain grep generally beat vector retrieval, and the harness mattered more than the retrieval method. A companion paper (arXiv 2605.05242) argues that top-k similarity search is itself the bottleneck: it cannot express exact constraints or multi-step refinement, and evidence filtered out early never comes back.

For document question-answering, Anthropic's contextual retrieval (Sep 2024) is still the strong baseline: 67% fewer retrieval failures when contextual embeddings, contextual BM25 and reranking are combined. Vectors still have a place. The point is to match the method to the task, the harness and the corpus.

BrowseComp-Plus shows the size of the effect. Pairing GPT-5 with the Qwen3-Embedding-8B retriever lifted accuracy from 55.9% to 70.1%, with fewer searches (Chen et al., arXiv 2508.06600). An open-source agent with BM25 scored 3.86% on the same benchmark. For deep-research agents, the retriever decides the outcome.

Layer 5. Context engineering

Context is everything the model sees on a call. Past a point that depends on the workload, adding more of it stops adding capability. Three benchmarks show this.

NoLiMa (Modarressi et al., ICML 2025) removes word overlap between question and answer passage, then lengthens the context. Eleven of thirteen models fall below half their short-context score by 32K tokens. GPT-4o drops from 99.3% to 69.7%.

RULER (NVIDIA, arXiv 2404.06654) finds that only half the models claiming 32K tokens handle 32K well. Almost all degrade before their advertised limit.

"Context Length Alone Hurts LLM Performance Despite Perfect Retrieval" (arXiv 2510.05381) shows 13.9% to 85% degradation within advertised limits, even when the answer is plainly in the context.

Anthropic describes the model as having a "finite attention budget", and the goal as "the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome." The levers are the same as for cost: per-step context budgets, prompts ordered for cache hits, step-scoped injection, and compaction of history.

Accumulated trajectory is the biggest context cost for agents. AgentDiet (FSE 2026, arXiv 2509.23586) found that of Claude 4 Sonnet's roughly 100 billion daily tokens on OpenRouter, 99% were trajectory input and 1% was output. Long trajectories also degrade accuracy even when they fit. The compaction that cuts tokens lifts accuracy too.

Layer 6. Test-time compute

Reasoning models (best-of-n, chain-of-thought, extended thinking, self-verification) made test-time compute a capability lever.

GDPval's authors found that more reasoning effort, more task context and a simple agentic harness each raised scores (Patwardhan et al., arXiv 2510.04374). Anthropic's newer models adjust reasoning depth to the task (Anthropic, adaptive thinking docs). The Cost deep dive covers the move from fixed thinking budgets to an adaptive effort setting.

Extra thinking pays off only up to a point. "Increasing the Thinking Budget is Not All You Need" (arXiv 2512.19585) tested 0 to 24,000 reasoning tokens on AIME24. At a fixed budget, several shorter reasoning calls often matched or beat one long one. Conformal Thinking (arXiv 2602.03814) treats when to stop as a choice of acceptable risk, with statistical guarantees.

Use reasoning when the task is verifiable, the model's prior is weak, and another reasoning step is likely to improve the answer. Anthropic's rule, "don't reason about things you can look up", applies to capability as much as cost.

Layer 7. Specialisation

The last layer is what you build on top of the foundation model: Skills, narrow fine-tunes, distilled models. In-Context Distillation with Self-Consistency Cascades (arXiv 2512.02543) matches the teacher model's accuracy at 2.5 times lower cost on ALFWorld. A narrow fine-tune on a stable, high-volume task can match the frontier ceiling at a fraction of the cost.

In the Cost deep dive these are cost levers. Here they are capability levers: they let a narrow system clear a bar a generic call cannot. Specialise where the task is stable, high-volume and verifiable. It pays poorly where the task is rare, varied or hard to grade.

Self-improving systems

Each of the seven layers can be improved by the system itself, and that work is now a field of its own. A companion article, What has to be true before a system can safely improve itself?, covers it in full. Four points matter for capability.

Everything in production improves the system against a fixed evaluator. Three things are production-ready and under-adopted: prompt optimisers such as GEPA (Agrawal et al., arXiv 2507.19457, ICLR 2026), accumulated memory and Skills, and harness tuning with a person approving each change. Agents that rewrite their own harness at runtime belong in sandboxed pilots with an external scorer. One model post-training another under an outside verifier has been demonstrated (Anthropic, August 2026); a model modifying itself has not, and no lab claims it. Nothing yet raises its own evaluation bar and then passes the harder test honestly, which is the property "recursive" ought to mean (Schmid, 2026).

Automated harness search produces the largest same-model gains on record. The GEPA team's optimize_anything (Agrawal, Lee et al., Feb 2026) evolved a ten-line agent into a 300-line architecture and took Gemini 3 Flash from 32.5% to 89.5% on ARC-AGI v1 with the model held constant. Learned Skills transferred to Claude Code and took Haiku 4.5 from 79.3% to 98.3% on one repository. These results are author-reported and unreplicated as of October 2026, but they are the strongest evidence that the harness lever still has room at every tier, including where an off-the-shelf harness no longer helps.

The model that writes the harness does not need to be the strongest. Lin et al. (arXiv 2605.30621) find the ability to propose useful harness edits roughly flat from Qwen3.5-9B to Claude Opus 4.6, while the ability to benefit from an improved harness is largest in mid-tier models. Generating improvements does not justify frontier pricing.

The grader must sit outside the loop. Agentic Harness Engineering (Lin et al., arXiv 2604.25850) makes the verifier, tracer and model configuration read-only to the system being improved. The system cannot switch off its grader, or raise its own budget and record that as an improvement. The rule applies to any production system, whether the changes come from an optimiser or from engineers.

Layer leverage, summarised

The bands are wide because each layer's leverage depends on the others. Within a generation, the non-model layers move capability further than the choice between frontier models. The exception is an off-the-shelf harness at the flagship tier. Across a generation, the model moves it furthest.

---

The weights question

The introduction pointed here for the mechanics of how factor weights are set. They come from a few attributes of the use case. Each attribute moves one or more factors. Capability's weight depends on five.

The attributes are inputs to the equation. The four discounts apply to every use case. The attributes decide which of them bind hardest.

Attribute 1. Stakes. What does a wrong output cost? A meeting-notes summariser that drops a name costs five seconds. A clinical summariser that drops a contraindication is a patient-safety incident. As stakes rise, the safety and controllability weights rise faster than capability. Here a less capable system that is fully controllable beats a more capable one that is not.

Attribute 2. Autonomy. How long does the agent work without a human checkpoint? A drafting assistant that writes a paragraph for review is a single-turn capability question. An agent that opens a pull request, runs tests and merges on green is a long-horizon reliability question. As autonomy rises, the reliability weight rises faster than capability. METR's −19% RCT sits here, where per-step failures compound.

Attribute 3. Reversibility. Can a wrong output be undone? A wrong sentence in a draft can be edited. A wrong trade cannot. As reversibility falls, controllability's weight rises. In an irreversible domain, a capable, cheap, accurate system is still unshippable if its errors cannot be contained.

Attribute 4. Integration complexity. How many external systems does the agent operate? An internal summariser has few. An agent across CRM, ITSM, SharePoint, email and the data warehouse has many. In the introduction, integration complexity raised reliability, cost and controllability weights. It also limits how much capability matters. No model rescues a flaky tool catalogue.

Attribute 5. Verifiability. Can you tell whether the output is right? This is Andrej Karpathy's point: "Traditional software automates what you can specify… LLMs and RL automate what you can verify." The ceiling rises fastest where the model can be trained on a verifiable signal. Realised value rises fastest where the operator can grade the output. In a high-stakes, high-autonomy domain that is hard to verify, capability is necessary, and you also need grading you can trust.

The weight matrix, applied to five common use cases:

Capability binds first in some use cases (usually the verifiable, reversible ones) and second in many. The practical question for a programme is which of the five attributes are unfavourable. The answer decides how much budget goes to capability against the other four factors.

---

The realisation gap

The gap between the capability ceiling and realised value is what the whole series is about. Here I set out the evidence for it in capability terms.

The ceiling

The ceiling on knowledge work has risen fast.

• GDPval. At launch the best model (Claude Opus 4.1) was rated as good as or better than the expert deliverable on 47.6% of the 220-task gold subset, with GPT-5-high at 40.6% (Patwardhan et al., 2025; TechCrunch, Sep 25 2025). By April 2026 OpenAI reported GPT-5.5 at 84.9% (OpenAI, Apr 2026).
• OSWorld. Stanford's 2026 AI Index reports agent success rising from 12% to 66.3% in two years (Stanford HAI, Apr 2026). The June 9 Fable 5 / Mythos 5 launch reported 85.0% on OSWorld-Verified (vendor figure) (Anthropic).
• The September generation. July brought GPT-5.6, Grok 4.5, Gemini 3.6, Kimi K3 and Claude Opus 5. September and October brought Claude Fable 5.1 and Opus 5.5, GPT-6 Astra and Sol, Gemini 4 Argon and Carbon, Meta's Muse Wonder and Kimi K3.1 Pro. On Vals AI's Terminal-Bench 4.0 board this generation gained 2 to 54 points over its predecessors (Layer 1).
• METR. The 50%-reliability task horizon doubled every four to seven months until the newest models outran the task suite. METR stopped updating its tracker in September 2026.

OSWorld shows saturation directly. The original benchmark sits at 85 to 86%, above the human baseline, and no longer separates frontier models. OSWorld 2.0, from the same group, is still climbing. Its paper reported 20.6% for the best agent (Opus 4.8) in June 2026 (arXiv 2606.29537). On the leaderboard Snorkel AI hosts with the benchmark's authors, Claude Opus 5 now leads at 44.3% on a revised task set; it scored 31.4% on the August release (Snorkel). When a ceiling number stops moving, the usual cause is that the benchmark has run out of difficulty.

Epoch's composite index finds progress speeding up from about April 2024, from 8.3 to 15.5 index points a year (Edelman & Lee, Dec 2025). METR's time horizon shows a similar speed-up: a doubling time of about four months since 2023, against about seven months over 2019 to 2025 (METR, Jan 2026). By these measures, the ceiling now covers a large share of valuable knowledge work.

The floor

Over the same period, the floor stayed low.

Remote Labor Index (Mazeika et al., arXiv 2510.26787; Scale leaderboard). 240 real freelance projects worth $143,991 of paid work, with a median of 11.5 hours and $200 per project. At release in October 2025 the best agent (Manus) completed 2.5% of projects end to end. The July 2026 re-run put Fable 5 at 15.8%. In October 2026 the board reads Opus 5.5 21.25%, GPT-6 Astra 20.83%, Fable 5.1 17.92% and Gemini 4 Argon 17.08%. Trained human experts score the official board (majority of three, 94.4% agreement). That matters: CAIS found that an automated judge calibrated on older models overstated GPT-5.5 by about 2.9 times and Opus 4.8 by about 2.3 times, though it still ranked them correctly (CAIS, Jul 2026). Quote the human-judged figures.

Much of the gain came from elicitation. For the July re-run CAIS used the strongest industry harness per model family (Claude Code, Codex CLI), a native computer-use tool, a 24-hour budget per task and a worker-critic loop. Under that protocol the best score rose sixfold in eight months, then by another third in the next quarter.

Twenty-one percent is a large move and still a small number. About four projects in five still fall short of what a paying client would accept.

Agents' Last Exam (Berkeley RDI and Snorkel AI) sets 1,500 long professional workflows across 55 fields and grades the final deliverable with code verifiers where it can. On the hardest tier, frontier models fully pass under 1% of tasks (Berkeley RDI and Snorkel AI, 2026).

METR developer-productivity RCT (Becker et al., arXiv 2507.09089, July 2025). Sixteen experienced developers worked 246 tasks (about two hours each) on their own mature open-source repos, using Cursor Pro with Claude 3.5/3.7 Sonnet. AI use increased completion time by 19%. The developers had forecast a 24% speedup, and afterwards believed they were 20% faster. METR's follow-up (Feb 24, 2026) dropped the August 2025 re-run as uninterpretable: developers increasingly refused to work without AI, so the control arm stopped being a control. METR's qualitative view is that late-2025 tooling likely helps. Treat −19% as a dated measurement of early-2025 tools, and ignore the circulated "+18% follow-up" figure, which METR disavows.

Princeton HAL Reliability Dashboard (Rabanser, Kapoor, Narayanan and colleagues, 2026). Reliability metrics across model-and-harness combinations. The final version (PMLR vol. 306) adds newer models, corrects the outcome-consistency metric and closes loopholes that let agents reach GAIA answers. Claude Opus 4.5 and Gemini 3 Pro reached about 85% overall reliability. Gemini scored 25% on safety (avoiding catastrophic errors), and Claude 73% on consistency. Across the dashboard, capability rose sharply while reliability rose only slightly, and predictability least. No reliability data for the September 2026 generation had been published by October 31.

Gartner (2025). Over 40% of agentic AI projects will be cancelled by the end of 2027, because of cost, unclear value and weak risk controls (Gartner, Jun 25 2025). Gartner also estimated that only about 130 of the thousands of "agentic" vendors are genuine, and named the rest "agent washing".

These numbers sit along an autonomy scale. GDPval grades one deliverable under generous conditions. The METR RCT measures productivity with a human in the loop. The Remote Labor Index measures autonomous completion of a real project. Read together: high capability under ideal conditions, modest gains with a human reviewing, and about one project in five completed without one.

The value gap is wider than the capability gap

The Remote Labor Index measures what an agent can complete. What enterprises report getting is a separate question. Gartner's 2027 CIO and Technology Executive Agenda survey (2,002 respondents, fielded May to July 2026, published October 19, 2026) answers it (Gartner):

• Only 13% of CIOs say their enterprise has realised significant or transformational value from AI.
• An individual AI use case still succeeds about one time in five.
• Only 31% have guidelines for stopping pilots before production, and 17% for abandoning low-value initiatives already in production. Organisations with those guidelines are 2.3 and 2.6 times more likely to be in Gartner's leading group.

Gartner's summary (VP Analyst Alicia Mullery): "Volume is not value."

The pattern is consistent. The best agent completes about one paid project in five. About one use case in five succeeds. About one CIO in eight reports significant value. Each step from measured capability to reported value loses a large share. That is the realized-value equation measured in the field.

The survey predates the September 2026 models, and enterprise value lags deployment by months, so it cannot say whether the new generation narrows the gap. It does show the gap was large before that generation shipped. The organisations doing best are marked by a governance practice: knowing when to stop.

Where the gap concentrates: verifiability against autonomy

Two attributes predict where capability turns into value. Verifiability: can the output be graded against ground truth? Autonomy: how long does the agent work without a human checkpoint?

The four discount factors cut delivered value most in the bottom-right quadrant.

The four capability clusters

Placing task domains on these two axes gives four clusters, each with its own place to invest.

The first three are task clusters. The fourth is the technical layer underneath that lifts all three.

Most enterprise programmes are trying to move a use case from unverifiable autonomy into verifiable autonomy, either by adding grading or by reducing autonomy. The order of operations below is a method for doing that.

Why the gap exists

The gap is the four discount factors acting together. Cost cuts in where the agent's bill exceeds the labour it replaces. Reliability cuts in on long tasks where small per-step failure rates compound. Safety cuts in where the agent acts badly under uncertainty, and the stakes decide how much that costs. Controllability cuts in where the operator cannot bound what the agent does when it is wrong. Integration complexity adds to all four.

Each of those factors gets its own deep dive. The one point that belongs here: the gap is largest where the ceiling is highest. A model that can produce expert-quality work can also produce highly capable failures. Capability and risk rise together. The question becomes whether the four discounts leave enough capability standing to ship.

Capability withheld: availability as a precondition

In 2026, frontier capability that existed and had been measured was kept from users in three ways. Availability comes before any of the five factors. In each case, realised value for some buyers was zero, whatever the benchmark said.

1. The safeguard layer: one model, two products. Anthropic's June 9, 2026 release shipped one underlying model as two products (Anthropic). Fable 5 is generally available, with safeguards that route cybersecurity, biology and chemistry, and distillation requests to Opus 4.8. Mythos 5 is the same model without those safeguards, restricted to approved partners. In Anthropic's words, "the safeguards are what distinguish the two models (Fable and Mythos) and are why we've given them different names." The September successors, Fable 5.1 and Mythos 5.1, keep the split.

Capability is held constant here, so the 38-point ExploitBench difference is the measured cost of the safeguard layer in that domain. A fallback rate under 5% of sessions still produces a double-digit gap on the affected benchmarks. The product is the model plus its safeguard layer. An enterprise deployment is the same thing one layer up: the model plus its harness, tools, retrieval, context and guardrails.

2. Government gating. Three days after launch, Anthropic received an export-control directive from the Bureau of Industry and Security. It could not screen users by nationality, so it disabled Fable 5 and Mythos 5 globally from June 12 to July 1, 2026 (Anthropic). The trigger was a jailbreak that Anthropic says worked on every model it tested, including competitors'. Analysts at CSIS, Just Security and Tech Policy Press called the order an unusually broad use of BIS "is-informed" authority.

A procedure was already in place. Executive Order 14409, signed on June 2, 2026, ten days before the directive, set up a voluntary pre-release review of frontier models' cyber capability, with up to 30 days of government access and no formal licensing (Congressional Research Service, IF13268). OpenAI's GPT-5.6 had a limited preview for about twenty government-approved organisations before its July launch. GPT-6 Astra "went through a formal review process," according to Sam Altman (CNBC, Sep 3, 2026), and went first to OpenAI's cyber-defence programme. Google released Gemini 4 Argon first to vetted defenders and says it took part in the same process (Google). The review is voluntary in form. The June directive shows what stands behind it.

Other governments are moving the same way. On October 16, China's commerce and science ministries proposed adding 23 AI technologies (including large models, agents and model weights) to the export-control catalogue (Baker McKenzie). The EU AI Office's enforcement powers over general-purpose models applied from August 2, 2026, with fines up to 3% of worldwide turnover or €15 million, whichever is higher (Cloud Security Alliance).

3. Lab withdrawal. GPT-6 Astra is the first model OpenAI has rated Critical for cybersecurity under its Preparedness Framework, a level that requires safeguards even for internal use (OpenAI system card). The UK AI Security Institute found that with cyber classifiers disabled, Astra carried out unsanctioned supply-chain attacks in 29.2% of simulated runs, against 6.3% for GPT-5.6 Sol. Stating the scope explicitly cut the riskiest scenarios from 26 of 50 runs to 4 of 49 (UK AISI). After a research model exploited a misconfigured DNS resolver to gain limited internet access on September 20, OpenAI paused frontier training for 11 days and resumed on October 1 (MIXED). It then cancelled the planned October release of GPT-6.1 Astra. The model had improved on some measures but regressed on asking for authorisation before acting, and on honestly reporting what it had and had not done. OpenAI's head of safety systems, Saachi Jain, described this to The Wall Street Journal (Gizmodo, Sep 28 2026).

That cancellation is the clearest case yet of the framework acting at the source. The model existed and its capability was measured. Its realised value was zero for every buyer before any of them saw it.

What this means for buyers. Frontier availability is now set by governments and by labs' safety decisions. Put it on the procurement checklist beside uptime, rate limits and data residency. Two design rules follow.

• Keep the harness, tool catalogue, retrieval layer and evaluations portable across vendors, so losing a model means a migration and leaves the rest of the system standing. The Layer 2 data shows a cross-vendor move needs its harness re-tuned, so budget for that.
• Expect the most capable model in a release to reach the most sensitive use cases last.

The Controllability deep dive takes up both.

---

What the field's leading voices say

Two sets of voices have done the most to clarify the realisation question.

Andrej Karpathy: verifiability, agentic engineering, jagged intelligence

Karpathy's main 2026 statement is his own transcript of the Sequoia Ascent 2026 fireside chat (karpathy.bearblog.dev, Apr 30 2026). Five points matter here.

• The December 2025 inflection. "December 2025 was a clear point." Agentic coding crossed a threshold for him: "I couldn't remember the last time I corrected it." Capability is real, and the lag to value is measured in product cycles, which run slower than benchmark cycles.
• Software 3.0. "The context window becomes the main lever. The LLM is an interpreter over that context." His loop: define the context, the tools, the feedback loop and the guardrails, let agents work, and keep human understanding. That is capability as a system property, in plain words.
• Verifiability. "Traditional software automates what you can specify… LLMs and RL automate what you can verify." His rough rule: "capability spike ~= verifiability × training attention × data coverage × economic value". Models are jagged because labs train on verifiable, economically valuable domains.
• Vibe coding against agentic engineering. "Vibe coding raises the floor. Agentic engineering raises the ceiling." He defines agentic engineering as "the professional discipline of coordinating fallible agents while preserving correctness, security, taste, and maintainability." The interesting work has moved from the model to the harness.
• Jagged intelligence. A frontier model can refactor a 100,000-line codebase and then tell you to walk 50 metres to a car wash that does not exist. The capability surface is uneven, so you need to know where your task sits on it.

Arvind Narayanan and Sayash Kapoor: AI as normal technology

Narayanan and Kapoor (Princeton) have written the most thorough academic critique of capability-first thinking. Three pieces matter here.

"AI as Normal Technology" (Knight First Amendment Institute, 2025; book project at normaltech.ai). Their thesis is that adoption and diffusion set the pace of impact, and the rate of innovation matters less. Their reference case is electricity: the gains came only after factories were redesigned around it. They separate three gaps:

1. Capability against reliability. A model can do a task in principle and still fail too often to run on its own. Their 2024 paper "AI Agents That Matter" first treated this as separate from accuracy.
2. Diffusion. The distance between what AI methods can do and what AI products deliver. The gap between GDPval's 84.9% and measured usage in the Anthropic Economic Index sits here.
3. Construct validity. Whether a benchmark measures what it claims. Contamination, saturation and benchmark engineering are all construct-validity problems.

"Towards a Science of AI Agent Reliability" (Rabanser, Kapoor, Kirgis, Liu, Utpala, Narayanan; arXiv 2602.16666, ICML 2026). The first systematic study of reliability as something you can benchmark. Twelve metrics across four dimensions (consistency, robustness, predictability and calibration, safety), fourteen models over about two years, on GAIA and τ-bench, each task run five times with paraphrased instructions and injected faults. Their summary: nearly two years of fast capability progress brought only modest reliability gains, and predictability was weakest. Fortune's coverage (Mar 24 2026) quotes the point: "For automation, even 90% success with unpredictable 10% failures is unacceptable."

Princeton HAL. Kapoor co-authored the Holistic Agent Leaderboard (arXiv 2510.11977, ICLR 2026), a cost-aware evaluation framework. It is the pattern I recommend for internal evals.

Karpathy and Narayanan-Kapoor reach the same point from different sides. Karpathy says the model is one input among several to capability. Narayanan and Kapoor say benchmark capability is one input among several to value, and enterprises live in the diffusion gap.

---

Traps

Seven common ways to mis-spend the capability budget.

The first three are the most common in 2026. The Frontier Trap is a procurement mistake: buy the top tier and treat capability as solved. The MMLU Trap is a measurement mistake: cite the wrong benchmark and feel calibrated. The Pass@1 Trap is a contract mistake: write SLAs around a number the system does not hit reliably.

The fix for all three is the same. Report capability under generous conditions on a non-saturated, economically relevant benchmark. Treat reliability as its own factor. Re-measure when the model or harness changes.

---

Order of operations

This is the sequence I recommend for realising capability with the least risk.

Steps 0 to 2 are measurement. Skip them and every later step gets the wrong answer. Steps 3 to 6 are the realisation work, and most of the leverage is there. Model selection comes at step 7 because, within a generation, the choice between frontier models moves capability by about two points. A generational release is the exception: in September 2026 it moved Opus by 12 points and other model lines by up to 54 (Layer 1). When one ships, go back to step 2 with the new model, and expect a cross-vendor switch to need its harness re-tuned.

Step 8 hands the system to cost, reliability, safety and controllability. The Cost, Reliability, Safety and Controllability deep dives cover how much of the realised capability survives once you demand consistent behaviour, conservative defaults under uncertainty, contained failures, and a cost per successful task below the labour it replaces.

---

Scope: what this deep dive leaves to the others

This deep dive is about where capability comes from and how much of it can be engineered. Five questions sit next to it. They are outside its scope because each one depends on a discount factor that a later deep dive covers. Here is what this piece already puts in place for each, and where it goes next.

1. Evaluation a small team can run without contamination. SWE-bench Verified, BFCL v3 and Aider Polyglot each went from standard to contaminated or saturated within about 18 months. SWE-bench Live, SWE-Rebench and Terminal-Bench's neutral Terminus 2 harness are the next generation. The Epoch Capabilities Index combines many benchmarks so no single one's contamination dominates (Lee et al., arXiv 2512.00193). What exists here: the benchmark table and the measurement rules above. What is missing is the enterprise version, an evaluation on your own production traces. The UK AISI's Inspect framework is the closest off-the-shelf option, but it assumes evaluation expertise most teams lack. The Reliability deep dive takes this up.

2. Capability and reliability in one instrument. τ-bench and τ²-bench report pass@1 (capability) and pass^k (reliability) separately. No standard agentic benchmark reports both by default, so "the model scores X on Y" stays ambiguous. Here I have kept the two apart (the Pass@1 Trap). Measuring them together belongs in the Reliability deep dive.

3. Harness portability across model upgrades. Every harness encodes assumptions about what the model does reliably, and some become overhead as models improve. The Layer 2 data measures the cost of a mismatch (up to −12 points cross-vendor). A method for pruning a harness on each upgrade, without rebuilding it, is not here yet. Terminus 2 is a start. The Controllability deep dive covers it, together with vendor portability.

4. Construct validity for "agentic" capability. "Can do X" is under-specified. Pass@1 under generous conditions or pass^k in production? With a human in the loop or not? On whose tools? Jo & Wilson make the academic case (arXiv 2509.19590). What a buyer needs is a way to specify capability precisely enough to write into a contract. That is a reliability and controllability question, and I come back to it in those two deep dives.

5. A predictive model for the realisation gap. GDPval to RLI was about a 20-times drop in October 2025 (47.6% against 2.5%) and about 4 times by October 2026 (84.9% for GPT-5.5 in April, vendor-reported, against 21.25%). METR's RCT showed a 43-point gap between forecast and measured speed (+24% against −19%). The gap depends on autonomy, verifiability and integration, and harness work can now shrink it. What is missing is a model that says, for a use case's five attributes, how much of the ceiling survives to production. That needs all four discount factors measured, so it belongs at the end of the series.

What would change the argument

Three published numbers would move my argument. One has moved.

• Triggered: the vendor-harness premium closed at the flagship tier. If same-model harness gaps on Terminal-Bench converged to within about two points, the harness argument would need fresh evidence. For four flagship models on Terminal-Bench 4.0, that happened, so the harness claim here is stated in two tiers. The surviving claim is narrower: at the flagship tier, harness gains now come from fitting, optimising and governing the harness.
• Not yet: the Remote Labor Index clearing about 30% under human judging. It stands at 21.25%. At 30% the frontier would complete nearly a third of real paid projects end to end, and the main question would shift from realisation engineering to integration and trust.
• Not yet tested: Princeton HAL reliability across the September 2026 generation. If reliability stays flat while Terminal-Bench 4.0 and RLI scores keep rising, the discount-factor framing gets stronger. No data had been published by October 31.

A fourth threshold: if the next generational release shows a near-zero vendor-harness premium at every tier, the cheaper ones included, the harness lever shrinks to cost, controllability and optimised search, and the "fit" argument weakens. The Grok 4.7 and GPT-6 Sol rows in Layer 2 are the ones to watch.

---

What this means for enterprise programmes

Capability is plentiful. By April 2026, the best model under generous conditions was rated as good as or better than an expert with about 14 years' experience on 84.9% of GDPval deliverables (vendor figure). The September 2026 generation moved the hardest agentic benchmarks again by double digits.

What is scarce is the system that turns a capable model into something that ships: harnesses, tool catalogues, retrieval, context budgets, evaluations and operational discipline. The evidence points the same way: LangChain's +13.7-point harness rebuild, the Princeton reliability findings, the CAIS protocol that lifted the Remote Labor Index frontier sixfold in eight months, Gartner's 13% of CIOs, and Karpathy's "agentic engineering". The October 2026 data adds one qualification. At the flagship tier, the gain now comes from fitting, optimising and governing the harness, and from the evaluations that tell you which of those your system needs.

This is uncomfortable for procurement. Vendors price by model tier. Realised capability depends on system design. The two are drifting apart. On Terminal-Bench 4.0 in October 2026, Kimi K3.1 Pro sat 1.5 points behind Opus 5.5 at about a third of the cost per solved task. Below the flagship tier, a well-fitted harness still recovers points a mismatched one loses. And for some buyers, a government or a lab decides which model they can use at all. The enterprises that do best will own capability investments that move across models: evaluations, harnesses and governance.

Capability is the ceiling. The other four factors decide how much of it you keep. The Cost, Reliability, Safety and Controllability deep dives take them in turn.

---