“Add another agent” has become the reflex answer to an unreliable agent. Sometimes it’s right. The math for when it’s right is simple enough that it’s strange how rarely anyone does it.
Here’s the part that surprised me when I wrote it down. Under a simple model of correlated failure, a three-agent majority vote improves on a single agent by:
gain = (1 − ρ) × p(1 − p)(2p − 1)
where p is the single-agent success rate and ρ is how correlated the agents’ failures are. That function has a maximum, and you can compute it: 9.6 percentage points, at p ≈ 79%, and only if failures are fully independent.
| Single-agent success | Independent | ρ = 0.5 | ρ = 0.9 |
|---|---|---|---|
| 60% | +4.8 | +2.4 | +0.5 |
| 70% | +8.4 | +4.2 | +0.8 |
| 80% | +9.6 | +4.8 | +1.0 |
| 90% | +7.2 | +3.6 | +0.7 |
| 95% | +4.3 | +2.1 | +0.4 |
Three things to take from the shape.
Below 50%, voting makes you worse. If each agent is more likely wrong than right, the majority amplifies the error.
As the baseline improves, the gain goes away. Better single agents leave less for a team to fix.
Correlation is the whole game. Three copies of the same model with the same prompt reading the same misleading ticket will agree, confidently, and be wrong together. The correlation model I’m using is simple: with probability ρ a task has a common cause and every attempt gets the same outcome. At ρ = 0.9, you’re paying three times the cost for less than a point.
Compare it with a cheaper option
Voting isn’t the only way to spend a second model call. A verifier that checks one worker’s output and triggers one retry is usually much cheaper, because the retry only happens when something got flagged. The break-even calculator puts them side by side for an 80% baseline, ρ = 0.5 and a $0.50 run:
architecture success cost worth it above
single agent 80.0% 0.50 -
verifier + one retry 84.8% 0.76 5.42
three agents, majority 84.8% 1.88 28.65
Same success rate, at under half the cost. The last column is the break-even value: how much a successful task has to be worth to you before the extra spend pays off. It’s just ΔCost / ΔSuccess. The verifier numbers assume it catches 70% of bad output and false-alarms on 10% of good output, and those two assumptions carry the result. Measure yours; a verifier that can’t actually tell good from bad is pure overhead.
I took the 3.75× team-cost multiplier from Anthropic’s write-up of their research system, which reports agents using about 4× the tokens of chat and multi-agent systems about 15×. That’s one system’s ratio. Yours may differ.
Pipelines: every handoff is a tax
A pipeline of specialists (planner, coder, reviewer, tester) looks like a tidy org chart. Its math is unkind, since the stages multiply:
success = p^stages × handoff^(stages − 1)
If each handoff keeps the work intact 95% of the time, a four-stage pipeline needs each specialist to be 3.9% better than a generalist doing the same stage, just to tie. A 90% generalist needs 93.5% specialists. Specialists do sometimes clear that bar, because a narrower prompt and tool set can really help. You should know that’s the bar you’re trying to clear.
Fan-out: the case that actually works
Independent subtasks in clean contexts are where the math turns in favour of a team:
team = (p × synthesis)^m, single = (p × context_penalty)^m
The team wins exactly when you lose less in synthesis than a single agent loses to a crowded context. That’s the honest argument for sub-agents, and it’s a context engineering argument more than an “agents collaborating” one. Isolation is the benefit.
What the research says
The model above is mine and it’s deliberately simple. The experimental evidence points the same way, and it’s worth reading directly.
Kim et al., in Nature Machine Intelligence (2026), ran 260 configurations across six benchmarks, five architectures and three model families, holding prompts, tools and compute budgets fixed. Single-agent baseline performance was the most reliable predictor of whether coordination helped. They identify a capability-saturation threshold, which correctly predicted the effect of coordination in 94% of validation configurations on SWE-bench Verified and Terminal-Bench. The abstract of their preprint gives the range: +80.8% on decomposable financial reasoning and −70.0% on sequential planning, relative to a single agent. The sign of the effect depends on the task.
Anthropic’s post reports a 90.2% improvement of multi-agent over single-agent on their internal research eval, and says plainly where the approach fits: highly parallel, valuable work, or information that exceeds one context window. It also notes that most coding tasks have fewer parallelizable parts than research does.
Those aren’t in conflict. They’re the same curve at different points.
How to decide
Measure two things before you draw any boxes.
Your single-agent baseline, on your own eval set. If it’s under 50%, fix that first. No topology rescues an agent that’s usually wrong.
Your failure correlation. Run every case twice and count:
from breakeven import estimate_rho
p, rho = estimate_rho(both_pass=140, only_one=30, both_fail=30)
# p = 0.775, rho ≈ 0.57
If two runs of the same agent fail on the same cases, more copies of that agent won’t help. You need something that fails differently: a different model, different tools, a different framing of the task, or a deterministic check.
Then, in order of what I’d try:
- A deterministic check, such as a schema, a test or a policy. It costs almost nothing and doesn’t share the model’s blind spots.
- A verifier plus one retry, if you can show the verifier catches real failures.
- Fan-out, if subtasks really are independent or a single context can’t hold the material.
- A vote, only for high-value discrete decisions and with visibly different voters.
- A sequential pipeline last, and only with evidence the specialists beat the handoff tax.
Limits
These are models. They assume outcomes can be scored as success or failure, and voting assumes outputs that can be compared at all, which free-form work often can’t. I’ve ignored latency and rate limits, both of which get worse with more agents. The closed forms are verified against Monte Carlo in the test suite, which shows the arithmetic is right. Whether your agent fits the assumptions is a question for your eval set.
If you want the qualitative version of this argument, I covered the patterns in the agentic patterns decision guide.
Comments