· 5 min read AIReliabilitySRE

The 47th approval click is the bug

Human-in-the-loop is a real control until volume turns it into a reflex. How approval fatigue converts oversight into a rubber stamp — and the fix.

You built the gate for the right reason. The agent proposes a remediation, a human reviews it, and nothing touches production without a person saying yes. Every governance framework asks for it. Every reviewer nods at it.

Then it ships, and the agent turns out to be right most of the time — which is why you deployed it. And somewhere around the fortieth approval of the week, a thing happens that no architecture diagram records: your reviewer stops reading.

The click still happens. The audit log still records an approval. The control is now theatre, and the log is now evidence of something that did not occur.

Why being right erodes the control

This is not carelessness, and treating it as a discipline problem is how teams fail to fix it.

A reviewer’s job at the gate is to detect the rare wrong proposal. Their capacity to do that is driven by their prior. When the last thirty proposals were correct, the thirty-first arrives against an overwhelming expectation of correctness, and a human evaluating a low-information decision against a strong prior does the rational thing: they spend less on it. Reading becomes skimming. Skimming becomes recognising the shape of a familiar proposal. Recognising becomes clicking.

The uncomfortable part is the direction of the effect. A more accurate agent degrades its own oversight faster. An agent that was wrong a third of the time would keep its reviewers sharp. The good one puts them to sleep.

So the gate is strongest exactly when you need it least, and weakest exactly when a subtly wrong proposal finally arrives. That is not a control. That is a control-shaped object.

The number that tells you it already happened

You do not need a survey. Two measurements:

Rejection rate. Over the last hundred approvals, how many were rejected? If the answer is zero or one, the gate is not filtering. It is a logging mechanism with a human-shaped delay in front of it.

Time to decision. Median seconds between the proposal appearing and the click. If a reviewer decides a multi-step remediation in four seconds, they did not evaluate it. They recognised it.

A third, if you want the uncomfortable version: rejection concentration. If one person accounts for nearly all rejections, the control is that person, not the process — and it goes on holiday when they do.

I’d argue a gate with a zero rejection rate is worse than no gate at all. With no gate, everyone downstream knows the action was automatic and treats it accordingly. With a rubber-stamped gate, the audit trail asserts that a qualified human reviewed and approved — and that assertion is now false in a document you will hand a regulator.

The fix is fewer gates, not better reviewers

The instinct is to fix the human: better UI, more context in the proposal, training, a checklist. Those help at the margin and none of them survive volume, because the mechanism is not ignorance. It is economics.

The actual fix is to stop sending routine decisions to the gate.

Route by reversibility, not by uniform policy. Not every action deserves the same treatment, and pretending otherwise is what generated the volume:

  • Reversible, verifiable, tested rollback → execute automatically, report after the fact. Restarting a stateless pod. Scaling out. Requeuing a job. Being wrong here costs a few seconds and an audit line.
  • Recoverable but expensive → execute within a budget, escalate past it. Rolling back a deploy at 2 a.m. is probably fine; the fifth rollback this hour is a signal, not a remediation.
  • Irreversible → always a human, forever, with no volume pressure on that decision because the routine traffic no longer flows through it.

The point of removing the routine cases is not throughput. It is that the remaining gate is rare enough that people still read it. Ten approvals a week where the reviewer genuinely might say no is a functioning control. Two hundred where they never do is a liability with better ergonomics.

This is the same argument as the autonomy ladder, approached from the other end. The ladder asks what an agent has earned. This asks what your humans can still actually perform.

The honest objection

“You’re describing how to give an agent more authority, dressed up as a safety argument.”

Partly, yes — and the direction matters. I’ve argued at length that an agent should not be allowed to act until the reversibility tiers, the verification probe, the tested rollback, the attributable audit record, and a working kill switch all exist. That is a high bar and most teams do not clear it.

But once you have cleared it, keeping every reversible action behind a human click does not buy safety. It buys the appearance of safety, at the cost of the attention you need for the decisions that actually matter. You have spent your reviewer’s scrutiny on pod restarts and have none left for the schema migration.

The failure mode I’d genuinely worry about is a team that reads this, removes the gates, and has not built the five things. Then you have not routed the volume away from a tired human. You have removed the only thing catching the agent at all.

What to do Monday

  1. Pull the numbers. Rejection rate and median time-to-decision over the last hundred approvals. Do not estimate them.
  2. If almost nothing is being rejected, say out loud that the gate is not currently a control. I treat a rejection rate near zero as the threshold, but the exact number matters less than the honesty of the sentence — that is the hard part.
  3. Tier your actions by reversibility and pick the smallest set that could execute automatically — the ones with a probe and a tested inverse.
  4. Move exactly those, and keep everything else behind the gate.
  5. Re-measure in a month. If the rejection rate on the remaining gate has gone up, the control came back to life. That is the number you were trying to move all along.

The goal was never to maximise the number of approvals. It was to make sure that when a human is asked, they are able to answer.

Frequently asked questions

What is approval fatigue in AI agent governance?

Approval fatigue is the degradation of a human-in-the-loop control under volume. The first few approvals get genuine scrutiny because the reviewer is uncertain. As the agent proves itself right most of the time, the reviewer's prior shifts, reading gives way to skimming, and skimming gives way to clicking. The control still appears in the architecture diagram and in the audit log, but it has stopped filtering anything — which is worse than having no gate, because everyone downstream believes a human checked.

Does requiring human approval make an AI agent safer?

Only at low volume, and only where the reviewer can actually evaluate the proposal. Approval is a real control when a human has the context, the time, and a genuine expectation of sometimes saying no. At high volume on routine, low-risk actions it becomes a reflex, and a reflex is not oversight. The fix is not more approvals — it is fewer, aimed at decisions that deserve them, with the routine reversible actions moved to bounded automatic execution.

How do you tell whether an approval gate is still working?

Measure the rejection rate and the time-to-decision. A gate with a rejection rate near zero and a median decision time of a few seconds is not filtering anything — it is a logging mechanism with extra latency. Also track whether rejections cluster in one reviewer: if one person accounts for nearly all of them, the control depends on that individual rather than on the process, and it disappears when they go on leave.

What should replace blanket human approval for AI agent actions?

Route by reversibility rather than by uniform policy. Reversible actions with a working verification probe and a tested rollback execute automatically and are reported after the fact. Irreversible actions always require a human. The gate that remains is rare enough that people still read it, and the volume that used to erode attention now runs through a path where being wrong is recoverable rather than one where it is merely witnessed.

Comments