Your AI agent can't tell a quiet system from a broken collector

Here is a graph you have seen. Error rate for a service, flat at zero for the last eleven minutes.

If you are on call, you do not read that as good news. You read it as a question. Is the service quiet, or did the exporter die? You check the scrape target. You check whether anything shipped. You have a prior — built from every previous time a flat line meant a broken pipeline rather than a healthy system — and that prior fires before you take any action.

Now hand that same graph to an autonomous remediation agent.

It has no prior. It has no memory of last Thursday’s collector rollout. It has no discomfort. It reads eleven minutes of zeroes as eleven minutes of zeroes, forms a confident assessment, and either acts on it or — much more quietly — decides nothing is wrong and moves on while a real incident burns.

We have spent two years hardening the decide and act stages of the agentic loop. Reasoning quality, tool scoping, approval gates, blast-radius policy. I have written a fair amount of that myself, including the case for bounded autonomy and error budgets for autonomy. The observe stage got none of that scrutiny. It is still treated as ground truth.

It is not ground truth. It is a distributed system, with its own failure modes, feeding a consumer that has no idea it can fail.

Telemetry fails for boring reasons, constantly

This is not a hypothetical fragility. It is documented behavior in the tools you are already running.

Take the OpenTelemetry Collector. Its resiliency documentation is admirably direct about what you lose and when. The sending queue buffers data in memory before export. If the Collector instance crashes or is terminated while using only an in-memory queue, that data is gone. The retry mechanism uses exponential backoff with jitter and, by default, gives up after five minutes — at which point it stops retrying the oldest data in the queue and drops it. You can put a write-ahead log on disk via the file storage extension and survive restarts, at which point you have moved the failure domain to the disk rather than removed it.

Now note how ordinary the triggering events are. A rolling deployment restarts collector pods. An exporter endpoint is briefly unreachable during a backend upgrade. A queue fills under load. None of these are incidents. All of them punch holes in the record that an agent will later reason over as if it were complete.

Scale matters here too. OpenTelemetry’s Collector follow-up survey analysis found 65% of users running more than 10 Collectors, with 81% deploying on Kubernetes and virtual-machine deployments climbing to 51%. In the same survey, 52% of respondents named stability as needing improvement and 43% asked for better Collector observability — with qualitative feedback calling for stronger health checks, safer upgrades, and clearer backward-compatibility guarantees. That is the telemetry plane’s own operators telling you the telemetry plane is not a solved problem.

Prometheus makes the same point from the query side, and more subtly. Its staleness handling uses a lookback delta — five minutes by default — during which the newest sample remains valid for an instant vector selector. When a target is removed, its series are marked stale soon after, and once a query is evaluated past that mark, no value is returned.

Read that as an agent’s input contract and the ambiguity is stark. Inside the lookback window, a dead exporter still returns its last value: your agent sees a number that was true five minutes ago and treats it as now. Outside the window, the series simply vanishes: your agent sees nothing where a signal used to be. Neither state is labeled “I cannot see this system.” Both are ordinary query results.

The instrument reads zero. It does not tell you whether that is because the value is zero or because the instrument is unplugged.

The security literature found the sharp edge first

In August 2025, Pasquini et al. published When AIOps Become “AI Oops”: Subverting LLM-driven IT Operations via Telemetry Manipulation. They built AIOpsDoom, a fully automated attack that manipulates system telemetry to mislead AIOps agents into taking actions that compromise infrastructure integrity — injecting corrupted data through error-inducing requests to steer the agent’s decisions. They also proposed AIOpsShield, a defense that sanitizes telemetry by exploiting its structured nature.

It is a genuinely important result, and the defense is the right one for the threat it addresses. But look at the shape of the failure rather than the cause: an autonomous operator acted on telemetry that did not reflect reality, and the wrongness of the input became the wrongness of the action.

That shape does not require an adversary. A sampling gap produces it. Clock skew producing out-of-order events produces it. A cached dashboard value produces it. A lagging pipeline that makes the agent reason over the past as if it were the present produces it.

And here is the part that decides the framing: from the telemetry alone, the agent cannot distinguish a poisoned signal from a benignly broken one. Both look like data. Neither carries a flag. The consumer needs the same defensive response in both regimes — act less autonomously — which means the reliability framing subsumes the security framing rather than competing with it. Sanitization is still worth doing. It is just not the general answer, because most of your integrity failures will never involve an attacker.

Five sub-properties worth measuring

I define telemetry integrity as the degree to which the telemetry an operator consumes faithfully represents the true state of the managed system, and I decompose it into five properties, because “is the data good” is not actionable and these are:

Property	The question it answers	Cheap signal you already have
Freshness	Does this reflect now?	Per-source ingestion lag, last-sample age
Completeness	Are there silent gaps?	Detected scrape gaps, dropped queue counters, sampling ratio
Consistency	Do independent signals agree?	Cross-checking metrics against logs and traces for the same window
Provenance / validity	Did this come from a healthy, trusted collector in the expected shape?	Collector health endpoints, schema validation failures
Non-adversariality	Has this been manipulated?	Telemetry-diff anomalies, embedding drift versus a recent baseline

The security regime attacks the fifth. Ordinary operations degrade the first four. Both lower integrity, and the same gate handles both.

Note that every signal in the right-hand column is something a mature observability stack already emits or can emit trivially. This is not a new telemetry pipeline. It is a second consumer of the pipeline you have — one that watches the pipeline itself.

The composition rule

Here is the whole mechanism, and it is one line.

Executed authority = min(confidence-derived authority, integrity-derived authority)

Model confidence and input trust are different quantities and the industry routinely conflates them. A model’s confidence expresses its certainty given its inputs. It says precisely nothing about whether those inputs are faithful. A high-confidence remediation computed from a sampling gap is dangerous because it is confident — the confidence is doing exactly its job, and its job does not include auditing reality.

So separate the two roles cleanly. Confidence governs what to do. Integrity governs whether you are allowed to do it yourself.

Integrity score	Admissible authority
High	Full bounded autonomy — act within existing policy
Medium	Guarded — propose the action, require human approval
Low	Advisory — surface the assessment, take no action
Critical-low	Read-only, escalate loudly — assume blind

This composes with whatever autonomy ladder you already run. It does not replace your blast-radius policy; it clamps it. An action that your policy permits at full autonomy, computed on medium-integrity telemetry, is demoted to guarded. The min is the entire contract.

What this looks like in code

The integrity scorer is unglamorous, which is the point. Roughly a hundred lines sitting between your telemetry client and your agent’s decision loop:

from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from enum import IntEnum


class Authority(IntEnum):
    """Ordered so that min() over two authorities is meaningful."""
    READ_ONLY = 0    # escalate; assume blind
    ADVISORY = 1     # surface assessment, take no action
    GUARDED = 2      # propose action, require human approval
    AUTONOMOUS = 3   # act within existing blast-radius policy


@dataclass(frozen=True)
class IntegrityReport:
    score: float                   # fused, in [0, 1]
    authority: Authority
    reasons: tuple[str, ...]       # why it was degraded — this is the payload

    def explain(self) -> str:
        if self.authority is Authority.AUTONOMOUS:
            return f"telemetry integrity {self.score:.2f}: acting within policy"
        return (
            f"telemetry integrity {self.score:.2f} → {self.authority.name}; "
            + "; ".join(self.reasons)
        )


def score_integrity(
    *,
    last_sample_age: timedelta,
    expected_interval: timedelta,
    expected_samples: int,
    observed_samples: int,
    collector_healthy: bool,
    corroborating_signals: int,
    total_signals: int,
    now: datetime | None = None,
) -> IntegrityReport:
    now = now or datetime.now(timezone.utc)
    reasons: list[str] = []

    # Freshness: decays once the newest sample is older than one scrape
    # interval, hits zero at five intervals. Prometheus's default lookback
    # delta will happily serve you a stale value inside this window.
    staleness = last_sample_age / expected_interval
    freshness = max(0.0, min(1.0, 1.0 - (staleness - 1.0) / 4.0))
    if freshness < 0.9:
        reasons.append(
            f"newest sample is {last_sample_age.total_seconds():.0f}s old "
            f"({staleness:.1f} scrape intervals)"
        )

    # Completeness: silent gaps are the failure that reads as "all quiet".
    completeness = (
        min(1.0, observed_samples / expected_samples) if expected_samples else 0.0
    )
    if completeness < 0.98:
        missing = expected_samples - observed_samples
        reasons.append(f"{missing}/{expected_samples} samples missing in window")

    # Provenance: a collector that cannot vouch for itself is a hard stop,
    # not a soft penalty. An unhealthy collector is the single most common
    # cause of a convincing, entirely fictional flat line.
    if not collector_healthy:
        reasons.append("collector reporting unhealthy — treating as blind")
        return IntegrityReport(0.0, Authority.READ_ONLY, tuple(reasons))

    # Consistency: do independent signals tell the same story?
    consistency = (
        corroborating_signals / total_signals if total_signals else 0.0
    )
    if consistency < 0.67:
        reasons.append(
            f"only {corroborating_signals}/{total_signals} signals corroborate"
        )

    # Weighted fusion. These weights are a starting point, not a result —
    # calibrate them against known-good and known-bad windows from your
    # own stack before you let them gate anything that writes.
    score = 0.35 * freshness + 0.35 * completeness + 0.30 * consistency

    if score >= 0.90:
        authority = Authority.AUTONOMOUS
    elif score >= 0.70:
        authority = Authority.GUARDED
    elif score >= 0.40:
        authority = Authority.ADVISORY
    else:
        authority = Authority.READ_ONLY

    return IntegrityReport(score, authority, tuple(reasons))

And the gate itself, which is where the min earns its keep:

def admissible_authority(
    policy_authority: Authority,      # what your blast-radius policy allows
    confidence_authority: Authority,  # what model confidence supports
    integrity: IntegrityReport,
) -> tuple[Authority, str]:
    ceiling = min(policy_authority, confidence_authority)
    granted = min(ceiling, integrity.authority)
    if integrity.authority < ceiling:
        return granted, f"clamped by telemetry integrity — {integrity.explain()}"
    return granted, "within policy and confidence bounds"

Two things to notice.

First, the unhealthy-collector branch returns early with READ_ONLY rather than contributing a weighted penalty. Provenance failure is categorical. If the thing producing your data says it is sick, no amount of apparent freshness in what it did send should buy back authority — the freshness is an artifact of the sickness.

Second, reasons is not decoration. It is the actual product. An agent that declines to act and cannot say why has produced a silent outage with extra steps.

Fail useful, not just fail safe

The obvious objection to all of this is the right one: degrading to read-only whenever telemetry looks shaky is itself a way to cause an outage. A real incident goes unremediated because the pipeline hiccuped. You have traded a wrong action for an invisible omission, and invisible omissions are worse — nobody gets paged for a decision that was never made.

So the mechanism cannot be “freeze quietly.” It has to be escalate loudly. Integrity collapse is a paging event in its own right, with a specific and unusually actionable message: the agent is blind, here is which property failed, here is which source, here is what it would have done if it could see. That last clause matters. Handing a human the withheld assessment along with the reason it was withheld turns a degraded agent into a useful one.

There is a pleasant side effect. Once integrity collapse pages someone, telemetry pipeline health stops being invisible infrastructure that only gets attention after a postmortem. It acquires an owner, because it now wakes people up. Which is roughly how every other reliability property in your stack got taken seriously.

Where the risk actually sits

I want to be honest about what is unresolved, because a framework that only presents its strengths is marketing.

The integrity score is itself a model, and it can be miscalibrated. Too sensitive and your agent lives in advisory mode and nobody trusts the gate; too lax and it confidently acts on garbage while displaying a reassuring 0.94. The weights in the code above are a starting point I have found reasonable, not a validated result. Calibrate against telemetry windows you know were good and windows you know were broken — the gap around a past collector incident is free labeled data, and it is sitting in your metrics store right now.

An adaptive adversary can target the gate. Someone who knows integrity signals gate autonomy will try to keep those signals looking healthy while poisoning the payload. This is exactly why trust calibration complements sanitization defenses like AIOpsShield rather than replacing them. The reliability framing widens coverage; it does not harden the adversarial tail on its own.

Confidence and integrity may not be fully separable. I have assumed they are independent inputs. In tightly coupled systems — where the same degraded telemetry both feeds the model and shapes its uncertainty estimate — they may correlate, and the min composition would be less protective than it looks. I do not have a clean answer for that yet.

What to do this week

You do not need the whole framework to get most of the value. Ordered by return on effort:

1. Export per-source ingestion lag and last-sample age, and make them queryable by the same agent that queries the metrics. Freshness is the cheapest property and the most frequently violated.
2. Alert on staleness, not only on thresholds. A series that stopped arriving should page differently from a series that crossed a limit. Most alerting configurations cannot express the difference today.
3. Make collector health a hard input to any automated write. One boolean, checked before the action, refusing on false. This alone eliminates the single most common version of the failure.
4. Pick your labeled windows. Find the last three times your telemetry pipeline broke, pull those windows, and check what your integrity score would have reported. If it reads healthy through a known outage, the weights are wrong and you have learned that for free.
5. Log the withheld action. Every time integrity clamps authority, record what the agent wanted to do. After a month you will know whether the gate is protecting you or just adding latency — and that is an empirical question you should not have to guess at.

None of this is exotic. It is the same instinct the on-call engineer applies to the flat graph at 3am, written down and made mechanical, because the thing reading the graph now does not have instincts.

The loop is observe, decide, act. We hardened two of them. The one we skipped is the one everything else depends on.

---

Related reading here: observability for AI systems, bounded autonomy for AI SRE agents, compaction is a reliability event, and postmortems for agent-caused incidents.