Reliability
27 posts — newest first.
-
Agent workflows belong in code, not just visual builders
OpenAI is winding down Agent Builder and platform Evals. Production agent workflows need code, tests, traces, and reviewable contracts.
-
Agents are deleting production and nobody is writing the postmortem
Your incident template assumes a deterministic system and a human decision-maker. Agent incidents have neither. Five sections that fix the template.
-
The biggest outages still start with a config change — and now agents write them
Fresh postmortems keep confirming it: config changes, not code, cause the largest incidents. Agentic ops multiplies the volume. Here's the defense.
-
Google just published the reference architecture for agentic SRE
Google SRE splits autonomous incident response into reasoning, actuation, and evaluation. The split is the lesson — here's what to copy at normal scale.
-
The EU AI Act blinked — your logging requirements didn't
The EU AI Act omnibus moved high-risk deadlines to 2027–28. The logging, oversight, and inventory work is still yours — and reliability needs it anyway.
-
Every observability vendor now sells an AI SRE agent. Here's how to evaluate one.
Datadog, Dynatrace, New Relic, AWS — every incumbent now ships an AI SRE agent. A field guide for evaluating one before it touches production.
-
Frontier models still fail half your incidents: reading ITBench-AA like an SRE
ITBench-AA put frontier models against 59 real Kubernetes incident diagnoses — all scored below 50%. What the benchmark measures and how to use it.
-
Autonomy is a budget, not a toggle: error budgets for AI operators
SRE solved runaway release risk with error budgets. The same mechanism governs AI agents: authority per action class, demoted fast, promoted slow.
-
The reliability gap: a framework for trusting autonomous SRE agents
An autonomous airline agent rebooked 1,247 passengers wrong in one weather event. Trusting agents is a reliability problem — here's how to measure it.
-
Agentic AI Patterns: Where They Break in Production (Part 2 of 3)
Every agentic AI pattern looks clean in a demo. Where each one breaks in production, the signals you're hitting them, and mitigations that actually work.
-
Chaos engineering for MCP: break your tool-call plane before production does
LLM calls fail 1–5% of the time and agent tasks fan out into 10–20 tool calls. How to fault-inject your MCP layer with mcp-chaos before production does.
-
The trust gap: bounded autonomy for AI SRE agents
SREs face 50+ alerts a day at 60% false positives while vendors promise autonomous resolution. The autonomy ladder: what an AI agent should never do alone.
-
The AI-native SRE stack — a 2026 reference guide
A practitioner's map of the AI-native SRE stack in 2026: six layers from telemetry to bounded remediation, and an honest read on where AI pays off.
-
Context engineering: the window is a budget, not a bucket
The context window is your agent's working memory, not a junk drawer. Four operations — write, select, compress, isolate — and a token budget you allocate.
-
Agent sprawl is your next production incident
Teams shipping AI agents are recreating 2015's microservices sprawl with worse observability. The governance surface that contains it before it pages you.
-
Observability for AI systems — what changes when your service calls an LLM
Golden signals miss the failure that pages you: a confident, well-formed, wrong answer. What AI observability adds — context as a span, quality as a signal.
-
Harness engineering: the third phase of AI maturity
Agent = Model + Harness, and in 2026 the harness is the bottleneck. What a production-grade SRE harness contains, with a ~40-line reference implementation.
-
Toil and the 50% rule — what it is, how to measure it, and how to kill it
A primer on toil — the manual, repetitive work that eats SRE teams. Google's six-part definition, the 50% cap, and how AI agents change the playbook.
-
SLI, SLO, SLA, and error budgets — the reliability contract explained
SLIs, SLOs, SLAs, and error budgets — the four numbers every SRE team must agree on, the math behind 'nines,' and what changes when agents burn the budget.
-
What is Site Reliability Engineering (SRE)?
A primer on Site Reliability Engineering — where it came from, how it differs from DevOps and Platform Engineering, and what changes as AI joins on-call.
-
Mental models for applying AI to infrastructure
Tutorials answer how; mental models answer whether. Seven I use as the front gate before any LLM goes near a production system.
-
Prompt engineering for SRE: patterns that actually work in production
Prompt advice is written for chatbots; SRE workloads are different. Six patterns I've shipped to production for SRE LLM tools, and why each earned its place.
-
The MCP gateway pattern: five jobs your agent runtime can't skip
Letting agents call MCP servers directly repeats the no-API-gateway mistake. The five jobs an MCP gateway must do, with reproducible patterns for each.
-
Skills for AI agents that do SRE work
Most agent skills are chatbot prompts in disguise. Three operator-grade SRE skills — opinionated, output-contracted, portable across agent runtimes.
-
When NOT to Use AI in Production SRE
Most AI-for-SRE writing tells you where AI helps. Here are seven places it actively hurts — and the operational rule of thumb I use to decide.
-
Garbage Collection: The Convenience That Shows Up in Your p99
GC frees you from manual memory management — and occasionally freezes your program. Reachability, generational GC, and why GC tuning is a p99 problem.
-
Deadlocks, Locks, and Race Conditions: The Bugs That Only Happen Sometimes
Races and deadlocks only fail sometimes — that's what makes them hard. Locks, atomicity, the four Coffman conditions, and why hung agents are the same bug.