evaluation
3 posts — newest first.
-
Most of your AI platform's traffic doesn't need a frontier model
Small models got good enough in 2026. But the win isn't using them — it's making model choice a platform tier with a router, eval gate, and demotion path.
-
Every observability vendor now sells an AI SRE agent. Here's how to evaluate one.
Datadog, Dynatrace, New Relic, AWS — every incumbent now ships an AI SRE agent. A field guide for evaluating one before it touches production.
-
Frontier models still fail half your incidents: reading ITBench-AA like an SRE
ITBench-AA put frontier models against 59 real Kubernetes incident diagnoses — all scored below 50%. What the benchmark measures and how to use it.