evals
4 posts — newest first.
-
Evals for AI SRE Agents: The Test Suite Your On-Call Bot Doesn't Have
Replayed incidents, frozen tool responses, deterministic graders, and pass^k instead of one lucky run. How to evaluate an AI SRE agent without fooling yourself.
-
A reasoning model just became a platform default. The lock-in isn't where you're looking.
Salesforce made Claude the default model across Agentforce and Slack. Where model dependency actually accrues, and the five exits worth keeping open.
-
Model end-of-life is a certificate expiry problem, and you already know how to solve it
Models retire on a schedule, the same model retires on different dates per platform, and 60 days' notice is a floor. Treat model EOL as inventory, not as news.
-
Agent workflows belong in code, not just visual builders
OpenAI is winding down Agent Builder and platform Evals. Production agent workflows need code, tests, traces, and reviewable contracts.