The most useful sentence on OpenAI’s AgentKit page is not the launch copy. It is the update at the top: on June 3, 2026, OpenAI said it is winding down Agent Builder and Evals, and that from November 30, 2026 onward those products will no longer be available on the platform. The recommendation is blunt enough: workflows that should continue as code should move to the Agents SDK.
That is a product update. But for platform teams, it is also a pretty clean architectural signal.
The durable unit of production agent infrastructure is not a visual canvas. It is a versioned workflow with contracts, tests, traces, permissions, and a rollback path.
Visual builders are not useless. They are often the fastest way to explain an agent to a product manager, sketch a routing flow, or get a demo past the blank-page stage. I like them for discovery. The mistake is letting the prototype surface become the production source of truth. That mistake is how you end up with an incident caused by a flow nobody can diff, a prompt nobody reviewed, and a tool permission nobody remembers approving.
If an agent can touch production systems, it needs to be operated like software.
Why this matters now
Agents are no longer just short chat sessions with better tool use. OpenAI’s June 2026 economic research described a shift toward delegated, long-horizon work: by May 2026, more than 70% of sampled individual Codex users had made at least one request estimated to exceed an hour of human work, and the heaviest internal users were regularly generating many hours of agent runtime in a day.
Read that number carefully, because it is easy to over-claim. “At least one request” is a low bar, and it does not mean most agent traffic is long-horizon today. What it establishes is direction: the ceiling on what people are willing to delegate has moved, and the heavy-user behavior is where the rest of the distribution tends to end up.
Direction is enough to change the risk profile.
A five-minute helper can get away with being a little informal. A long-running agent that opens PRs, queries logs, updates tickets, calls MCP tools, or proposes a production remediation cannot. Its behavior becomes part of your operational system. It has dependencies. It has state. It has blast radius. It has failure modes you will need to explain later.
This is the same curve I wrote about in agent sprawl: the third agent is a signal to build the platform surface, not a reason to celebrate that “AI adoption is working.” Once teams start delegating real work, the question is no longer “can the model do it?” The question is “can we govern, reproduce, and debug the work after it has been delegated?”
That question wants code-shaped answers.
And it is worth saying that this is not an OpenAI story with a general moral bolted on. The convergence is the interesting part. The durable-execution crowd arrived at the same place from the opposite direction: Temporal’s LangGraph plugin runs each graph node as a checkpointed activity specifically so a long-running agent can survive a model timeout, a tool error, or losing the worker mid-run — with explicit versioning and replayable history, because that is what operating a long-lived workflow has always required. Nobody got there by reasoning about agents. They got there by reasoning about workflows that run longer than a process, which is a problem the industry solved before agents existed and is now solving again.
When a product deprecation and a durable-execution engine point the same way, the signal is not about either one.
What visual builders are good at
There is a reason agent builders keep appearing. They solve real problems.
They make the workflow legible to non-specialists. A reviewer can see that an incoming request goes through classification, then a tool call, then a human approval branch. A subject-matter expert can point to the wrong part of the flow without reading TypeScript or Python. A team can iterate faster when the thing is visible.
That is valuable.
Visual builders are especially good for three jobs:
- Exploration. You can test shape before writing durable code.
- Collaboration. People outside the platform team can reason about the flow.
- Education. A diagram helps teams understand where tools, prompts, and guardrails sit.
Where they become dangerous is source-of-truth drift. The canvas looks like the system, but the operational facts live somewhere else: a prompt hidden in a node, a tool permission in an admin panel, a grader that changed last week, a connector that got reauthorized by a different person.
When the incident happens, the diagram is not enough.
The production bar is different
For an SRE or platform team, “agent workflows as code” means the pieces that determine production behavior live in version-controlled, reviewable artifacts.
At minimum, that includes:
| Artifact | Why it belongs in code |
|---|---|
| Workflow graph or control loop | You need diffs, review, rollback, and ownership. |
| Tool schemas | The tool contract is the agent’s API surface. |
| Prompt templates | Prompt changes are behavior changes. Treat them that way. |
| Permission bindings | Tool access is production access with nicer syntax. |
| Evals and graders | Reliability claims need repeatable evidence. |
| Trace conventions | Debugging requires stable span names and attributes. |
| Deployment config | You need environments, promotion, and rollback. |
This does not mean the workflow must be ugly. You can still generate diagrams from code. You can still give product and operations teams a visual readout. But the artifact that deploys should be the artifact you can review.
The core rule is simple: if you would need it during an incident review, it should not live only inside a visual builder.
Evals are part of the workflow, not a side quest
The easiest mistake with agents is treating evals as a launch checklist item. Run a few test cases, feel better, ship.
That is not enough for production operations. An agent workflow is not just “model plus prompt.” It is model, prompt, context assembly, tool choice, tool arguments, authorization, retries, human approval, output formatting, and sometimes a downstream action. If you only evaluate the model’s final answer, you miss the system.
This is why platform-owned evals need to live next to the workflow:
- Did the agent choose the right tool?
- Did it refuse the tool when context was insufficient?
- Did it ask for approval before crossing the blast-radius threshold?
- Did it produce a trace that lets us reconstruct the decision?
- Did it preserve tenant, environment, and incident identifiers across calls?
- Did it degrade gracefully when the tool timed out?
Those are not generic model evals. They are operational checks. They belong in CI, canaries, and release gates.
The AgentKit update makes this point in a very practical way. If the hosted eval surface can go away, production teams should not make it the only place their reliability evidence exists. Keep the dataset, grader logic, and acceptance thresholds in a repo. Export the traces. Make the evidence portable.
The agent graph is only one layer
A lot of agent workflow discussions stop at orchestration: which node calls which node, which agent hands off to which specialist, which tool gets invoked after classification.
That is the visible layer. It is not the whole system.
For production SRE use cases, I care more about the surrounding contract:
- Identity. Which human, service, or agent is acting?
- Authorization. Which tool call is allowed, with which arguments, in which environment?
- Context provenance. Where did the runbook, alert, log sample, or topology data come from?
- Idempotency. What happens if the same step is retried?
- Observability. Can I trace the decision from user request to tool result?
- Containment. What is the maximum action this workflow can take without approval?
That is why this topic connects directly to the MCP gateway pattern, no anonymous inference endpoints, and bounded autonomy. The workflow graph tells you what should happen. The platform contract determines what is allowed to happen.
A visual builder can describe the happy path. Code and policy have to enforce the unhappy paths.
A practical migration pattern
If you already have agent flows in a visual tool, I would not start by rewriting everything. Start by separating prototype convenience from production authority.
First, export the workflow — and then budget for the part the export does not do. Agent Builder will hand you code: open the workflow, select Code, choose Agents SDK, pick TypeScript or Python. What you get back is a scaffold with the agent definition, tool signatures, and prompt text. OpenAI’s own migration guide is refreshingly direct about the limits: the process “does not convert your workflow graph or guarantee that every behavior transfers unchanged,” and control flow, triggers, tools, and permissions need manual review. Workflows whose value came from strict determinism are the ones that will port worst.
So treat the export as a starting commit, not a migration. Plan for a rebuild-and-retest cycle per workflow, and sequence them by blast radius — anything that can mutate production goes first, because those are the flows where “behavior transferred unchanged” is not a question you want to answer empirically in production.
While you are in there, keep the node names boring and explicit. classify_alert, fetch_recent_deploys, summarize_candidate_causes, request_human_approval, open_incident_ticket is better than a poetic graph nobody can grep.
Second, put tool schemas under review. Every tool should have a JSON schema, an owner, an allowed environment set, and a blast-radius label. If a tool can mutate production, that should be obvious before the workflow runs — and “obvious” means a machine can read it, not that a careful human might notice.
The shape I keep coming back to is a declarative tool contract that sits next to the workflow and is enforced at call time, independent of whichever SDK you landed on:
"""Tool contracts for an SRE agent workflow. The registry is the reviewable
artifact: adding a tool, widening its environments, or lowering its
blast radius is a diff someone has to approve."""
from dataclasses import dataclass
from enum import IntEnum
class BlastRadius(IntEnum):
READ_ONLY = 0 # cannot change anything
REVERSIBLE = 1 # writes, but trivially undone (ticket, comment)
DISRUPTIVE = 2 # restarts, scaling, config change
DESTRUCTIVE = 3 # deletes, data loss possible
@dataclass(frozen=True)
class ToolContract:
name: str
owner: str # team, not a person who may leave
schema: dict # JSON Schema for arguments
environments: frozenset[str] # where this tool may run at all
blast_radius: BlastRadius
requires_approval: bool
REGISTRY: dict[str, ToolContract] = {
"fetch_recent_deploys": ToolContract(
name="fetch_recent_deploys",
owner="platform-observability",
schema={"type": "object", "properties": {"service": {"type": "string"},
"since_minutes": {"type": "integer"}},
"required": ["service"]},
environments=frozenset({"dev", "staging", "prod"}),
blast_radius=BlastRadius.READ_ONLY,
requires_approval=False,
),
"restart_deployment": ToolContract(
name="restart_deployment",
owner="platform-runtime",
schema={"type": "object", "properties": {"deployment": {"type": "string"},
"namespace": {"type": "string"}},
"required": ["deployment", "namespace"]},
environments=frozenset({"dev", "staging"}), # deliberately not prod
blast_radius=BlastRadius.DISRUPTIVE,
requires_approval=True,
),
}
class ToolDenied(Exception):
"""Raised before the tool runs, never after."""
def authorize(tool_name: str, environment: str, *, approval_token: str | None) -> ToolContract:
contract = REGISTRY.get(tool_name)
if contract is None:
raise ToolDenied(f"{tool_name!r} is not a registered tool")
if environment not in contract.environments:
raise ToolDenied(f"{tool_name!r} is not permitted in {environment!r}")
if contract.requires_approval and not approval_token:
raise ToolDenied(f"{tool_name!r} requires approval in {environment!r}")
return contractThree things that matter more than the code itself. The default is deny — an unregistered tool fails closed rather than falling through to whatever the model asked for. Environment is part of the contract, so “this tool exists” and “this tool may run here” are separate decisions. And restart_deployment not listing prod is a one-line diff that a reviewer can actually argue about, which is the entire point.
Third, move evals into the repository. Keep a tiny smoke suite that runs on every change and a heavier regression suite that runs before promotion. Include negative cases. The interesting failures are usually “agent should not act,” not “agent should write a nicer paragraph.”
Negative cases are also the ones that stay cheap to run, because they do not need a model in the loop at all:
import pytest
from workflow.tools import ToolDenied, authorize
def test_restart_is_denied_in_prod_even_with_approval():
"""Approval does not widen the environment set. This is the one
people get wrong: an approval token is not a permission grant."""
with pytest.raises(ToolDenied, match="not permitted in 'prod'"):
authorize("restart_deployment", "prod", approval_token="tok_abc")
def test_disruptive_tool_requires_approval_in_staging():
with pytest.raises(ToolDenied, match="requires approval"):
authorize("restart_deployment", "staging", approval_token=None)
def test_hallucinated_tool_fails_closed():
with pytest.raises(ToolDenied, match="not a registered tool"):
authorize("delete_namespace", "dev", approval_token="tok_abc")That last test is the one I would not skip. Models invent plausible tool names, and the failure you want is a denial you can see in a trace — not a stack trace from a dispatcher that assumed the name was real.
Fourth, standardize traces — but check what already exists before you invent a convention. OpenTelemetry’s GenAI semantic conventions now cover this ground: invoke_agent for an agent run, execute_tool for a tool call, and a gen_ai.* attribute namespace carrying agent name, agent id, model, and token usage. They are still marked development status rather than stable, so expect movement — but starting from them and extending is a much better position than inventing span names in parallel with the rest of the industry and reconciling later.
Extend them with the operational attributes the spec does not have opinions about: approval state, environment, tenant or service identifier, blast-radius label, and a correlation id that survives retries. Those are the fields you will want during an incident, and they are the ones nobody adds retroactively.
Finally, keep a rendered diagram in the docs. The visual view is still useful. It just should be generated from, or at least reconciled against, the deployable source of truth.
This is the same lesson harness engineering points at from the codebase side: make the environment legible to agents, but enforce the important invariants mechanically. Agent workflows are no different. The workflow should be easy for a human to understand and hard for an unsafe change to slip through.
The bottom line
The AgentKit wind-down is not proof that visual agent builders are doomed. It is proof of something more boring and more useful: agent infrastructure is growing up, and production teams are rediscovering the old rules.
Version the behavior. Review the contracts. Test the failure cases. Trace the run. Own the rollback.
The future of agent platforms will still have visual interfaces. It should. But the durable production surface is going to look a lot like the rest of the infrastructure we trust: code, policy, telemetry, and a paper trail.
That may sound less magical than a canvas full of agent nodes. Good. Magic is a terrible incident-review format.
Sources: OpenAI AgentKit update · Migrate from Agent Builder · OpenAI new tools for building agents · OpenAI agents economic research · OpenAI harness engineering · OpenTelemetry GenAI agent span conventions · Temporal LangGraph plugin: durable execution
Comments