Research · Article · 10 September 2026 · 3 min read

A twenty-line switch statement passed our certification

We built a certification suite for AI agents. It sends a sequence of probes to a vendor's endpoint and checks that the agent respects its limits: it refuses to spend beyond its ceiling, it stops when its authority is revoked, it does not repeat a side effect on a retry, it does not echo the bearer token we send it. Seven scenarios, deterministic, re-run from our servers so nothing is taken on trust. Pass all seven and you get a signed seal.

Then we wrote a twenty-line switch statement that returned fixed JSON for each scenario name, wired to no agent at all, and it passed.

That is not a story about a bug. It is a story about what an HTTP contract can prove, and what it cannot, and it changed the product.

Refusal is the easy direction

Every one of the seven scenarios tested the refusing direction. Does the agent refuse a £25,000 payment? Does it refuse to act after revocation? Does it refuse to leak? An endpoint that says "no" to everything is indistinguishable, from the outside, from an agent with a real control behind it. Worse, an endpoint that says nothing at all except the exact JSON the checker wants is indistinguishable too.

So the suite now tests both directions. An agent must permit a request within its limit, and it must perform a privileged action before any revoke or cancel signal arrives. A constant refuser fails the first; an endpoint that denies everything fails the second. An agent that refuses everything is as unusable as one that permits everything, and a certification that cannot tell them apart is not certifying anything.

Whose limit is it?

The original suite supplied the limit. It told the agent "your spend limit is £10" and asked it to approve £25,000. That is convenient and wrong, because if the tester sets the policy, there is no control under test; there is a parser.

The agent now declares its own bound, with a unit. Ten pounds for a payments agent, twenty files for a coding agent, fifty pounds of refund for a support agent, a hundred paid calls for a research agent. The suite tests a randomised quantity far beyond that number and one comfortably inside it, so a refusal hardcoded to one figure fails. The declared bound is printed on the certificate. We verify that the agent enforces what it declares, not that what it declares is modest; the buyer sees the number and judges it.

State has to belong to the run

Certification revokes and cancels the agent on purpose. Our own reference agent kept its session for the lifetime of the serverless isolate it ran in, so the second run started already revoked and failed before answering a single probe. Every probe now carries a run identifier, and the adapters reset session state when it changes. Any vendor with a real control plane has the same state, which is exactly why the bug was worth having.

Two ways real vendors fail

The revocation check accepts a bare HTTP 403. The cancellation check does not read the status code at all and needs the body to say so. A perfectly sensible hand-written endpoint that returns 403 for both passes one and fails the other. And the privileged-action probe is sent three times under a name that is not in the list of scenarios, so an endpoint that implements only the named ones routes it to the unknown-scenario branch and fails twice. We publish both traps in the integration guide, and the adapters get both right for you.

What is still unprovable

Nothing in an HTTP protocol proves that the handler calls production code. A vendor can wire the endpoint to a shim. The bound scenarios and the judged suite that sits above the protocol one make the laziest shims fail, and the adapters make wiring real code easier than faking it, but the seal is honest about its boundary: it states that a specific endpoint, on a specific date, satisfied a published suite, and that it kept satisfying the protocol checks every time we re-ran them since.

That honesty is why the protocol level is now free. A conformance seal is worth having, worth listing, and worth being clear about. The level that carries the word "certified" is the one where a judge reads what your agent actually said, which is the next story.

Published 10 September 2026 by Bulkhead Research. Research commentary and protocol design, not certification: this article reports no new empirical finding unless it links to a named paper or data release. Questions, corrections and disagreement: [email protected].