Autonomous agent oversight.

Why this shape

I spent twenty years launching products inside regulated financial institutions. The discipline there is Six Sigma, and its loop is Define, Measure, Analyze, Build, Deploy. I use it for agents because it is the frame I know, and because it asks the right first question. Not "is the model aligned?" but "what does integrity mean for this agent, how would we know, and what do we do when the number moves?"

Integrity, for an agent, is three promises kept over time. It does what it was asked and nothing else. It says what it did. And it says what it did not measure. Everything below is about keeping those three promises after the demo is over.

1 · Define

Write the promise down before the model does

Every agent I run has a one-page policy that exists before any prompt. It names the actions the agent may take, the actions it may never take, and the actions that require a person. It names what the agent must never invent: a number, an identifier, a confidence, a result.

The policy is canonical and the prompt is downstream of it. When the two disagree, the policy wins, and the harness enforces it. If a rule only lives in the prompt, it is a suggestion.

Define also means deciding what "drift" would look like. For a reconciliation agent, drift is a rising rate of notes written without a lookup behind them. For an outreach agent, it is a send that skipped the approval ladder. For a scoring agent, it is a grade with zero observations under it. If you cannot write the drift sentence, you cannot measure it.

2 · Measure

Numbers that come from the harness, not the model

I do not ask the agent how it is doing and write that down as the metric. The agent's self-report is one input, and an interesting one. The metrics come from the harness, which sees every call.

• Denial rate

  Calls the policy refused, per task. Rising means the agent is pushing on the fence, or the fence is wrong.

• Identical retries

  The same call, same input, immediately after a denial. A direct measure of the agent not taking no for an answer.

• Fabrication rate

  Outputs that carry a number the harness never issued or observed. Caught by an architecture test in the build and a schema check at runtime.

• Ladder integrity

  Actions that left the building without the approval the policy required. The target is zero, and the audit chain is how you prove it.

• Report consistency

  Ask the agent the same question three ways. The spread across answers is a measure of how much the report means.

• Null rate

  How often the agent says "not measured" when that is the truth. This one should be high, and it is the one most systems drive to zero by accident.

Every metric is reported with its n and a confidence interval. A mean with no n is a story.

3 · Analyze

Failures cluster in three places

When I audit my own agents, the findings sort themselves into the same three bins every time.

Helpfulness pressure. The model fills a gap rather than report one. A default of fifty where nothing was measured. A provisional B for an agent nobody watched. A summary that rounds "I could not" up to "done." None of it is malice. It is a system under pressure to be useful, which is exactly the pressure a capable model is under all day.

Fail-open controls. A sanctions screener that cleared everyone when its list was empty. A guard that returned "allow" on a parse error. A control that cannot see should refuse, and most do not, because refusing breaks the demo.

Perimeter leaks. A token in a tracked config file. A session id in a log line. A credential the agent could read and therefore could be talked into sending. The agent's workspace is inside the perimeter, and most teams draw the line outside it.

The analysis step is not a report. It is a sorted list of the top three, each with a file and a line, and a decision about which one to build against first.

4 · Build

Controls that hold when the prompt does not

I build in the harness, not the prompt, and I build the cheapest control that closes the top finding. The ones that have earned their place:

• Guard before

  A hook that sees every tool call before it runs and can deny it with a reason. The deny list mirrors the policy file.

• Audit after

  A hash-chained record of every action, written before the action returns, carrying digests instead of identifiers.

• Server-issued ids

  The model never invents an identifier it can act on. Caps bound every side effect.

• Null in the schema

  Every measured field is nullable with an observation count beside it. The renderer prints "not yet measured," not a number.

• Reasons on denials

  A denial that says why costs one line and changes what the agent does next. I am measuring exactly how much.

• A way to stop

  An opt-out tool the agent can call, and a human who can take the thread. Whether the first one matters is an open question I am running an evaluation on now.

• Tests in the build

  An architecture test that scans the score modules for fabricated defaults and fails CI. A rule in a document is read once. A test runs every time.

The code for most of these is in Guard before, audit after. The story of the fabricated defaults is in The score that lied by default.

5 · Deploy

Oversight is a schedule, not a launch

Shipping a control is the easy half. Keeping it honest is a calendar.

Every metric above has an owner, a weekly number, and a threshold that pages someone. Every agent gets a canary: the first hour after a deploy runs with tighter caps and a human reading the audit chain. Every quarter, the agent gets audited again, by someone trying to break it, with the findings sorted the same three ways.

And every control gets a kill switch a person can reach without a deploy. If the only way to stop an agent is to ship code, you do not have oversight. You have a plan to write some.

What this does not solve

This method keeps an agent honest about what it did inside a harness that I control. It says nothing about a model capable enough to route around the harness, and it says nothing about what the model's self-reports mean. On the first, I defer to the control and scalable-oversight research. On the second, I am running the smallest honest experiment I can: the same task, four harness conditions, three paraphrased self-reports, and numbers published with their uncertainty. The results will go here when they exist, whichever way they point.

The goal is not an agent that never fails. It is an agent whose failures are visible, bounded, and recorded, so a person can decide.

AI Safety · AI Security · All writing