Enterprise AI

The Immune System: How to Decide What Your Agent Does Without Asking

Title card for The Immune System, showing an agent's actions sorted onto four autonomy rungs by how expensive they are to undo and to wait on

Dana Okafor gives two weeks' notice. Her agent turns off her production database access at 2 am with nobody watching, then stops and waits for a person before sending her last paycheck. The database is the more dangerous system by a wide margin, and it is the one running alone. Both settings are correct. What follows is the method that gets you there.

If you only have a minute, here's what you need to know.

  • Governance attaches to verbs, not systems. Reading payroll to compute Dana's final number is safe. Submitting the payment is not. Both are "payroll access."
  • Use four rungs, not two. Hand off, ask first, act and tell, act and log. Ship two and every action is stuck in a queue or invisible.
  • Two questions place every verb. What does undoing it cost, and what does waiting cost. Not system sensitivity, which is why Dana's database runs alone and her paycheck stops.
  • An approval showing only a number is not a control. Show the four inputs behind Dana's payment or you have added a delay and a name to blame.
  • Record the decision, not only the action. That her payment waited eleven hours is the evidence the control worked, and it is all an auditor gets.

If you are picking this up mid-series, the premise fits in a paragraph. An agentic system is eleven layers grouped into seven organ systems, and the model is one of them. The series walks one ordinary event through all eleven: Dana, a senior engineer, resigns. Part 0 drew the map. Part 1 gave the agent a name of its own, Part 2 a plan, Part 3 hands that reach real systems, Part 4 a full picture of what she owns.

So it can now shut off access, move money, and close records in six production systems on its own. This layer decides what it may do without asking, and leaves behind the proof. That is governance, and in most organizations it arrives as a document beside the system rather than a mechanism inside it. The method below builds the version that runs inside.

Step one: write down the verbs

You assign settings to single actions, like "submit a payment." Not to systems, like "payroll." Enterprise permissions work the other way, granted by system and role, because that is how identity management has worked for thirty years. An agent inherits those permissions and they say nothing useful about risk.

Dana's paycheck shows it. The agent reads payroll to work out her final number, as safe as reading anything else in the company. The same agent then submits the payment, the least reversible thing it does in fourteen days. Both are "payroll access." One of them deserves a stop.

So write down the actions instead. Call them verbs, because each is something done to something. Dana's plan holds twelve.

A real agent has twenty to two hundred. The list stays manageable because near-identical verbs share one setting, and Dana's four revocations are one entry applied four times. Teams that skip this step end up with one global setting, because they never built a list to set anything against.

Step two: pick the rungs

Every verb gets one of four settings. Dana's plan uses all four.

Hand off. The agent prepares everything and a human performs the final action. The agent never holds the credential. Dana's termination filing with the state goes here, because standing write access to a regulator's portal is a bigger decision than any single filing.

Ask first. The agent prepares, a human approves, the agent executes. Dana's final paycheck.

Act and tell. The agent executes immediately, a named person is notified, and the reversal is one documented step. Dana's six direct reports get reassigned this way, with notice to her skip-level manager.

Act and log. The agent executes, nobody is notified, the record exists. Dana's access revocations.

The four autonomy rungs: hand off, ask first, act and tell, and act and log, each with the human role and the reversal step

Most programs ship only ask first and act and log, then find every verb is either stuck in a queue or invisible. The two skipped rungs carry the value. Hand off is for when the credential itself is the risk, and it is the honest answer when write access will not get approved this quarter. Act and tell is the throughput rung, where the human is a smoke alarm and not a gate, and it only works if the notice reaches a person. Send Dana's reporting-line change to a channel and you have built a firehose nobody reads.

Step three: two questions place every verb

The intuitive method is to rank systems by how much damage a mistake could do and put the crown jewels behind an approval. Run Dana through that and the agent stops at the production database, which is exactly backwards. Ask these instead.

What does undoing it cost? Measure the path your organization actually has, in wall clock and people. Restoring Dana's database access is one administrator and twenty minutes. Recovering her payment is a phone call to someone who no longer works there. Anything leaving the company is expensive by default, and reversible-in-theory but three-weeks-and-four-teams is not reversible.

What does waiting cost? Dana's open database access gets worse every hour. Her archived documents cost nothing to sit on. Some actions decay and most do not.

Two axes, cost to undo and cost to wait, that place every verb onto its autonomy rung

Put the answers together and the rung falls out.

Dana Okafor's twelve offboarding verbs sorted onto the four autonomy rungs by their cost to undo and cost to wait

When both answers are expensive

Nothing in Dana's offboarding lands here, which makes an offboarding a gentle place to learn the method. Three things that do:

An approval queue makes these worse, because the waiting is the damage. Three ways out:

Make the undo cheaper. This one is engineering work. If the database change reverses with one command, or the frozen account thaws itself in thirty minutes unless a person confirms it, the action becomes one of the three ordinary cases. It is the only option that makes the problem smaller, so try it first.

Decide in advance. Write down, before the bad night, that this exact action under these exact conditions runs alone. Write it as a rule the system can check by itself, something like "error rate above five percent for ten minutes." It fails when the conditions need a human read.

Wake someone up. Keep the person, but make it a phone call to a named individual with a deadline attached, rather than a ticket in a queue. This is the answer when the call genuinely needs judgment, and it is the most expensive of the three to operate.

Some actions get blocked before anyone is asked

The rungs decide who has to say yes. A second kind of check decides whether the action is allowed to happen at all, and it runs first. Guardrails.

Dana's plan trips three of them.

The lawyer's hold is the sharpest, because it has to stop something the agent has every other reason to do, in the second it tries. A log that faithfully records the deletion of documents a court asked you to keep is an excellent record of an expensive mistake. Both get called governance in the same breath, and only one prevents anything.

The other half is duller and gets skipped more often. The agent is reading Dana's files to hand her work over, and plenty of them arrived by email from outside the company. Someone can bury an instruction inside a document and the agent will follow it, because an agent cannot tell an instruction from a sentence.

What an approval has to show

One question tells you whether a stop is real. Could the approver detect a wrong answer from what is on their screen?

Show a manager "final payment: $18,412.60" and an approve button and the answer is no. They handle forty of these a week. They will click it. The stop exists in the architecture diagram and nowhere else. You have not added a control. You have added a delay and a name to blame.

Show them Dana's four inputs and where each came from, and the answer is yes. Prorated salary and the dates behind it. The vacation balance and the system that holds it. The forfeited equity and the vesting schedule. The laptop deduction and the asset record. Now the approver can catch the figure that came from the wrong system.

Two things decide whether stops survive real operations. More than a handful per person per week and people stop reading, which is a problem with the rung. And a rejection has to stick, because in plenty of systems it lands in a queue nobody owns.

What you write down

Nine months from now an auditor asks one narrow question. Was Dana Okafor's production database access turned off on her last day?

"Probably" fails. "The runbook says it should have been" fails. What passes is a record of the action, the time, which agent performed it, what rung it ran under, and what changed. Dana's fourteen days produce roughly forty of these.

Nothing in it can change. Entries are added, never updated or deleted. A log the system could have rewritten proves nothing, and proof is the only reason you kept it.

It names the agent, not a borrowed account. This is where Part 1 pays off. If the agent used her manager's credentials, the log says he revoked her production access at 2 am, and he was asleep. You cannot rename a year of entries after the fact.

It records the decision, not only the action. That her payment stopped and waited eleven hours matters as much as the payment. A log capturing only what happened cannot show that a control worked.

An audit log entry showing the agent's identity, the rung it ran under, the decision, and the eleven-hour wait before Dana's payment cleared

This is not the agent's own diary from Part 4. That one feeds the agent's reasoning and can be trimmed. This one exists for people, and your regulators set how long you keep it.

Rungs move, and that is the point

A dial that only turns down is a switch with extra steps.

If Dana's reporting-line change sat on ask first for two hundred runs and got approved every time with no corrections, either the control was never needed or the approval was never real. Both should change the setting. It runs the other way too: an action unattended for a year that just produced its first bad outcome should move up a rung that afternoon, without shipping code. So rung assignments belong in configuration, and the review is a standing calendar item.

Why you cannot bolt this on later

Gartner polled more than 3,400 organizations and predicts over 40% of agentic AI projects will be canceled by the end of 2027. The three named causes are rising cost, unclear business value, and weak risk controls. Model capability is not on the list.

Governance is the third, and it behaves differently from the other two. Cost can be fixed after the fact. Risk controls mostly cannot.

An agent that ran eight months without its own identity has eight months of logs it cannot rebuild. If rung assignments were never written down, nothing shows that Dana's payment stop existed before the incident, only people saying it did.

Governance added afterwards gives you a description of a system, and an auditor is trained to notice that a description is not evidence. It is also the most common reason a pilot that worked never reaches production. Nothing was wrong with the agent, and nothing about it could be proven. I made the organizational version of this argument in The Trust Boundary Moved and the adoption version in You Installed Claude. Now the Hard Work Starts.

Three questions for your own program

Take your riskiest automated action and ask what undoes it. If the answer is a person, a phone call, and a week, that action is irreversible whatever the architecture says, and the fact that it runs unattended is a finding.

Look at your last fifty approvals and count the rejections. Zero means the approvals are unreadable, the approver has no authority, or the rung is too cautious.

Ask someone to prove a specific action happened on a specific date. Not to assert it. To produce the record, with the identity that performed it. If that takes more than a few minutes, or the answer arrives with the word "should," you have a runbook where you need an audit log.

What the immune system decides

The front door gave the agent a name. The brain stem gave it a plan. The bloodstream gave it reach. The memory gave it the full picture. This layer decides what all of that may do without asking.

The method fits on an index card. List the verbs. Score each on what undoing costs and what waiting costs. Put it on a rung. Write down what happened. Dana's twelve verbs sort into four rungs and three guardrails, and a single global setting would have gotten half of them wrong.

Get it right and you can defend the actions the agent takes alone, which is what lets you have more of them. Get it wrong in the permissive direction and you find out during an incident. Get it wrong in the cautious direction and you have built an expensive queue, the more common failure and the harder one to see, because nothing goes wrong and nothing gets faster.

Next: The Nervous System. Dana's offboarding says it finished. Proving it actually revoked everything is a different problem, and you cannot debug an agent by reading its answer.

Matthew Kruczek is Managing Director at EY, leading Microsoft domain initiatives within Digital Engineering. Connect with Matthew on LinkedIn to discuss the governance layer underneath your agent program.

References

  1. Gartner. "Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027." June 25, 2025.
  2. Anthropic. "Building Effective Agents." December 2024.
  3. NIST. "Artificial Intelligence Risk Management Framework (AI RMF 1.0)." January 2023.
  4. Matthew Kruczek. "The Trust Boundary Moved: Governing Agentic AI at the Gateway." matthewkruczek.ai, 2026.
  5. Matthew Kruczek. "You Installed Claude. Now the Hard Work Starts." matthewkruczek.ai, 2026.
  6. Matthew Kruczek. "The Memory: Your Agent Has Two, and You Only Built One." matthewkruczek.ai, 2026.

Continue Reading