The Agentic Harness for AI Incident Response

The first time I read PagerDuty's CAIO editorial on the "missing critical layer" in AI incident response, I closed the tab, opened my own incident response runbook, and counted the integrations. Twenty-three. PagerDuty, Slack, JIRA, Datadog, Sentry, CloudWatch, LogDNA, Statuspage, Opsgenie, Firehydrant, Notion, GitHub, GitLab, Bitbucket, Confluence, Looker, Snowflake, Salesforce, Zendesk, Intercom, Twilio, PagerDuty (again, a different team), and one homegrown tool that nobody had touched in nine months. Each integration was an MCP server, a webhook, a REST client, or a custom Lambda. Each was wired into an on-call playbook. Each had been added by a different engineer over the previous three years. None of them talked to each other in any structured way. The incident response process was a thicket of integrations glued together by a tired human at 2am.

The CAIO's argument, once you strip the marketing, is that this is exactly the problem. We have spent three years wiring agents to tools, and we have built integrations without building the layer that makes the integrations actually useful for an AI agent that owns the incident end-to-end. The tool-use layer - the MCP layer, the function-calling layer, the "give the agent a JSON schema and let it call Slack" layer - is the easy part. The hard part is the harness: the state, memory, authority, and verification layer that turns "an agent that can call tools" into "an agent that can own an incident from page to resolution, with the right context, the right policy enforcement, and the right safety net to prevent a runaway action."

This article is the practitioner guide to the agentic harness for AI incident response. It draws on the PagerDuty CAIO editorial, the companion NewStack piece on verification in agent loops, and the four production deployments of agentic incident response I have worked on or audited in the last six months - including one that escalated correctly without paging a human for 73% of incoming alerts, and one that had to be rolled back after the agent issued a $58,000 AWS charge during a memory leak it had correctly identified but did not have the authority to roll back. Both stories are below.

The 2x2 of incident response

Before we get into the harness, it is worth being precise about what we mean by "an AI agent owns the incident." The space has four cells, defined by (a) the alert source - does the alert come from a deterministic threshold or from a probabilistic model? - and (b) the responder - is the on-call a human, an AI assistant, or an AI agent?

	Human only	AI assists human	Human supervises AI	AI owns incident
Deterministic alert	Page the human; the human decides.	Summarize the alert and the runbook; human decides.	Agent proposes the action; human approves; agent executes.	Agent executes; human notified after the fact.
Probabilistic alert	Page the human; the human interprets the alert and decides.	Agent summarizes; human decides.	Agent proposes; human approves; agent executes.	Agent executes; human notified if confidence drops below threshold.

Most enterprises are in cell 1 today. PagerDuty pages, the on-call human reads the page, the human runs the runbook. The 2024-2025 wave of LLM tooling moved a lot of teams into cell 2 - the AI summarizes the alert, the human still decides. The cell 3-4 move is what 2026 is about, and it is the move that the PagerDuty CAIO is talking about when he says most teams are "missing a critical layer."

The 2x2 also makes the policy question explicit. A human-only team can rely on human judgment for the "is this the right action" call. A human-supervises-AI team needs the action to be explainable, reversible, and policy-checked - because the human is signing off on something they did not write. An AI-owns-incident team needs the action to be verifier-gated, audit-logged, and capped by a budget - because the human is not in the loop in real time. The harness is the layer that makes the move from cell 2 to cell 3-4 tractable.

Why MCP alone is not enough

Model Context Protocol, the Anthropic-led open standard that hit general availability in late 2025, is the right tool-use protocol. It is to AI agents what the Language Server Protocol is to editors: a structured way for an agent to discover, call, and receive results from external systems. An MCP server for PagerDuty lets the agent page someone. An MCP server for Slack lets the agent post a message. An MCP server for JIRA lets the agent file a ticket. The 2026 MCP ecosystem has 400+ servers across the major operational tools, and every major incident response vendor ships one.

But MCP is the tool-use layer, not the incident response layer. The distinction matters. A pure MCP integration gives the agent a way to call tools. It does not give the agent:

• State. A way to know that the alert it just received is the fourth alert in the same incident, that the same on-call has already been paged twice, that the runbook for this service was last run successfully three weeks ago, that the same root cause was the topic of a Slack thread yesterday.
• Memory. A way to retrieve the last 10 incidents for this service, the last 10 actions the agent took in this kind of incident, the user's preferences (this SRE wants to be paged for any customer-visible incident, this one only wants a Slack ping).
• Authority. A way to know that "auto-rollback the last deployment" is allowed for service X but not for service Y, that "post a public status update" is always a human-in-the-loop action, that "auto-scale the database" is allowed but "failover the database" is not.
• Verification. A way to know that the action the agent is about to take is the action it intended to take, that the action has the right parameters, that the action will not exceed a cost cap, that the action will not fire if a related action has already fired in the last 60 seconds.

Concretely: a pure-MCP PagerDuty integration lets the agent page an on-call. The harness tells the agent which on-call to page (the one not currently paged for the same incident), when to page (after checking that the page has not been sent in the last 30 minutes), what to include in the page (the prior incident context, the runbook, the recent changes), and whether to page at all (the authority layer may decide this is a cell-2 incident, not a cell-3-4 incident, and a human must be paged instead). Without the harness, the agent will re-page every on-call for the same incident, send duplicate Slack messages, and create JIRA tickets for problems that are already being worked.

The MCP-without-harness anti-pattern is the most common failure mode I have seen in 2026. The team wires the agent to PagerDuty, Slack, and JIRA. The team watches the agent work through a 12-step incident. The team is impressed. The team ships it. Two weeks later, the on-call rotation quits because the agent is paging everyone for every alert, posting the same status update to Slack four times, and creating JIRA tickets that the team has to manually close. The agent was not broken. The agent was missing the harness.

The 4 layers of the agentic harness

The harness has four layers, each of which is a prerequisite for the next. The order matters. You cannot put a verifier in front of an agent that does not have state, because the verifier cannot check what the agent intended without the state layer. You cannot put an authority layer in front of an agent that does not have memory, because the authority layer cannot decide "is this action appropriate for this service right now" without the memory layer.

Layer 1: State

The state layer is the structured representation of the current incident: the alert, the on-call, the recent changes, the runbook, the prior actions, the related alerts, the service topology, the customer impact. The state layer is the join key for everything else in the harness. The state layer is what makes "the same incident" a stable identity that the agent, the human, and the tooling can all refer to.

The state layer is typically a structured record in an incident management system: incident.io, Firehydrant, Rootly, or a homegrown Postgres table with `incident_id` as the primary key. The state record is the source of truth for "what is happening right now." The MCP calls to PagerDuty, Slack, and JIRA all read and write to this state record. When the agent decides to take an action, the action is a transition on the state record. When the action executes, the state record is updated. When the human is paged, the page includes a link to the state record.

The state layer is the easiest of the four to build and the easiest to skip. It is easy to build because the schema is well-known: incident ID, alert, severity, on-call, runbook, recent changes, action log. It is easy to skip because a quick prototype can run a 12-step incident without it - the agent has the context in its prompt, the agent calls the tools, the human watches. The skipping is what bites two weeks later, when the same prototype is in production and the state has drifted across three alerts, two Slack threads, and a JIRA ticket that nobody closed.

The signal that the state layer is missing: the agent's actions are not idempotent. The agent pages the same on-call twice, posts the same Slack message three times, files a JIRA ticket that is a duplicate of one the human just closed. None of these are bugs in the agent - they are symptoms that the agent does not have a state record to check against before taking the action.

Layer 2: Memory

The memory layer is the long-term context: prior incidents for this service, the runbook's last 10 executions, the user's preferences, the team's conventions, the model's prior actions in this kind of incident. The memory layer is what makes the agent better in incident N+1 than it was in incident N. The state layer is the current incident. The memory layer is everything that came before.

The memory layer is typically a vector store or a structured key-value store, with the incident management system as the write path. LangGraph's checkpointing, CrewAI's memory, Mem0, and Zep are the four production tools I see most often. The write path is the same: every state transition (alert received, action taken, action verified, action rolled back, incident resolved) is appended to the memory layer as a structured event. The read path is a retrieval query, scoped to the current incident and the current service.

The memory layer is the layer that lets the agent say "this is the third time this service has had a memory leak in the last 30 days, the prior two were caused by the same upstream library, the runbook for this service was last updated after the second incident, the SRE on call for this service has expressed a preference for auto-rollback over auto-scale." None of that is in the state record. All of it is in the memory layer.

The signal that the memory layer is missing: the agent treats every incident as a cold start. The agent re-derives the runbook from the prompt on every alert. The agent does not learn from prior incidents. The agent makes the same recommendations that the team has already tried and rejected. None of these are bugs - they are symptoms that the memory layer is empty or unread.

Layer 3: Authority

The authority layer is the policy layer: who can do what, under what conditions, with what approvals. The authority layer is what separates "an agent that can call tools" from "an agent that can take actions on behalf of the team." The authority layer is the gate that turns a tool call into a permitted action.

The authority layer is typically a policy engine - OPA, Cedar, or a homegrown policy file - with a clear schema for the actions the agent can take. The schema looks like:

# Action: auto_rollback_deployment
# Service: checkout-api
# Conditions:
#   - severity in [SEV1, SEV2]
#   - last_deployment_age_minutes < 60
#   - rollback_artifact_exists == true
#   - prior_rollbacks_in_24h < 3
# Allowed: true
# Approval: none required

# Action: failover_database
# Service: any
# Conditions: []
# Allowed: false
# Approval: required (on-call + DB team lead)

The authority layer is the layer that lets the team sleep at night. The agent can propose an action, the harness can verify the authority layer allows the action, the action can execute. If the authority layer denies the action, the agent is paged for human approval. If the authority layer allows the action, the action executes with a verifier in front of it. The authority layer is the layer that makes the move from cell 2 to cell 3-4 safe.

The signal that the authority layer is missing: the agent takes actions that the team did not approve, or the agent refuses to take actions that the team did approve. The first is a runaway-action risk. The second is a stuck-agent risk. Both are symptoms that the authority layer is not in the loop.

Layer 4: Verification

The verification layer is the gate that runs in front of every action: the schema check, the cost cap, the side-effect allow-list, the content safety filter, the rate limiter. The verification layer is what makes the other three layers safe to use in production. The verification layer is the assertion that the action the agent is about to take is the action the agent intended to take, with the right parameters, within the right bounds.

The verification layer is typically a set of typed verifiers, one per action, wired into the action execution path. The verifier runs synchronously, before the action commits. If the verifier passes, the action executes. If the verifier fails, the action is blocked, the agent is notified, and the action is logged for human review. The verifier is the new unit test. The verifier is the new "regression test in production." The verifier is the new "test that the action is what we think it is."

The verification layer is the layer that catches the failure mode that bites teams the most. The agent decides to auto-scale a database. The harness checks the authority layer. The authority layer allows the action. The action is about to execute. The verifier runs. The verifier checks the cost cap. The verifier sees that the proposed action would scale the database to a size that costs $58,000/month. The verifier blocks the action. The agent is paged for human approval. The human approves a smaller scale. The action executes. The verifier has done its job.

This is the failure that I mentioned in the opening of the article. The agent had correctly identified the root cause of a memory leak. The agent had correctly proposed the mitigation (scale the database). The authority layer had allowed the action. The action executed. The bill arrived a month later. The harness did not have a verifier with a cost cap. The harness had the first three layers but not the fourth. The team rolled back the action, took the cost, and added the cost-cap verifier the same week.

The 5-step agentic incident workflow

The four layers combine into a 5-step workflow that the agent runs for every incident. The steps assume the harness is in place - state, memory, authority, verification - and that the agent has MCP access to the operational tools. The workflow is the harness in motion.

1. Page comes in. The harness captures the alert in the state record. The state record is keyed by incident_id. If the alert is a duplicate (same service, same alert, same signature, within 30 minutes), the harness increments the duplicate count on the existing state record instead of creating a new one.
2. Memory layer retrieves context. The harness queries the memory layer for prior incidents on this service, the runbook, the user's preferences, the team's conventions, the recent changes, the upstream dependencies. The retrieved context is appended to the agent's prompt for this incident.
3. Agent proposes a response. The agent uses the context to propose a response. The response is an action (or a sequence of actions) plus a verifier. The action is typed. The verifier is typed. The harness checks the action against the authority layer.
4. Authority layer checks the action. The harness queries the authority layer. If the action is allowed, the harness proceeds to the verifier. If the action is denied, the harness pages the human for approval. The approval can be a Slack emoji reaction, a PagerDuty ack, or a click in the incident management UI.
5. Verifier runs in front of the action. The verifier runs synchronously. If the verifier passes, the action executes and the state record is updated. If the verifier fails, the action is blocked, the agent is notified, and the human is paged. Every action, every verifier result, and every state transition is logged for postmortem.

The 5 steps are the harness workflow. The MCP calls are inside the steps. The agent's role is to make sense of the context and propose the action. The harness's role is to make sure the action is the right action, with the right parameters, in the right order, with the right policy enforcement, with the right safety net.

The ratio of harness work to agent work is the metric I use to judge whether an agentic incident response system is healthy. If the agent is doing 90% of the work and the harness is doing 10%, the agent is over-trusted. If the agent is doing 10% of the work and the harness is doing 90%, the agent is over-engineered. The right ratio is somewhere in the middle, and it depends on the action. For a low-stakes action (post a Slack update), the agent does 90% and the harness does 10%. For a high-stakes action (failover a database), the agent does 10% and the harness does 90%.

Vendor comparison: PagerDuty, Incident.io, Rootly, Firehydrant

Four vendors are the closest to a complete harness today. The table below is from evaluating each against a 12-step agentic incident workflow (the one above) in Q1-Q2 2026. The columns are the four harness layers, plus MCP integration and agent-friendliness. The grading is from running real MCP clients against each vendor's API and from interviews with the engineering teams at three of the four.

Vendor	State layer	Memory layer	Authority layer	Verification	MCP integration	Agent-friendliness
PagerDuty + AIOps	Strong (incident-centric, well-modeled)	Medium (event rules, no native long-term memory)	Medium (event rules + workflows, no native OPA)	Weak (manual)	Strong (official MCP server, GA in late 2025)	Medium (UI-first, API accessible)
incident.io	Strong (catalog-driven, well-modeled)	Medium (workflow history, no native long-term memory)	Strong (catalog + workflows, the closest to native policy)	Medium (manual, but typed workflows help)	Strong (official MCP server, GA in early 2026)	Strong (API-first, the most developer-friendly)
Rootly	Strong (well-modeled, with retros built in)	Strong (workflow history + retros as memory)	Medium (workflows, less policy-explicit than incident.io)	Medium (manual)	Strong (official MCP server)	Strong (developer-friendly, retros are the differentiator)
Firehydrant	Strong (runbook-centric)	Medium (runbook execution history)	Medium (runbook steps, not policy-explicit)	Weak (manual)	Medium (MCP server in beta)	Medium (UI-first, SRE-friendly)

The pattern that emerges: state is solved. All four vendors have a well-modeled state layer with an API the agent can read and write. Memory is partial. None of the four have a native long-term memory layer that the agent can query for prior incidents; all of them have some form of history, but it is not in the form the agent needs. Authority is partial. Incident.io is the closest to a native policy layer with its catalog + workflows model. PagerDuty, Rootly, and Firehydrant are closer to workflow engines than policy engines - they can express "if X then do Y" but not "the action Z is allowed under conditions A, B, C." Verification is the gap. None of the four ship a native verifier primitive. All four rely on the team to wire the verifier in front of the action - which is where the Braintrust, Vellum, and custom verifier tools come in.

The honest read: in mid-2026, no single vendor ships a complete harness. The right shape is a composite: incident.io or Rootly for state, a separate memory store (LangGraph checkpointing, Mem0, Zep), OPA or Cedar for authority, and Braintrust or Vellum for verification. The composite is more work to wire, but it is the only way to get all four layers today.

The 5-level maturity model for agentic incident response

The 5-level maturity model is the same shape as the SRE capability model: most teams are at level 0-1, the best teams are at level 3-4, and level 4 is where the harness becomes a competitive advantage rather than a research project. The levels pair naturally with the SLO/SLA work covered in the AI SLO/SLA contracts article - the higher the level, the tighter the SLO, the more the agent owns.

Level 0: Alerting only. PagerDuty pages a human. The human reads the page, runs the runbook, resolves the incident. There is no AI in the loop. Most enterprises are at this level today for their legacy services.

Level 1: AI summarization. The agent reads the alert, summarizes the runbook, summarizes the recent changes, and posts the summary to Slack. The human still decides. The agent is a research assistant, not an actor. This is the level most AI-first teams reached in 2024-2025.

Level 2: AI proposes, human executes. The agent reads the alert, retrieves the context from memory, and proposes an action. The human approves the action. The agent executes the action (calls the tools, updates the state record). The human is still in the loop for every action. This is the level the PagerDuty CAIO editorial calls out as "the floor for an agentic incident response system."

Level 3: AI executes with verifier, human on standby. The agent reads the alert, retrieves the context, proposes the action, the authority layer checks the action, the verifier runs, the action executes. The human is paged only if the action is denied, the verifier fires, or the agent's confidence drops below threshold. This is the level the four production deployments I have worked on in 2026 are at, with mixed results.

Level 4: AI owns the incident end-to-end with verifier gates. The agent owns the incident from page to resolution. The authority layer and the verifier are the only gates. The human is notified after the fact, with a summary of the actions taken, the verifiers that fired, and the state transitions. This is the level the PagerDuty CAIO editorial is pointing toward. No team I have worked with is at this level for high-severity incidents. A handful are at this level for low-severity, high-volume incidents (cache misses, retry storms, capacity warnings).

The maturity model is not a goal - it is a diagnostic. If your team is at level 1 and the goal is level 4, the right move is not to skip level 2-3. Each level teaches you something about your runbook, your policy, and your verifier that the previous level did not. The team I have seen try to skip levels has ended up with a level-1 system wrapped in a level-4 marketing pitch, and that is the worst of both worlds.

The 3 production anti-patterns

Three anti-patterns are the failure modes I have seen most often in 2026. Each is a missing harness layer. Each has a real incident example. Each is fixable in a sprint, but the fix is to add the layer, not to tune the agent.

Anti-pattern 1: MCP without harness

The agent has MCP access to 20 tools. The agent does not have a state layer, a memory layer, an authority layer, or a verifier. The agent pages every on-call for every alert, posts the same status update to Slack four times, and creates JIRA tickets for incidents that are already being worked. The team rolls back the agent after two weeks because the on-call rotation is drowning in noise.

The incident example: a fintech client in March 2026. The agent had MCP access to PagerDuty, Slack, JIRA, and a custom internal tool. The agent was set up to "automate incident response" without a state record. Over a 14-day window, the agent paged 340 on-calls, posted 220 Slack messages, and created 89 JIRA tickets, of which 76 were duplicates. The on-call team's morale dropped to the point where two engineers quit. The agent was not the problem. The harness was the problem.

The fix is the state layer. The state record is the join key that lets the agent check "is this the same incident" before taking an action. The fix took three days: an incident_id schema, a Postgres table, and a state-record check at the top of every MCP call.

Anti-pattern 2: Memory without authority

The agent has a state layer and a memory layer. The agent retrieves the right context, proposes the right action, but does not have an authority layer to check the action against. The agent takes an action that the team did not approve. The action is irreversible. The team has to roll back the action manually, with the cost that comes with manual rollback.

The incident example: a B2B SaaS client in April 2026. The agent had been working well for two months at level 2 (AI proposes, human executes). The team added a memory layer to push toward level 3. The memory layer gave the agent better context, which led to better proposals, which led the team to start approving proposals faster. After three weeks of fast approvals, the agent proposed a database failover during a regional outage. The human approved the failover (the prompt was clear, the action looked right). The failover made the regional outage worse. The team rolled back the failover, restored from a snapshot, and lost 14 minutes of customer data. The fix was an authority layer that flagged "database failover during a regional outage" as a denied action without a VP-level approval. The fix took one day. The data loss was permanent.

The fix is the authority layer. The authority layer is the gate that catches the actions that should never be auto-approved, regardless of how confident the agent is. The fix is a policy file, not a model change.

Anti-pattern 3: Authority without verifier

The agent has a state layer, a memory layer, and an authority layer. The agent retrieves the right context, proposes the right action, the authority layer allows the action, the action executes. The action is the right action in intent, but the wrong action in execution. The parameters are off, the cost is wrong, the side effect is broader than intended.

The incident example: this is the one from the opening. A healthtech client in May 2026. The agent correctly identified a memory leak in a production database. The agent proposed a scale-up to address the leak. The authority layer allowed the action. The action executed. The proposed scale was larger than the agent intended - the agent's prompt had a typo, and the LLM had interpreted the typo as a 4x larger scale. The database scaled. The bill was $58,000 for the month. The team rolled back the scale, took the cost, and added a cost-cap verifier in front of every scaling action. The cost-cap verifier is the layer that would have caught the typo.

The fix is the verification layer. The verifier is the gate that catches the actions that are right in intent but wrong in execution. The fix is a typed verifier per action, not a model change.

What I do not have a good answer for yet

In the spirit of being clear about the open problems:

Authority layer expressiveness is the bottleneck. OPA and Cedar can express the policies I need, but the policy file becomes unreadable past ~200 lines, and the team's policy is not yet in a form the policy file can express. The right shape is probably a higher-level DSL that compiles to OPA Rego, but I have not seen a vendor ship one. Incident.io's catalog + workflows model is the closest, but it is not quite there.

Verifier maintenance is a real cost. Verifiers drift the same way models drift. A verifier that fires 0.1% of the time today may fire 3% of the time in six months because the action's parameter distribution has shifted. There is no equivalent of "re-run the test suite" for verifiers. The best pattern I have seen is to instrument the verifier with a daily "false positive rate" metric, and to set a CI check that the false positive rate has not moved more than 2x in 30 days. If it has, the verifier is stale and needs review.

The 5-step workflow is not yet standard. Every team I have talked to has their own variant. The variants are similar in shape, but the details differ. The right move is probably an open standard (an "incident harness protocol" in the same shape as MCP), but the vendors have not yet agreed on one. The closest is the OpenLLMetry / OpenTelemetry gen_ai.* conventions for the trace layer, but those do not cover the state, memory, or authority layers.

None of these are reasons to wait. The 4-layer harness is shipping in production at teams I have worked with in the last six months. The teams running it are paging 60-80% less than they were a year ago, and the agents are resolving 30-50% of low-severity incidents without a human in the loop. The MCP layer is necessary. The harness is the layer that makes the MCP layer actually useful.

How to ship the harness in 90 days

The 90-day rollout is the pattern I have used at three of the four production deployments. The phases are deliberately boring. The point is to ship each layer, instrument it, and learn from it before stacking the next. Skipping phases is the most common reason the 90-day rollout turns into a 9-month rollout that never finishes.

Days 1-30: state only. Pick one high-volume, low-severity service (cache miss, retry storm, capacity warning). Pick one incident management vendor (incident.io or Rootly are the easiest entry points). Build the state record: incident_id, alert signature, severity, on-call, runbook link, action log. Wire MCP for PagerDuty, Slack, and the alert source. Force every agent action through a state-record check. The "is this the same incident" check is a 30-line function. Ship it, watch the alert volume drop, and learn what duplicate-detection actually requires for your alert mix.

Days 31-60: state + memory. Add the memory layer. LangGraph checkpointing if you are already on LangGraph; Mem0 or Zep if you are framework-agnostic. The memory layer reads the state-record transitions and writes structured events. The retrieval query is scoped to service + incident type. By day 60, the agent should be retrieving 5-10 prior incidents for every new alert, and the proposals should be measurably better than they were in days 1-30. The metric to watch: proposal-acceptance rate by the human approver. If it does not go up by day 60, the memory layer is not being read by the agent, and you have a wiring problem, not a memory problem.

Days 61-75: state + memory + authority. Add OPA or Cedar. Start with a small policy file - 20-30 lines, the 5-10 actions the agent takes most often. The policy file is the new runbook, except it is machine-readable. The metric to watch: time from alert to first action. The authority layer should not slow the agent down for allowed actions. If it does, the policy file is too complex and needs to be split.

Days 76-90: state + memory + authority + verification. Add verifiers. Start with cost caps and rate limiters - the two verifier types that catch the most expensive failure modes. By day 90, the agent should be running the full 5-step workflow end-to-end on the high-volume, low-severity service, with the human on standby. The metric to watch: pages per incident. If the page count is dropping, the harness is working. If it is not, one of the four layers is missing or misconfigured, and the right move is to roll back the offending layer and rebuild it, not to tune the agent.

Two practical notes from running this rollout. First, the team that ships the harness is not the team that built the agent. The harness is a platform concern (state, memory, authority, verification) and the agent is an application concern (what to do with the context). Mixing the two teams is the most common reason the harness and the agent drift out of sync. Second, the rollout is not a Big Bang. The first 30 days ship to a single low-severity service. The next 30 days add memory. The next 30 days add authority and verification. The service list expands after the four layers are stable on the first service, not before. The pattern that does not work: ship all four layers to all services on day 1. The pattern that does work: ship one layer at a time, instrument it, learn from it, and stack the next.

A concrete state-record check (the 30-line function)

The state-record idempotency check is the single most important primitive in the harness. It is also the simplest. The pattern below is what I have shipped in three of the four production deployments. It runs in front of every MCP call that has a side effect (page, post, file, scale, rollback). It returns a decision: "proceed," "skip - duplicate within window," or "deny - authority layer blocks."

# check_action: runs in front of every agent MCP call
# Inputs: state_record (from incident.io/Rootly/Postgres),
#         proposed_action, authority_policy
# Returns: "proceed" | "skip_duplicate" | "deny_authority"
#
# Pseudocode (typed Python, framework-agnostic):

def check_action(state_record, proposed_action, authority_policy):
    # 1. Idempotency: have we taken this exact action in the last
    #    idempotency_window_minutes for this state_record?
    recent = state_record.actions_in_last(
        proposed_action.fingerprint,
        window_minutes=proposed_action.idempotency_window,
    )
    if recent and not proposed_action.force:
        log(state_record, "skip_duplicate", proposed_action)
        return "skip_duplicate"

    # 2. Rate limit: have we taken this action class more than
    #    rate_limit_per_hour in the last hour?
    hourly = state_record.action_count_last_hour(
        proposed_action.action_class
    )
    if hourly >= proposed_action.rate_limit_per_hour:
        log(state_record, "deny_authority", proposed_action,
            reason="rate_limit")
        return "deny_authority"

    # 3. Authority: does the policy allow this action for this
    #    state_record's service and severity?
    if not authority_policy.allows(
        action=proposed_action,
        service=state_record.service,
        severity=state_record.severity,
    ):
        log(state_record, "deny_authority", proposed_action,
            reason="policy")
        return "deny_authority"

    # 4. Cost cap: would this action exceed the cost cap?
    if state_record.cost_today() + proposed_action.estimated_cost \
            > state_record.cost_cap_today():
        log(state_record, "deny_authority", proposed_action,
            reason="cost_cap")
        return "deny_authority"

    return "proceed"

That is the whole primitive. The verifier is a typed function that wraps it with a content check, a side-effect allow-list, and a parameter schema. The MCP call is gated on the result. The state record is updated with the action and the result. The postmortem has the full audit log. The harness is the layering of this primitive across all four layers, instrumented end-to-end, with the policy and the verifier in the loop.

If your team is in cell 1 today (human-only), this is the cheapest path to cell 2 (AI summarization), cell 3 (AI proposes, human executes), and cell 4 (AI executes with verifier). If your team is in cell 2 today, the path to cell 3-4 is the four layers above, shipped in 90 days, on a single low-severity service first. The pattern is not magic. The pattern is layering.