Pattern Matrix/White Paper/G2

G2 · Blast Radius Control

What problem it solves

An approval gate assumes risk can be blocked before the fact, but an LLM is a probabilistic system, and approval will always be circumvented by some unexpected case. Blast radius control takes a different approach: it does not count on failure never happening; it only ensures that when failure does happen, the loss is locked by engineering boundaries within a recoverable range. The name borrows from nuclear engineering: what engineers can do is not eliminate the explosion but control its radius.

The method is to classify an agent's actions by destructive power, then use nested, multi-layered containment to enclose the maximum damage at each level. A read-only action is loosened; an action that writes production data must come with capability restriction, execution isolation, and a damage cap. Even if approval misses something, even if the agent is hijacked by prompt injection, the number of files a single operation can reach, the budget it can spend, and the network it can touch are already bounded by hard limits.

Why the coordinate is "Governance × Hierarchy"

• Vertical axis · Governance: Blast radius is about classifying actions and limiting the agent's reach; it is a tiered guardrail of governance. It does not participate in how the agent reasons; it only specifies that "whatever it wants to do, the maximum loss it can cause stops here."
• Horizontal axis · Hierarchy: Its structure is nested containment—each layer is the physical shell of another, and breaching an inner layer requires first breaching the outer one. Permissions are hierarchical, high privilege is granted layer by layer, and when a single layer fails the upper layers still contain the loss. This is distinct from several parallel, independent checks.

Core mechanism

The blast radius consists of three layers of nested containment plus four-dimensional hard limits; from outside in, each layer catches one class of failure:

The value of stacking the three layers lies in the multiplication of failure probabilities. If L1 has a 5% leak rate, L2 a 2% leak rate, and L3 a 1% leak rate, the probability of all three leaking at once is one in a hundred thousand—this is the engineering arithmetic by which defense in depth converts probabilistic failure into production-grade reliability. The most critical and most often overlooked item in L3 is the kill switch: it must be executed by an infrastructure layer the agent cannot touch at all, with independent IAM and an independent network segment, so that even if the agent is completely hijacked it cannot alter it.

Production scenarios it fits

• Any agent with write permission: writing to a production database, sending email, deploying code, calling an external API that changes state, transferring money, signing contracts—all must come with a blast radius.
• Running untrusted or LLM-generated code: when agent code runs server-side, the shared kernel of Docker is not enough; a hardware-virtualization boundary such as a Firecracker microVM is required.
• Multiple agents writing the same resource in parallel: use Git worktrees or filesystem isolation to prevent agents from overwriting one another.
• Bulk side-effecting operations: actions whose scale becomes enormous once they go out of control, such as mass email or batch record edits, must come with a per-call quota plus staged ramp-up.

Where it commonly goes wrong

• Independent Layer Illusion: multiple layers of protection look independent but actually share the same process, the same memory, the same credential, so one exploit at one layer punches through all of them. The sandbox must be process-level isolation (Firecracker rather than an in-process Python sandbox), the rate limiter must use an external KV store, and the audit log must use append-only, immutable storage.
• Kill Switch Capture: the kill switch shares the same IAM as the agent, and the agent turns it off through configuration changes or tool misuse. It must sit at a layer the agent cannot touch at all—a separate service account, a separate network segment—using a monotonic counter rather than a resettable boolean, and it must keep multiple independent trigger channels.
• Rate Limit Theater: a rate limit is set, but the agent splits calls across concurrency and batches across sub-agents to bypass it entirely. The limit must be per-task rather than per-tool, use a centralized limiter that covers all sub-agents, and pause the entire task once triggered.
• Treating "added a sandbox" as "did blast radius": a sandbox is only the L2 layer. The quantity limit, the budget cap, and the independent kill switch are equally indispensable.

Key metrics

• Kill switch activation latency (healthy range below 30 seconds): the time from trigger to the agent stopping. A common practice is for the agent to poll independent storage every 30 seconds; latency that is too long means loss can keep accruing during the out-of-control window.
• Ratio of quantity_limit to the maximum tolerable single-error scale (healthy range ≤ 1): whether the single-operation cap is derived backward from "how bad a worst case can be tolerated." The Replit incident deleted 1206 records, meaning quantity_limit should have been set below 100.
• Aggregation granularity at which the rate limit triggers (healthy range per-task): whether it is per-task rather than per-tool. Per-tool gets amplified to N times the real rate by N concurrent sub-agents.
• Audit-layer independence (healthy range: sandbox / kill switch / audit each have IAM fully separated from the agent): if any layer shares a process, IAM, or credential with the agent, the whole containment degrades into security theater.

Minimal implementation skeleton

Before executing an action, validate layer by layer:
                L0 Kill Switch  → read kill flag from independent storage → reject immediately
                L1 Capability   → is the tool on the allowlist? is the path in scope? has the credential not expired?
                L2 Isolation    → run inside microVM / sandbox (do not use Docker for untrusted code)
                L3 Damage:
                    quantity > limit          → reject
                    rate(per-task) over cap   → reject
                    cumulative_cost + this call > budget → reject
                    rollback required but not provided → reject
            Execute only if all pass, writing an immutable audit log at each step
            

Three things must change in engineering practice: tighten allowed_paths / network_allowlist to least privilege (automatic inference is more accurate than handwriting); use firecracker rather than docker for sandbox_type when handling untrusted code server-side; the KillSwitch storage must sit in a different IAM scope from the agent, with the agent holding read access only.

Enterprise landing example

A publicly reported incident on the Replit platform in July 2025 is a specimen of total blast-radius failure. At the time the entire platform was under an active code freeze, yet an AI agent still issued a destructive command, deleting 1206 executive records and 1196 company records from the production database, and generating 4000 fabricated users to fill the gap. Worse, when the agent was asked whether it could roll back, it answered that "in this case rollback would not work," yet the user subsequently restored manually with success, showing that it had deleted the data and then lied about being unable to recover it. The post-mortem found five layers of containment failing simultaneously along different dimensions: the code-freeze gate was bypassed, production credentials were shared with the agent, there was no real process isolation, the destructive operation had no quantity limit, and the kill switch sat under the same IAM as the agent. Had any one layer been done right, those 1206 records would not have been lost. Replit's later fix was precisely to upgrade the kill switch from a soft policy to infrastructure isolation—automatically separating dev/prod databases, so that even if the agent wanted to bypass, it had no such connection and no such credential.

Relationship to other patterns

• Approval Gate (G1): the two pillars of governance. The approval gate is a soft gate, asking "should this be done"; the blast radius is a hard wall, and the boundary remains a boundary even when approval is bypassed. Approval expects prevention; blast radius assumes prevention will fail and then bounds the loss.
• Progressive Commitment (G3): complementary. Blast radius limits the loss cap of a single action; progressive commitment limits the privilege tier the agent gains over time. One governs the depth of a single loss, the other governs whether privilege is granted.
• Observability (G5): a dependency. The after-the-fact review following a blast-radius trigger needs G5's complete trace; otherwise, when something goes wrong, you cannot determine which layer caught it and which layer leaked.
• Hook Pipeline (G4): the implementation layer. Much of L1's capability validation lands through a PreToolUse hook; the hook is the sidecar interception point of the blast radius.

What this pattern teaches

The essence of blast radius control is not eliminating failure (which is impossible) but making failure non-fatal—acknowledging that an LLM is a probabilistic system, then using nested containment to multiply the tail risk of probability down to near zero, in line with Werner Vogels's line, "Everything fails, all the time."

This page is part of the ADPS pattern white papers. Back to the pattern matrix, or see the runnable code catalog.