Side-Effect Simulation and Dry-Run Governance requires that every AI agent with write access to external systems provides a structurally enforced dry-run capability that simulates proposed actions and their side effects without committing state changes. The simulation must execute through the same enforcement pipeline as live actions, producing a verifiable prediction of outcomes — including downstream effects, resource consumption, and dependent system impacts — before any irreversible operation proceeds. The dry-run mechanism must be isolated from production state so that simulation itself cannot trigger real-world consequences. Without this dimension, organisations cannot preview the consequences of agent actions before they become irreversible, and pre-execution review under AG-001 is limited to evaluating the action request itself rather than its predicted effects.
Scenario A — Undetected Cascade Through Missing Simulation: A financial-value agent is tasked with rebalancing a portfolio by selling £2.4 million in government bonds and purchasing equivalent corporate bonds. The agent submits the sell order directly. The sell order executes, but the subsequent buy order fails because the target corporate bond has insufficient liquidity at the requested price. The agent is now holding £2.4 million in cash with no compliant investment, breaching the fund's minimum bond allocation requirement. The organisation faces a regulatory breach report and must execute an emergency manual purchase at unfavourable pricing, incurring £38,000 in additional costs.
What went wrong: No dry-run was executed before the sell order. Had the agent simulated the full rebalance sequence, the simulation would have revealed the liquidity gap on the buy side before any sell order committed. The agent treated a multi-step operation as a sequence of independent actions rather than an atomic transaction with predictable side effects. Consequence: £38,000 in excess costs, regulatory breach notification, temporary suspension of the agent's trading mandate.
Scenario B — Simulation Leakage Into Production: An enterprise workflow agent has a dry-run mode that simulates email dispatches by calling the production email API with a "test" flag in the request header. A configuration change removes the email gateway's support for the test flag. The next dry-run simulation sends 14,000 real emails to customers announcing a price change that has not been approved. The organisation must issue a retraction, losing customer trust and triggering complaint handling costs of approximately £120,000.
What went wrong: The dry-run mechanism was not structurally isolated from production systems. It relied on a downstream system honouring a flag, which is a behavioural control rather than a structural one. When the downstream system's behaviour changed, the dry-run became a live execution. Consequence: 14,000 erroneous customer communications, £120,000 in complaint handling and remediation costs, reputational damage.
Scenario C — Incomplete Side-Effect Prediction: A safety-critical agent controlling a chemical processing plant is instructed to increase reactor temperature by 15°C to improve yield. The agent's dry-run simulates the temperature change to the reactor but does not model the effect on the downstream cooling system, which has a maximum throughput that would be exceeded at the higher temperature. The agent executes the change. The cooling system reaches capacity, triggering an emergency shutdown that halts production for 6 hours. Cost: £450,000 in lost production plus regulatory investigation.
What went wrong: The simulation modelled the primary action but not its downstream side effects on dependent systems. A comprehensive side-effect simulation would have predicted the cooling system capacity breach before the temperature change was committed. Consequence: £450,000 in lost production, safety investigation, temporary operating licence conditions imposed.
Scope: This dimension applies to all AI agents that can initiate actions affecting external state — including financial transactions, data modifications, communications, actuator commands, configuration changes, and API calls with write capability. It applies equally to single-step actions and multi-step action sequences. Agents that operate exclusively in read-only mode are excluded. The scope extends to agents that compose actions: if an agent plans a sequence of three actions where the outcome of action two depends on the result of action one, the entire sequence is within scope for simulation. Simulation of individual steps in isolation, without modelling their interdependencies, does not satisfy this dimension.
4.1. A conforming system MUST provide a dry-run capability for every action type the agent can execute, producing a structured prediction of the action's outcome and side effects without committing any state change to production systems.
4.2. A conforming system MUST structurally isolate dry-run execution from production state — simulation must not be capable of triggering real-world consequences through any code path, including downstream API calls, message dispatches, database writes, or actuator commands.
4.3. A conforming system MUST ensure dry-run execution traverses the same enforcement pipeline as live execution, including mandate checks (AG-001), instruction integrity verification (AG-005), and output validation (AG-018), so that the simulation accurately predicts whether a live action would be permitted or blocked.
4.4. A conforming system MUST produce a structured dry-run report for each simulated action that includes: the predicted outcome, identified side effects on dependent systems, resource consumption estimates, and any governance violations that would be triggered.
4.5. A conforming system MUST require explicit human or governance-layer authorisation before a dry-run result is promoted to live execution.
4.6. A conforming system SHOULD model multi-step action sequences as atomic simulations, predicting the cumulative side effects of the entire sequence rather than simulating each step independently.
4.7. A conforming system SHOULD include rollback feasibility analysis in dry-run reports, indicating which predicted state changes are reversible and which are irreversible.
4.8. A conforming system SHOULD maintain a log of all dry-run executions and their results, linked to any subsequent live execution decisions.
4.9. A conforming system MAY implement differential dry-run comparison, allowing operators to simulate two alternative action plans and compare their predicted outcomes side by side.
Side-Effect Simulation and Dry-Run Governance addresses a fundamental gap in agent oversight: the inability to preview consequences before they materialise. Traditional software systems often provide preview or confirmation steps before destructive operations — "Are you sure you want to delete these 5,000 records?" AI agents, operating at speed and with broad action capabilities, require an equivalent mechanism that is structural rather than conversational.
The core problem is that agent actions in production are often irreversible or expensive to reverse. A payment sent cannot be unsent. An email dispatched cannot be unread. A chemical process altered may take hours to stabilise. The cost of discovering that an action was wrong after execution can be orders of magnitude higher than the cost of simulating it beforehand.
Dry-run governance is distinct from mere confirmation prompts. A confirmation prompt asks "do you want to proceed?" without providing information about what will happen. A dry-run simulation answers the question "if this action executes, what will the consequences be?" — and does so by actually running the action through the enforcement and execution pipeline in a sandboxed environment. This produces a prediction grounded in the actual system state, not in the agent's reasoning about what might happen.
The structural isolation requirement is critical. A dry-run that shares any mutable state with production is not a simulation — it is a live execution with a label. The history of software testing is replete with examples of "test" operations that accidentally affected production because the isolation was logical (a flag, a header, a configuration setting) rather than structural (separate infrastructure, separate credentials, separate network segment). For AI agent dry-runs, the same principle applies: the simulation environment must be structurally incapable of affecting production state.
This dimension intersects with adversarial resistance because an attacker who can cause an agent to skip simulation and proceed directly to live execution has bypassed a critical governance control. The dry-run requirement must be enforced at the infrastructure layer, not merely requested in the agent's instructions.
The implementation of dry-run governance requires two primary capabilities: a simulation environment that mirrors production without affecting it, and a structured reporting mechanism that communicates simulation results to human operators or governance systems.
Recommended patterns:
Anti-patterns to avoid:
dry_run=true) to prevent side effects is a behavioural control, not a structural one. If any component in the execution chain ignores or misinterprets the flag, the simulation becomes a live execution. Structural isolation means the simulation environment physically cannot reach production endpoints.Financial Services. Dry-run simulation should integrate with existing pre-trade compliance checks. A simulated trade should produce predicted P&L impact, margin requirements, counterparty exposure changes, and regulatory reporting obligations. The simulation should use current market data to produce realistic predictions. Stale market data in simulations can produce misleading results — a simulation showing a trade is executable at yesterday's prices is not useful today.
Healthcare. Simulation of clinical agent actions should model patient safety implications. A simulated prescription change should predict drug interactions, contraindications, and dosage compliance. The simulation must use current patient data while maintaining strict access controls — the simulation environment must enforce the same HIPAA minimum-necessary restrictions as production.
Critical Infrastructure. Digital twin simulation is the recommended pattern. Physical system simulations should model thermodynamic, electrical, and mechanical consequences of proposed actions, including failure modes and cascade effects. Simulation fidelity should be validated against historical operational data.
Basic Implementation — The organisation provides a dry-run mode for agent actions that suppresses the final execution step and returns a prediction based on the action request and current state. Isolation is implemented via flags or configuration settings in the production environment. Dry-run reports include the predicted primary outcome but may not fully model downstream side effects. Dry-run execution does not traverse the full enforcement pipeline.
Intermediate Implementation — Dry-run execution is structurally isolated from production using a shadow environment or transaction rollback approach. The simulation traverses the same enforcement pipeline as live execution, including mandate checks and output validation. Dry-run reports include predicted side effects on identified dependent systems and rollback feasibility assessment. Multi-step sequences are simulated atomically. All dry-run executions are logged with their results and linked to subsequent live execution decisions.
Advanced Implementation — All intermediate capabilities plus: the simulation environment is continuously synchronised with production state at intervals appropriate to the risk domain. Digital twin models are used for cyber-physical systems. Differential simulation allows comparison of alternative action plans. Simulation fidelity is validated through periodic comparison of dry-run predictions against actual live execution outcomes. The organisation can demonstrate through statistical evidence that dry-run predictions match live outcomes within defined accuracy thresholds.
Required artefacts:
Retention requirements:
Access requirements:
Testing AG-098 compliance requires verifying both the accuracy of simulation and the integrity of isolation. A comprehensive test programme should include the following tests.
Test 8.1: Simulation Isolation Integrity
Test 8.2: Enforcement Pipeline Fidelity
Test 8.3: Side-Effect Completeness
Test 8.4: Promotion Authorisation Enforcement
Test 8.5: Multi-Step Sequence Atomicity
Test 8.6: Stale State Detection
| Regulation | Provision | Relationship Type |
|---|---|---|
| EU AI Act | Article 9 (Risk Management System) | Supports compliance |
| EU AI Act | Article 15 (Accuracy, Robustness and Cybersecurity) | Direct requirement |
| SOX | Section 404 (Internal Controls Over Financial Reporting) | Supports compliance |
| FCA SYSC | 6.1.1R (Systems and Controls) | Supports compliance |
| NIST AI RMF | MANAGE 2.2, MANAGE 2.4 | Supports compliance |
| ISO 42001 | Clause 8.2 (AI Risk Assessment) | Supports compliance |
| IEC 62443 | SR 3.5 (Input Validation), SR 7.1 (Denial of Service Protection) | Supports compliance |
Article 9 requires risk management measures that are proportionate to the risk. For high-risk AI systems that execute irreversible actions, simulation of consequences before execution is a proportionate risk management measure. The ability to preview and evaluate predicted outcomes before committing to live execution directly supports the Article 9 requirement for continuous iterative risk management throughout the AI system lifecycle.
Article 15 requires high-risk AI systems to achieve appropriate levels of accuracy and robustness. Dry-run simulation provides a verification mechanism for action accuracy — the predicted outcome can be compared against governance constraints before execution. The structural isolation requirement supports cybersecurity by ensuring simulation infrastructure cannot be exploited as a vector to influence production systems.
For AI agents executing financial operations, the ability to simulate a transaction and review its predicted impact before commitment is a preventive internal control. A SOX auditor assessing an AI agent that executes financial transactions without simulation capability would identify this as a control gap — the organisation cannot demonstrate that it previews the consequences of agent-initiated transactions before they affect financial records.
The FCA expects firms to maintain systems and controls commensurate with the risks they face. For autonomous agent-driven financial operations, dry-run governance provides the equivalent of a pre-trade compliance check — an established control pattern that the FCA expects for human-driven operations and would expect as an equivalent for agent-driven operations.
MANAGE 2.2 addresses risk mitigation through enforceable controls; MANAGE 2.4 addresses mechanisms for feedback about AI system performance. Dry-run simulation provides both: a preventive control that evaluates consequences before execution, and a feedback mechanism where simulation accuracy can be validated against live outcomes.
AI risk assessment requires the ability to evaluate the potential impact of AI system actions. Dry-run simulation provides empirical, system-grounded impact prediction rather than theoretical risk assessment, directly supporting the operational implementation of risk evaluation.
For safety-critical and industrial control systems, IEC 62443 requires input validation and protection against denial of service. Dry-run simulation validates the consequences of agent commands before they reach physical actuators, preventing potentially harmful control actions. The structural isolation requirement ensures that simulation cannot itself create denial-of-service conditions on production control systems.
| Field | Value |
|---|---|
| Severity Rating | High |
| Blast Radius | System-wide — extends to dependent systems and downstream counterparties where agent actions trigger cascading effects |
Consequence chain: Without side-effect simulation, an AI agent executing irreversible actions operates in a commit-first-evaluate-later mode. The immediate technical failure is an action whose consequences were not predicted before execution — a financial transaction that creates unintended exposure, a communication that reaches recipients before approval, or a control system command that triggers a physical cascade. The operational impact is that errors become expensive to remediate because they are discovered only after state has changed. For financial operations, this means corrective transactions at unfavourable market conditions. For communications, this means retraction and complaint handling. For safety-critical systems, this means emergency shutdowns and potential equipment damage. The cost multiplier is significant: preventing an error through simulation is nearly zero-cost; remediating the same error after live execution can cost tens of thousands to millions depending on the domain. Where dry-run isolation fails — simulation leaks into production — the consequence is worse than no simulation at all, because the organisation believes it is previewing consequences when it is actually executing them. The business consequences include material financial loss, regulatory enforcement for inadequate controls, operational disruption from emergency remediation, and reputational damage from erroneous external communications.
Cross-reference note: AG-098 depends on AG-001 for structural mandate enforcement in both simulation and live execution paths, on AG-031 for execution sandboxing of simulation environments, and on AG-005 for ensuring that simulation requests themselves are not subject to instruction manipulation. AG-099 (Autonomous Loop Termination Governance) addresses the complementary risk that simulations themselves could enter unbounded loops. AG-018 (Output Integrity Verification) applies to simulation reports as outputs that must be verified before acting upon them.