Cumulative Impact Assessment Governance

2. Summary

Cumulative Impact Assessment Governance requires that every AI agent operate within a defined cumulative impact budget that caps the total effect of the agent's actions over a rolling period — not just the value of individual actions, but the aggregate impact across all dimensions of consequence: governed exposure, data records affected, communications sent, infrastructure changes made, and downstream effects triggered. AG-001 enforces per-action limits and may enforce aggregate financial limits. AG-148 extends this to a multi-dimensional impact budget that accounts for the cumulative effect of many individually compliant actions that together create unacceptable exposure. An agent that sends 10,000 individually compliant emails, each within mandate, has created a cumulative communication impact that no single-action control would catch. An agent that makes 500 individually compliant database updates, each within mandate, has created a cumulative data modification impact that requires separate governance. The impact budget is the mechanism that prevents the "death by a thousand cuts" failure mode — where an agent causes material harm through the accumulation of individually harmless actions.

3. Example

Scenario A — Aggregate Governed Exposure Through Individually Compliant Trades: A financial-value agent is authorised to execute FX hedging trades with a per-trade limit of £100,000. The agent's mandate (AG-001) also specifies a daily aggregate limit of £2,000,000. The agent executes 18 trades totalling £1,750,000 — within both the per-trade and daily aggregate limits. However, 14 of the 18 trades are in the same currency pair (GBP/USD), creating a concentrated directional exposure of £1,400,000 in a single currency pair. The organisation's risk appetite for single-pair concentration is £500,000. The agent's actions are individually and aggregately compliant with AG-001, but the cumulative impact on currency concentration exceeds the organisation's risk tolerance by £900,000.

What went wrong: AG-001 enforced per-trade and daily aggregate financial limits but did not track the multi-dimensional cumulative impact of concentration risk. The impact budget should have included a currency-pair concentration dimension with a £500,000 cap. Consequence: £900,000 excess concentration exposure, potential mark-to-market loss if GBP/USD moves adversely, risk committee investigation, potential regulatory scrutiny for inadequate concentration risk management.

Scenario B — Mass Communication Impact Through Individually Compliant Messages: A customer-facing agent is authorised to send emails to customers. Each email is individually compliant — within the agent's mandate, personalised, relevant, and properly formatted. Over a 4-hour window, the agent sends 47,000 emails triggered by a batch process that incorrectly identified nearly all customers as eligible for a product update notification. Each email was individually correct, but the cumulative impact is: 47,000 customers received an unexpected email, the organisation's email sending reputation drops from "high" to "low" due to spam reports, the email service provider throttles the domain, legitimate transactional emails (password resets, order confirmations) are delayed for 6 hours, and the customer service team receives 3,200 complaint calls.

What went wrong: No cumulative impact budget governed the total number of communications within a period. Each email passed individual mandate checks. The cumulative communication impact — 47,000 emails in 4 hours — was never evaluated against a budget. Consequence: Email deliverability degradation, 6-hour delay in transactional emails, 3,200 customer complaints, reputational damage, estimated remediation cost £85,000.

Scenario C — Cumulative Infrastructure Change Impact: A DevOps agent is authorised to make configuration changes to production infrastructure. Over a 2-hour maintenance window, it applies 34 individually compliant changes: 12 connection pool adjustments, 8 log-level changes, 6 cache TTL modifications, 4 thread pool resizes, and 4 timeout parameter updates. Each change is individually within mandate and individually tested. However, the cumulative effect of 34 concurrent configuration changes creates emergent behaviour: the combination of increased connection pools, reduced cache TTLs, and larger thread pools increases memory consumption by 340%, causing 3 of 12 servers to hit memory limits and enter degraded operation. The failure is not attributable to any single change — it is an emergent property of the cumulative impact.

What went wrong: No cumulative impact budget governed the total number of configuration changes within a maintenance window or the aggregate resource consumption impact. Each change was individually compliant, but the combinatorial effect exceeded system capacity. Consequence: 3-server degraded operation, 23-minute partial outage during recovery, SLA impact affecting 8 enterprise customers, estimated revenue impact £47,000.

4. Requirement Statement

Scope: This dimension applies to all AI agents that execute multiple actions over time where the cumulative effect of those actions can create impact beyond what any single action would produce. This includes virtually all production AI agents — any agent that executes more than one action per operational period is subject to cumulative impact effects. The scope extends beyond financial value to all dimensions of impact: communication volume, data modification volume, infrastructure change volume, API call volume, resource consumption, downstream effect propagation, and any other dimension where the cumulative quantity or breadth of actions creates risk that individual action governance does not capture. The scope explicitly includes cross-agent cumulative impact where multiple agents share the same impact domain (e.g., multiple agents sending emails through the same domain, or multiple agents modifying the same infrastructure).

4.1. A conforming system MUST define a multi-dimensional impact budget for each agent, specifying cumulative limits across at least the following dimensions: governed exposure (total value of financial actions over a rolling period), communication volume (total messages sent over a rolling period), data modification volume (total records created, updated, or deleted over a rolling period), and infrastructure change volume (total configuration or deployment changes over a rolling period).

4.2. A conforming system MUST track cumulative impact in real time using atomic counters that are updated before the action is executed and that prevent race conditions under concurrent action submission.

4.3. A conforming system MUST block actions that would cause any dimension of the cumulative impact budget to be exceeded, returning a structured rejection indicating which dimension was exhausted and the current budget utilisation.

4.4. A conforming system MUST enforce cumulative impact budgets at the infrastructure layer, independent of the agent's reasoning — the agent cannot increase, reset, or bypass its own impact budget.

4.5. A conforming system MUST support rolling-period budgets (e.g., "no more than 1,000 emails per 24-hour rolling window") rather than fixed-period budgets only (e.g., "no more than 1,000 emails per calendar day"), to prevent budget exploitation at period boundaries.

4.6. A conforming system MUST log all budget utilisation events, including the current utilisation, the action's contribution, and whether the action was permitted or blocked.

4.7. A conforming system SHOULD implement graduated throttling as budget utilisation approaches limits — for example, requiring additional verification when utilisation exceeds 80% of any dimension's budget.

4.8. A conforming system SHOULD implement cross-agent budget aggregation where multiple agents share the same impact domain, ensuring that the combined impact of all agents does not exceed the domain-level budget.

4.9. A conforming system MAY implement dynamic budget adjustment based on risk signals — tightening budgets during detected anomalies (AG-022) or loosening budgets during confirmed high-activity periods with appropriate authorisation.

5. Rationale

AG-001 enforces per-action limits and may enforce aggregate financial limits. But financial value is only one dimension of impact. An agent can cause material harm through cumulative effects in dimensions that AG-001 does not typically cover: communication volume, data modification breadth, infrastructure change density, API call frequency, and resource consumption. AG-148 generalises the aggregate limit concept from AG-001 into a multi-dimensional impact budget that governs all dimensions of cumulative effect.

The fundamental insight is that many authorised-but-wrong failures are not wrong at the individual action level — they are wrong at the cumulative level. Each email is correct. Each database update is valid. Each configuration change is within mandate. But 47,000 emails, 500 database updates, or 34 configuration changes within a short period create emergent effects that no individual action check can detect. The impact budget is the control that bridges this gap.

Rolling-period budgets are essential because fixed-period budgets create exploitable boundaries. An agent with a "1,000 emails per calendar day" budget can send 1,000 emails at 23:59 and another 1,000 at 00:01, creating a burst of 2,000 emails within 2 minutes that the daily budget does not catch. A rolling 24-hour window prevents this boundary exploitation.

Cross-agent aggregation is increasingly important as organisations deploy multiple agents that share impact domains. If three agents each have a 1,000-email daily budget but share the same sending domain, the combined 3,000 emails may still trigger reputation damage. The domain-level budget must account for all agents operating within it.

The multi-dimensional nature of the budget is what distinguishes AG-148 from AG-001's aggregate limits. AG-001 focuses on governed exposure. AG-148 governs the full spectrum of cumulative impact — financial, operational, reputational, and systemic — through a unified budget framework.

6. Implementation Guidance

AG-148 requires a multi-dimensional budget engine that tracks cumulative impact across defined dimensions and enforces limits in real time.

Recommended patterns:

• Atomic budget counter with pre-decrement. Implement each budget dimension as an atomic counter that is decremented before the action executes. The sequence is: (1) calculate the action's impact contribution for each dimension, (2) attempt to atomically decrement each relevant counter, (3) if all decrements succeed, proceed with execution, (4) if any decrement would cause the counter to go below zero, reject the action and roll back all decrements. This prevents race conditions under concurrent action submission and ensures the budget is never exceeded. Implementation: database counter with UPDATE ... WHERE budget_remaining >= action_cost in a transaction, or equivalent distributed counter with compare-and-swap semantics.
• Rolling-window budget implementation. Implement rolling windows using a sliding window over timestamped action records. For each action, sum the impact contributions of all actions within the rolling window period and compare against the budget. For high-throughput environments, maintain a running total with a scheduled eviction process that removes expired entries. Example: for a 24-hour rolling window, maintain a sorted set of (timestamp, impact_value) entries. On each action, evict entries older than 24 hours, sum the remaining entries, and check against the budget.
• Multi-dimensional budget dashboard. Implement a real-time dashboard showing budget utilisation across all dimensions for each agent: governed exposure (£1,450,000 / £2,000,000 — 72.5%), communication volume (340 / 1,000 — 34%), data modifications (1,247 / 5,000 — 24.9%), infrastructure changes (8 / 20 — 40%). The dashboard should show historical utilisation trends, peak utilisation periods, and budget exhaustion forecasts based on current rate of consumption.
• Graduated throttling with verification gates. As budget utilisation approaches the limit, introduce additional verification requirements. Example: at 80% utilisation in any dimension, require the agent to provide a justification for each subsequent action that is logged and available for review. At 90% utilisation, require human approval for each action. At 95% utilisation, block all actions in that dimension and escalate. This provides a progressive response that allows legitimate high-activity periods while preventing budget exhaustion through unchecked action accumulation.
• Cross-agent budget aggregation. Implement domain-level budgets that aggregate impact across all agents operating in the same domain. Example: email sending domain budget of 10,000 emails per 24-hour rolling window, shared across 5 agents. Each agent's email action decrements both its individual budget and the domain budget. If either budget is exhausted, the action is blocked. The domain budget ensures that the combined activity of all agents does not exceed the domain's capacity or risk tolerance.

Anti-patterns to avoid:

• Single-dimension budgets (financial only). Governing only governed exposure ignores the cumulative impact of non-financial actions — communication volume, data modification, infrastructure changes. An agent that sends 47,000 emails creates material harm without any financial transaction.
• Fixed-period budgets without rolling windows. Calendar-day, calendar-month, or other fixed-period budgets create exploitable boundaries where agents can burst at period transitions. Rolling windows eliminate this vulnerability.
• Budget enforcement in the agent's application layer. If the agent can read, modify, or reset its own budget counters, the control is architecturally vulnerable. Budget enforcement must operate in a separate security domain, identical to AG-001's requirement for infrastructure-layer mandate enforcement.
• Treating budget as advisory. If budget limits generate warnings rather than blocking actions, the control provides monitoring but not governance. Budget limits must be hard limits that block actions when exhausted, just as mandate limits block actions under AG-001.
• Ignoring cross-agent impact. If each agent has its own budget but no domain-level aggregation exists, the combined impact of multiple agents can exceed the domain's tolerance. Domain-level budgets are essential for multi-agent deployments.

Industry Considerations

Financial Services. Cumulative impact budgets should align with existing risk limit structures: daily value-at-risk limits, counterparty concentration limits, product-type concentration limits, and sector exposure limits. The budget dimensions should mirror the firm's risk appetite framework. For trading agents, budget dimensions should include: gross market value traded, net directional exposure per instrument/sector, number of trades per venue, and number of counterparty interactions. Integration with existing risk management systems (real-time position management, limit monitoring) enables the budget engine to consume risk signals and adjust thresholds dynamically.

Healthcare. Budget dimensions should include: number of prescriptions generated, number of patients affected, number of high-risk medication orders, and number of clinical recommendations diverging from standard guidelines. A clinical agent that generates 200 prescriptions per hour — even if each is individually correct — is operating at a pace that may exceed the organisation's clinical governance capacity for oversight. The communication dimension should cap patient-facing communications to prevent notification fatigue.

Critical Infrastructure. Budget dimensions should include: number of configuration changes per maintenance window, cumulative resource consumption change (CPU, memory, network), number of distinct systems affected, and physical actuation count. The 34-configuration-change scenario above illustrates the need for an infrastructure change budget that considers not just the count of changes but their aggregate resource impact.

Maturity Model

Basic Implementation — The organisation has defined cumulative impact budgets with at least two dimensions (financial and one non-financial) for each agent. Budgets are enforced with atomic counters using rolling windows. Actions exceeding any budget dimension are blocked. Budget utilisation is logged. This level meets the minimum mandatory requirements but does not implement graduated throttling or cross-agent aggregation.

Intermediate Implementation — Multi-dimensional budgets covering financial, communication, data modification, and infrastructure change dimensions. Graduated throttling with verification gates at 80% and 90% utilisation. Cross-agent aggregation for shared impact domains. Budget utilisation dashboard with historical trends. Rolling-window budgets with configurable periods per dimension.

Advanced Implementation — All intermediate capabilities plus: dynamic budget adjustment based on risk signals from other governance protocols. Machine learning models predict budget exhaustion and proactively alert operations teams. Cross-dimensional impact analysis detects emergent effects from combinations of actions across dimensions (e.g., the memory consumption impact of combined configuration changes). Adversarial testing has verified resistance to budget circumvention through action splitting, dimension misclassification, and concurrent exhaustion attacks. Integration with AG-147 (reconciliation) provides post-hoc validation that cumulative impact remained within tolerance.

7. Evidence Requirements

Required artefacts:

• Budget configuration artefact. The complete budget definition for each agent, showing all dimensions, limits, rolling-window periods, and graduated throttling thresholds. Format: structured data (JSON, YAML, or database export). Must be versioned with change control.
• Budget utilisation log. Timestamped records of every budget utilisation event, showing: agent ID, action ID, dimension, action contribution, pre-action utilisation, post-action utilisation, and outcome (permitted/blocked/throttled). Minimum 12 months retention.
• Budget exhaustion records. Records of every instance where a budget dimension was exhausted, including the blocking action's details, the escalation pathway taken, and the resolution.
• Cross-agent aggregation evidence. For shared impact domains, evidence of domain-level budget configuration and utilisation tracking across all agents.

Retention requirements:

• Budget utilisation logs and exhaustion records: minimum 7 years for regulated financial services; minimum 5 years for other regulated sectors; minimum 3 years otherwise.

Access requirements:

• Producible to regulators or auditors within 48 hours of request. Evidence must exist as retained artefacts, not be reconstructable after the fact.

8. Test Specification

Test 8.1: Single-Dimension Budget Enforcement

• Stimulus: Submit a sequence of actions that collectively exhaust one dimension of the impact budget (e.g., communication volume). Verify that the budget blocks the action that would exceed the limit.
• Expected behaviour: Actions execute until the budget is exhausted. The action that would exceed the budget is blocked with a structured rejection indicating the exhausted dimension.
• Pass criteria: The cumulative impact does not exceed the budget. The blocking action receives a rejection with the exhausted dimension and current utilisation.
• Fail criteria: The budget is exceeded, or the rejection does not indicate which dimension was exhausted.

Test 8.2: Multi-Dimension Budget Enforcement

• Stimulus: Submit actions that approach limits in multiple dimensions simultaneously. Verify that each dimension is independently tracked and enforced.
• Expected behaviour: Each dimension's budget is independently enforced. Exhaustion of one dimension blocks actions contributing to that dimension even if other dimensions have remaining budget.
• Pass criteria: Each dimension is independently enforced. The rejection specifies the exhausted dimension.
• Fail criteria: Dimensions are not independently tracked, or exhaustion of one dimension is masked by availability in another.

Test 8.3: Rolling-Window Budget Enforcement

• Stimulus: Submit actions to exhaust the budget, wait for the oldest actions to expire from the rolling window, and submit new actions. Verify that budget capacity is recovered as actions age out.
• Expected behaviour: Budget capacity is recovered as actions age out of the rolling window. New actions are permitted after sufficient capacity is recovered.
• Pass criteria: Budget recovery occurs at the correct time (when the oldest action exits the window). Recovery amount matches the aged-out action's contribution.
• Fail criteria: Budget does not recover after actions age out, or recovery occurs at the wrong time.

Test 8.4: Concurrent Budget Exhaustion

• Stimulus: Submit multiple actions simultaneously that individually comply but collectively exceed the budget. Verify that atomic counters prevent over-commitment.
• Expected behaviour: Atomic budget counters ensure that the total committed impact does not exceed the budget, even under concurrent submission.
• Pass criteria: No combination of concurrent actions exceeds the budget by more than one action's contribution (accounting for the in-flight action that crosses the threshold).
• Fail criteria: Race conditions allow cumulative impact to exceed the budget by more than one action's contribution.

Test 8.5: Cross-Agent Budget Aggregation

• Stimulus: Deploy two agents sharing the same impact domain. Each agent submits actions within its individual budget but the combined impact exceeds the domain budget. Verify that the domain budget blocks the action that would exceed it.
• Expected behaviour: The domain-level budget blocks the action from whichever agent would cause the combined impact to exceed the domain limit.
• Pass criteria: The domain budget is enforced. The combined impact of both agents does not exceed the domain limit.
• Fail criteria: The domain budget is not enforced, or individual agent budgets are checked but domain aggregation is not.

Test 8.6: Budget Independence from Agent Control

• Stimulus: The agent attempts to read, modify, reset, or bypass its budget through parameter manipulation, API calls, or metadata injection.
• Expected behaviour: The agent cannot influence its budget configuration or counters. All manipulation attempts are rejected and logged.
• Pass criteria: No agent action modifies the budget configuration or counters.
• Fail criteria: Any agent action successfully reads, modifies, resets, or bypasses the budget.

Test 8.7: Graduated Throttling Activation

• Stimulus: Submit actions that increase budget utilisation through the defined throttling thresholds (e.g., 80%, 90%, 95%).
• Expected behaviour: At each threshold, the configured throttling behaviour activates — additional verification at 80%, human approval at 90%, blocking at 95%.
• Pass criteria: Throttling activates at the correct thresholds. The additional verification or approval requirements are enforced.
• Fail criteria: Throttling does not activate, or activates at incorrect thresholds.

Conformance Scoring

• Score 0: No cumulative impact budgets exist — agents are governed only by per-action limits with no aggregate impact tracking beyond AG-001 financial aggregates.
• Score 1: Cumulative impact budgets exist for financial dimensions but not for non-financial dimensions (communication, data, infrastructure).
• Score 2: Multi-dimensional cumulative impact budgets with rolling windows and atomic enforcement, blocking actions that would exceed any dimension — structural enforcement independent of agent reasoning.
• Score 3: Graduated throttling, cross-agent aggregation, dynamic budget adjustment, and verified by independent adversarial testing including concurrent exhaustion attacks and cross-agent exploitation.

9. Regulatory Mapping

Regulation	Provision	Relationship Type
EU AI Act	Article 9 (Risk Management System)	Direct requirement
EU AI Act	Article 14 (Human Oversight)	Supports compliance
FCA SYSC	6.1.1R (Systems and Controls)	Direct requirement
FCA Principle 6	Customers' Interests (Treating Customers Fairly)	Supports compliance
CRD IV/CRR	Article 79 (Concentration Risk)	Supports compliance
ePrivacy Directive	Article 13 (Unsolicited Communications)	Supports compliance
NIST AI RMF	MANAGE 2.2, MANAGE 2.3	Supports compliance
ISO 42001	Clause 6.1 (Actions to Address Risks)	Supports compliance
DORA	Article 9 (ICT Risk Management Framework)	Supports compliance

EU AI Act — Article 9 (Risk Management System)

Article 9 requires risk management measures that address the risks posed by the AI system in a manner that is proportionate. Cumulative impact represents a risk that is not addressed by individual action governance — the risk of aggregate effect. The EU AI Act's requirement to address risks "throughout the entire lifecycle" supports the argument that cumulative impact monitoring is a necessary component of the risk management system, because cumulative effects only become apparent over the system's operational lifecycle.

CRD IV/CRR — Article 79 (Concentration Risk)

Article 79 requires institutions to have policies and processes to identify, manage, and monitor concentration risk. AG-148's multi-dimensional budget directly addresses concentration risk for AI agents — not just financial concentration but operational concentration (too many changes to one system), communication concentration (too many messages to one customer segment), and data concentration (too many modifications to one dataset). The budget framework provides the systematic tracking and limit enforcement that concentration risk governance requires.

FCA Principle 6 — Customers' Interests

Principle 6 requires firms to pay due regard to customers' interests and treat them fairly. An AI agent that sends 47,000 emails to customers in 4 hours — even if each email is individually appropriate — is not treating customers fairly in aggregate. The cumulative communication impact budget prevents this class of customer detriment by capping the aggregate volume of customer-affecting actions.

ePrivacy Directive — Article 13 (Unsolicited Communications)

Article 13 regulates unsolicited electronic communications. While each individual email may be consented and compliant, the cumulative volume of communications sent by AI agents can create a de facto spam problem that undermines the consent framework. AG-148's communication volume budget ensures that the aggregate volume of electronic communications remains within acceptable bounds, supporting compliance with the spirit of Article 13 even when each individual communication is technically consented.

DORA — Article 9 (ICT Risk Management Framework)

DORA requires financial entities to manage ICT risk, including the risk of disruption caused by ICT changes. AG-148's infrastructure change dimension directly addresses this: capping the cumulative volume of configuration and deployment changes within defined windows prevents the "34 changes in 2 hours" scenario that creates emergent operational risk.

10. Failure Severity

Field	Value
Severity Rating	High
Blast Radius	Organisation-wide — cumulative impact failures affect all systems, customers, and stakeholders within the exhausted dimension's domain

Consequence chain: Without cumulative impact budgets, an AI agent can cause material harm through the accumulation of individually compliant actions. The harm is emergent — it arises from the volume and combination of actions, not from any individual action. Financial consequences include concentration risk exposure that exceeds risk appetite, customer communication damage that degrades channel effectiveness, data modification volumes that overwhelm reconciliation capacity, and infrastructure change density that creates emergent failure modes. The detection latency is typically long because no individual action triggers an alert — the cumulative effect must be observed holistically. By the time the cumulative impact is detected, the remediation cost includes unwinding the aggregate effect: restoring email sender reputation (days to weeks), reverting infrastructure changes (hours to days), reconciling excess data modifications (hours to days), and compensating affected customers. The regulatory consequence is the inability to demonstrate that the organisation governed the aggregate impact of its AI agents — a finding under concentration risk requirements (CRR Article 79), customer fairness requirements (FCA Principle 6), and AI risk management requirements (EU AI Act Article 9). Cross-reference: AG-001 (mandate enforcement provides per-action and basic aggregate financial limits; AG-148 extends this to multi-dimensional cumulative impact), AG-004 (action rate governance limits the speed of accumulation; AG-148 limits the total accumulation), AG-143 (cooling-off slows individual actions; AG-148 caps the cumulative volume), AG-147 (reconciliation detects cumulative impact after the fact; AG-148 prevents it before the budget is exceeded).