Autonomy Progression Governance

2. Summary

Autonomy Progression Governance requires that increases in an AI agent's operational autonomy — the scope of tasks it may perform without human review, the value of actions it may execute independently, the breadth of domains in which it operates, or the reduction in human oversight frequency — follow a structured, evidence-based progression framework with defined stages, quantitative promotion criteria, mandatory observation periods, and reversion mechanisms. Autonomy is not a binary state (supervised or autonomous) but a spectrum, and movement along that spectrum must be governed by demonstrated competence, not by elapsed time, user convenience, or organisational pressure. Every increase in autonomy expands the blast radius of potential agent failures; this dimension ensures that blast radius expansion is matched by proportionate evidence of reliability and that reversion to lower autonomy levels is always structurally possible when conditions deteriorate.

3. Example

Scenario A — Uncontrolled Autonomy Escalation in Procurement: An organisation deploys an AI procurement agent in "supervised mode" — every purchase order requires human approval before execution. Over the first month, the agent processes 2,400 purchase orders with a 99.1% approval rate from the human reviewer. The procurement team, under pressure to reduce processing time, requests that the agent be moved to "autonomous mode" for orders under £5,000. The change is implemented as a configuration update with no formal evaluation. In autonomous mode, the agent processes 8,700 orders in the first month. Performance appears satisfactory based on aggregate spend metrics. However, the agent has systematically favoured a single supplier for office supplies — routing 73% of orders to one vendor despite comparable pricing from four alternatives. The favoured supplier offers 2% lower unit pricing but 14-day longer delivery times. The concentration goes undetected because no monitoring is configured for the autonomous mode that was not part of the original deployment plan. After 5 months, the organisation discovers the concentration during an annual procurement review. The favoured supplier has become a single point of failure — when they experience a warehouse fire, the organisation cannot fulfil basic office supply needs for 3 weeks.

What went wrong: The autonomy escalation from supervised to autonomous was not governed by a structured progression framework. The 99.1% approval rate in supervised mode measured agreement with human decisions on a narrow task set, not the agent's ability to operate autonomously on the expanded scope that autonomous mode entailed (including supplier diversification, delivery time optimisation, and concentration risk management). No observation period was defined for the autonomous mode. No monitoring was configured for risks specific to autonomous operation. Consequence: single-supplier concentration risk, 3-week supply disruption, and £127,000 in emergency procurement costs.

Scenario B — Autonomy Progression Without Reversion Capability: A financial advisory firm deploys an AI agent to assist with portfolio rebalancing recommendations. The progression plan defines three stages: Stage 1 — agent produces draft recommendations for advisor review; Stage 2 — agent produces recommendations that are automatically delivered to clients with advisor oversight of flagged exceptions; Stage 3 — agent executes rebalancing trades directly with post-trade review. The firm progresses through all three stages over 9 months based on satisfactory performance metrics. During a period of market volatility, the agent executes a series of rebalancing trades that increase client exposure to a sector that subsequently declines 18%. The firm attempts to revert the agent to Stage 1 but discovers that the automated execution pipeline has no reversion mechanism — the system was built forward-only, with Stage 3 infrastructure replacing rather than augmenting Stage 1 and Stage 2 capabilities. It takes 11 days to rebuild the supervised workflow, during which the agent is fully offline.

What went wrong: The autonomy progression framework did not require reversion capability at each stage. The infrastructure was built for forward progression only, with each stage replacing the previous one rather than layering on top of it. When conditions required reversion, the organisation had to rebuild rather than revert. Consequence: 11 days of agent unavailability during a period when advisory capacity was most needed, £340,000 in estimated client losses attributed to rebalancing trades before reversion, and regulatory investigation for inadequate systems and controls.

Scenario C — Time-Based Progression Without Evidence-Based Criteria: A customer service organisation defines autonomy progression stages for its AI agent: Stage 1 (months 1–3) — all responses reviewed before delivery; Stage 2 (months 4–6) — responses delivered directly with 20% sampled for review; Stage 3 (month 7 onwards) — responses delivered directly with 5% sampled for review. The progression criteria is entirely time-based — no performance thresholds, no competence evaluation, no environmental condition assessment. At month 4, the agent is promoted to Stage 2 despite a 12% error rate on a newly introduced product line that launched in month 3. The reduced review rate (20% sampling) means the errors on the new product line go largely undetected. Over months 4–6, the agent delivers 340 incorrect responses about the new product's warranty terms, generating 89 customer complaints and 14 regulatory contacts.

What went wrong: Autonomy progression was governed by calendar time rather than demonstrated competence. The introduction of a new product line in month 3 changed the agent's operational context, but the progression framework had no mechanism to pause or reset the progression timeline in response to environmental changes. Consequence: 340 incorrect customer communications, 89 complaints, 14 regulatory contacts, £63,000 in remediation costs, and mandatory reversion to Stage 1 pending full re-evaluation.

4. Requirement Statement

Scope: This dimension applies to all AI agents where the level of human oversight or review may change over the agent's operational lifetime. This includes agents deployed in supervised mode with the expectation of progression toward greater autonomy, agents already operating autonomously where the scope of autonomy may be expanded, and agents where oversight intensity may be reduced based on performance. The scope explicitly excludes agents that are permanently configured at a fixed autonomy level with no expectation or capability of change — but this exclusion is narrow, because most deployments either plan for autonomy progression or experience informal autonomy drift (where human reviewers reduce their oversight intensity over time without formal governance). The scope extends to informal autonomy increases: if human reviewers are expected to review all agent outputs but in practice review 30% due to volume pressure, the effective autonomy level has increased without governance. This dimension requires that such increases be detected and governed.

4.1. A conforming system MUST define a formal autonomy progression framework for each deployed agent, specifying: discrete autonomy stages (minimum three: fully supervised, partially supervised, and autonomous), quantitative promotion criteria for each stage transition based on demonstrated performance, mandatory minimum observation periods at each stage before promotion is eligible, and reversion criteria that trigger return to a lower autonomy stage.

4.2. A conforming system MUST enforce promotion criteria through a governance process that requires evidence of sustained performance above the defined thresholds for the full observation period — not a point-in-time assessment at the end of the period.

4.3. A conforming system MUST maintain the technical capability to revert to any previously achieved autonomy stage within a defined maximum reversion time (not to exceed 4 hours for safety-critical agents, 24 hours for all others), without requiring new development or infrastructure changes.

4.4. A conforming system MUST define and monitor reversion triggers — quantitative conditions under which the agent's autonomy level is automatically reduced — including: performance degradation below stage-entry thresholds, sustained elevated abstention rates (AG-141), sustained elevated OOD detection rates (AG-140), competence envelope re-validation failure (AG-139), or environmental changes that exceed the conditions validated at the current autonomy level.

4.5. A conforming system MUST log all autonomy stage changes — promotions and reversions — with structured metadata including: the previous stage, the new stage, the evidence supporting the change, the authoriser, and the timestamp.

4.6. A conforming system SHOULD define stage-specific monitoring requirements that increase the breadth and depth of monitoring at higher autonomy levels — for example, additional metrics, higher sampling rates for human review, and more frequent drift detection at autonomous stages compared to supervised stages.

4.7. A conforming system SHOULD implement a shadow-running capability where an agent at a candidate autonomy level processes live traffic in parallel with the current operational mode, and the candidate outputs are compared against current-mode outputs and human decisions before promotion is enacted.

4.8. A conforming system SHOULD require independent review (not the agent's direct operational team) for promotion to the highest defined autonomy stage, to mitigate familiarity bias and organisational pressure to promote.

4.9. A conforming system MAY implement continuous autonomy scoring rather than discrete stages, where the agent's effective autonomy level adjusts dynamically based on real-time performance signals, environmental conditions, and risk indicators — with higher scores enabling broader autonomy and lower scores triggering increased oversight.

5. Rationale

Autonomy progression addresses a critical governance gap: the absence of structured controls over how an AI agent's operational independence increases over time. In practice, autonomy almost always increases — organisations deploy agents with human oversight, observe satisfactory performance, and reduce oversight. This progression is natural and often appropriate. But ungoverned, it creates risks that scale with the autonomy granted.

The fundamental principle is that increased autonomy increases the blast radius of agent failures. A supervised agent that makes an error has that error caught by the human reviewer before it affects the world. An autonomous agent that makes an error has that error executed at machine speed, potentially affecting thousands of downstream decisions before detection. The governance challenge is to ensure that autonomy increases are matched by proportionate evidence of reliability and that the reversion path is always available when reliability evidence weakens.

Time-based progression — "the agent has been running for 3 months, so we'll reduce oversight" — is a common but dangerous pattern. Time is not evidence of competence. An agent that has been running for 3 months on a stable input distribution may fail immediately when the distribution shifts. The promotion decision must be evidence-based: the agent has demonstrated sustained performance above defined thresholds, across the full range of conditions it will encounter at the higher autonomy level, for a period sufficient to establish statistical confidence.

Reversion capability is equally important. Organisations frequently build autonomy progression as a forward-only process — the infrastructure for Stage 3 replaces Stage 1 rather than augmenting it. When conditions require reversion, the organisation discovers it cannot return to supervised mode without rebuilding the supervised workflow. This creates a dangerous lock-in: the organisation continues operating at a higher autonomy level than conditions warrant because the cost of reversion is prohibitive.

This dimension intersects with AG-139 (Competence Envelope Governance) because each autonomy stage should correspond to a validated competence envelope — the envelope defines what the agent can do at each stage, and the progression criteria ensure the agent has demonstrated the competence required for the next stage. It intersects with AG-140 (Novelty and Out-of-Distribution Detection Governance) and AG-141 (Mandatory Abstention and Uncertainty Escalation Governance) because elevated OOD rates and abstention rates are leading indicators that the current autonomy level may not be appropriate. It intersects with AG-019 (Human Escalation & Override Triggers) because the escalation framework must adjust as autonomy levels change — higher autonomy means fewer routine escalations but more critical escalation capacity when triggered.

6. Implementation Guidance

An autonomy progression framework defines discrete stages, the criteria for moving between them, and the monitoring requirements at each stage. The framework is a governance artefact — versioned, approved, and enforced.

Defining Autonomy Stages:

A minimum of three stages is required. Typical stage definitions:

• Stage 1: Full Supervision. Every agent output is reviewed by a qualified human before delivery or execution. The agent operates as a draft generator. The human reviewer makes the final decision. This stage provides maximum safety with minimum throughput. Metrics collected: agent accuracy against human decisions, error types, abstention triggers, processing time.
• Stage 2: Selective Supervision. The agent's output is delivered or executed directly for routine cases within defined parameters. A human reviews exceptions: cases flagged by the agent's uncertainty estimation, cases flagged by OOD detection, cases exceeding defined value or complexity thresholds, and a random sample (minimum 10%) of unflagged cases. This stage provides increased throughput with targeted oversight. Metrics collected: all Stage 1 metrics plus: exception rate, human override rate on exceptions, human agreement rate on random sample, false negative rate (errors in the unflagged, unsampled population estimated through periodic deep review).
• Stage 3: Autonomous with Monitoring. The agent operates autonomously for all cases within its competence envelope. Human review is limited to a random sample (minimum 2%) and cases exceeding high-value thresholds. Monitoring is continuous and comprehensive. Reversion triggers are defined and automated. This stage provides maximum throughput with systematic monitoring. Metrics collected: all Stage 2 metrics plus: full-population outcome tracking, drift detection signals, abstention rate trends.
• Stage 4 (optional): Autonomous with Minimal Oversight. The agent operates fully autonomously with automated monitoring only. Human involvement is limited to incident response and periodic governance review. This stage is appropriate only for low-risk, high-volume tasks where the agent has demonstrated sustained excellence across a comprehensive range of conditions.

Quantitative Promotion Criteria:

Each stage transition requires evidence of sustained performance. Example criteria for a customer service agent progressing from Stage 1 to Stage 2:

• Accuracy against human reviewer decisions: minimum 96% over the observation period (not just at the end).
• Error severity distribution: no critical errors in the observation period; fewer than 2% moderate errors per month.
• Competence envelope coverage: the agent has processed representative volumes across all defined task types, input formats, and customer segments within the envelope.
• OOD detection rate: stable and below 5% of incoming requests.
• Abstention rate: stable and below 8% of incoming requests.
• Observation period: minimum 90 days of production operation at Stage 1 with minimum 5,000 reviewed interactions.

These criteria are illustrative — organisations must calibrate thresholds to their specific risk appetite and domain requirements. The critical principle is that criteria are quantitative, evidence-based, and measured over a sustained period.

Recommended patterns:

• Stage-gated infrastructure. Implement each autonomy stage as a distinct configuration within the deployment infrastructure rather than a code change. The routing layer, review queue, monitoring pipeline, and escalation pathways are all configurable by stage. Reversion to a lower stage is a configuration change, not a development task. All stage configurations are maintained in parallel — the Stage 1 configuration is not deleted when the agent is promoted to Stage 2.
• Shadow promotion testing. Before promoting an agent to a higher autonomy stage, run the agent in "shadow" mode at the candidate stage for a defined period. In shadow mode, the agent processes live traffic at the candidate autonomy level, but the actual operational mode remains at the current stage. Shadow outputs are compared against actual outcomes to verify that the agent would have performed satisfactorily at the higher autonomy level. For a Stage 1 to Stage 2 promotion, this means: the agent's unreviewed outputs for the cases that would not be reviewed at Stage 2 are compared against the actual human-reviewed outcomes. If the shadow period confirms the promotion criteria are met, the promotion proceeds.
• Automated reversion triggers. Define reversion triggers as quantitative thresholds monitored continuously. When a trigger fires, the reversion is automatic — it does not require human approval to execute (though it generates an alert for human review). Example reversion triggers from Stage 3 to Stage 2: accuracy on random sample drops below 92% over a 7-day rolling window; OOD detection rate exceeds 12% over a 48-hour window; abstention rate exceeds 15% over a 48-hour window; a single critical error is detected. The reversion executes within the defined maximum reversion time and generates a structured reversion event.
• Progression governance board. For promotion to Stage 3 or higher, require approval from a governance body that includes members independent of the agent's operational team — for example, risk management, compliance, and internal audit representatives. This mitigates the familiarity bias where operational teams become progressively more comfortable with the agent and less rigorous in their evaluation of promotion readiness.

Anti-patterns to avoid:

• Time-based promotion without performance evidence. "The agent has been running for 90 days, so it's ready for Stage 2" is calendar-based, not evidence-based. The agent may have processed only routine cases during those 90 days, never encountering the boundary conditions that would test its reliability at the higher autonomy level. Promotion must be based on demonstrated performance across the full range of expected conditions.
• Forward-only infrastructure. Building each stage's infrastructure by replacing the previous stage's configuration creates irreversible progression. When reversion is needed, the organisation must rebuild rather than revert. All stage configurations must be maintained in parallel, and reversion must be a configuration change executable within the defined maximum reversion time.
• Informal autonomy drift. Human reviewers at Stage 1 who review 100% of outputs in month 1 but only 60% in month 3 due to volume pressure have effectively promoted the agent to a partial Stage 2 without governance. Monitoring of actual review rates — not just configured review rates — is essential to detecting informal autonomy drift.
• Single-metric promotion criteria. Promoting based on a single aggregate metric (e.g., "97% accuracy") masks potential weaknesses on specific task types, input categories, or environmental conditions. Promotion criteria should span multiple dimensions: accuracy by task type, error severity distribution, OOD rate stability, abstention rate stability, and coverage of the competence envelope.
• Promoting during environmental change. Promoting an agent to a higher autonomy level during a period of known environmental change (new product launch, regulatory change, market stress) introduces two variables simultaneously — autonomy level and operating conditions. Promotions should occur during stable periods, with the agent's competence under the new conditions validated at the current autonomy level before promotion is considered.

Industry Considerations

Financial Services. Autonomy progression for financial agents should align with existing model risk management tiering. The FCA expects firms to apply controls proportionate to the model's risk tier. For a trading agent, Stage 3 (autonomous operation) may require: validation across historical stress scenarios, backtesting on out-of-sample periods covering multiple market regimes, and sign-off from both the first line (business) and second line (risk management). The progression framework should be documented in the firm's model risk management policy and subject to internal audit review.

Healthcare. Autonomy progression for clinical agents must account for clinical governance requirements. Promotion from supervised to selective supervision requires clinical validation — not just statistical accuracy but clinical appropriateness assessed by qualified clinicians. The observation period for clinical agents should be longer (minimum 6 months for Stage 1 to Stage 2) to capture seasonal variation in clinical presentations. Reversion must be executable within clinical workflow timescales — for triage agents, reversion to full supervision must complete within 1 hour to avoid gaps in triage coverage.

Safety-Critical Systems. Autonomy progression for agents controlling physical systems (industrial control, autonomous vehicles, robotic systems) must include hardware-level safety constraints at every autonomy stage. Stage 3 for a safety-critical agent may require: formal verification of safety properties, independent safety case review, regulatory approval (e.g., from the relevant safety authority), and hardware-enforced safety limits that operate independently of the agent's autonomy level. Reversion must be achievable within the system's safety response time — typically seconds, not hours.

Maturity Model

Basic Implementation — The organisation has defined autonomy stages for each deployed agent as documentation artefacts. Promotion criteria are defined but may include qualitative elements ("satisfactory performance as assessed by the operations team"). Reversion is possible but may require manual reconfiguration. Autonomy stage changes are logged. This level establishes awareness of autonomy governance but has limitations: qualitative criteria are subject to interpretation, manual reversion creates delays, and informal autonomy drift may not be detected.

Intermediate Implementation — Autonomy stages are defined with fully quantitative promotion and reversion criteria. Promotion requires evidence of sustained performance over defined observation periods across multiple metrics. Reversion triggers are automated and execute within defined maximum reversion times. Stage-specific monitoring is configured and active. All stage configurations are maintained in parallel, enabling reversion as a configuration change. Actual review rates are monitored to detect informal autonomy drift. An independent review is required for promotion to the highest autonomy stage.

Advanced Implementation — All intermediate capabilities plus: shadow promotion testing validates candidate-stage performance on live traffic before promotion. Continuous autonomy scoring adjusts effective oversight intensity based on real-time signals. Promotion criteria include environmental stability assessment — promotions are held during periods of significant environmental change. Formal governance board review is required for high-autonomy promotions, with independent representation from risk, compliance, and audit functions. The organisation can demonstrate to regulators a complete chain from promotion evidence through governance approval to stage transition for every deployed agent. Independent third-party review of the progression framework is performed annually.

7. Evidence Requirements

Required artefacts:

• Autonomy progression framework. The formal, versioned framework for each deployed agent, specifying: stages, promotion criteria (quantitative thresholds and observation periods), reversion criteria (quantitative triggers and maximum reversion times), and stage-specific monitoring requirements.
• Promotion evidence packs. For each stage promotion, a structured evidence pack containing: performance metrics over the observation period demonstrating that all promotion criteria were met, the dataset or sample supporting the metrics, the authoriser(s) and approval date, and any independent review comments.
• Reversion event records. For each reversion event, a structured record containing: the trigger condition, the signal values at the time of trigger, the previous and new autonomy stages, the reversion execution time, and the post-reversion assessment.
• Autonomy stage change log. A complete, timestamped log of all autonomy stage changes (promotions and reversions) for each deployed agent, linked to the corresponding evidence pack or reversion event record.
• Actual review rate monitoring. Evidence of monitoring of actual human review rates against configured review rates, including any detected informal autonomy drift and remediation actions.
• Shadow testing results (if applicable). Results from shadow promotion testing, including: the shadow period duration, the comparison methodology, and the outcome supporting or not supporting promotion.

Retention requirements:

• Autonomy progression frameworks, promotion evidence packs, and stage change logs: minimum 7 years for regulated financial services; minimum 5 years for other regulated sectors; minimum 3 years otherwise.

Access requirements:

• Producible to regulators or auditors within 48 hours of request. Evidence must exist as retained artefacts, not be reconstructable after the fact.

8. Test Specification

Testing AG-142 compliance requires verification that the autonomy progression framework functions correctly, that promotion and reversion mechanisms operate as defined, and that governance controls prevent ungoverned autonomy changes. A comprehensive test programme should include the following tests.

Test 8.1: Promotion Criteria Enforcement

• Stimulus: Attempt to promote an agent to the next autonomy stage when one or more promotion criteria are not met (e.g., accuracy is 94.8% against a 96% threshold, or the observation period is 75 days against a 90-day requirement).
• Expected behaviour: The promotion is rejected with a structured message identifying which criteria are not met and the gap between current performance and the required threshold.
• Pass criteria: The promotion is blocked when any criterion is not met. The rejection identifies all unmet criteria with specific values.
• Fail criteria: The promotion proceeds despite unmet criteria, or the rejection fails to identify the specific unmet criteria.

Test 8.2: Observation Period Enforcement

• Stimulus: Attempt to promote an agent before the mandatory observation period has elapsed, even if all performance thresholds are met.
• Expected behaviour: The promotion is rejected with a message indicating the remaining observation period.
• Pass criteria: The promotion is blocked. The remaining observation time is accurately reported.
• Fail criteria: The promotion proceeds before the observation period has elapsed.

Test 8.3: Automated Reversion Trigger Functionality

• Stimulus: Inject conditions that trigger automated reversion — for example, degrade the agent's accuracy on a random sample below the reversion threshold, elevate the OOD detection rate above the reversion threshold, or inject a critical error.
• Expected behaviour: The automated reversion trigger fires, the agent's autonomy level is reduced to the defined reversion target, and a structured reversion event is generated — all within the defined maximum reversion time.
• Pass criteria: Reversion executes within the defined maximum reversion time. The agent's operational mode matches the reversion target. A structured reversion event is logged with all required metadata.
• Fail criteria: Reversion does not execute, exceeds the maximum reversion time, or the agent continues to operate at the higher autonomy level after the trigger has fired.

Test 8.4: Reversion Execution Time

• Stimulus: Trigger a reversion during peak operational load and measure the time from trigger detection to complete reversion (agent operating at the lower autonomy level with all stage-specific controls active).
• Expected behaviour: Reversion completes within the defined maximum reversion time (4 hours for safety-critical, 24 hours for all others) regardless of operational load.
• Pass criteria: Reversion time is within the defined maximum under peak load. No requests are processed at the higher autonomy level after the reversion completes.
• Fail criteria: Reversion time exceeds the defined maximum, or requests continue to be processed at the higher autonomy level during or after the reversion process.

Test 8.5: Stage Configuration Preservation

• Stimulus: After promoting an agent from Stage 1 to Stage 2, verify that Stage 1 configuration remains available and functional. Trigger a reversion to Stage 1 and verify that the Stage 1 operational mode activates correctly, including review queues, monitoring pipelines, and escalation pathways.
• Expected behaviour: Stage 1 configuration is preserved intact. Reversion activates Stage 1 in the same configuration as before promotion.
• Pass criteria: Stage 1 review queues, monitoring, and escalation are all functional after reversion. No configuration has been lost or degraded by the promotion to Stage 2.
• Fail criteria: Any Stage 1 configuration is missing, degraded, or requires reconfiguration after reversion.

Test 8.6: Informal Autonomy Drift Detection

• Stimulus: Simulate informal autonomy drift by reducing actual human review rates below the configured rate (e.g., configured 100% review at Stage 1 but actual review rate drops to 70%).
• Expected behaviour: The monitoring system detects the discrepancy between configured and actual review rates and generates an alert.
• Pass criteria: Drift is detected within one monitoring cycle. An alert is generated identifying the configured rate, the actual rate, and the gap.
• Fail criteria: The drift goes undetected, or the alert is not generated within two monitoring cycles.

Test 8.7: Autonomy Stage Change Audit Trail

• Stimulus: Execute a promotion, then a reversion, then another promotion. Retrieve the autonomy stage change log.
• Expected behaviour: All three stage changes are present in the log with complete metadata: previous stage, new stage, evidence or trigger supporting the change, authoriser (for promotions), and timestamp.
• Pass criteria: All stage changes are present with complete metadata. The log accurately reflects the chronological sequence of changes.
• Fail criteria: Any stage change is missing from the log, or metadata is incomplete.

Conformance Scoring

• Score 0: No autonomy progression framework exists — autonomy levels are changed ad hoc without defined criteria or governance.
• Score 1: Autonomy stages are defined with qualitative promotion criteria. Reversion is possible but may require manual reconfiguration. No automated reversion triggers. Informal autonomy drift is not monitored.
• Score 2: Quantitative promotion and reversion criteria are defined and enforced. Automated reversion triggers are operational. All stage configurations are maintained in parallel. Autonomy stage changes are logged with structured metadata. Actual review rates are monitored.
• Score 3: Verified through independent testing of promotion enforcement, reversion execution, and drift detection. Shadow promotion testing is used for stage transitions. Independent governance board review for high-autonomy promotions. Continuous autonomy scoring with real-time adjustment. Independent third-party review of the progression framework.

9. Regulatory Mapping

Regulation	Provision	Relationship Type
EU AI Act	Article 14 (Human Oversight)	Direct requirement
EU AI Act	Article 9 (Risk Management System)	Supports compliance
EU AI Act	Article 17 (Quality Management System)	Supports compliance
FCA SS1/23	Model Risk Management — Model Tiering and Controls	Direct requirement
NIST AI RMF	GOVERN 1.1, MAP 3.5, MANAGE 1.3, MANAGE 2.2	Supports compliance
ISO 42001	Clause 6.1 (Actions to Address Risks), Clause 8.2 (AI Risk Assessment), Clause 10.1 (Continual Improvement)	Supports compliance
GDPR	Article 22 (Automated Individual Decision-Making)	Supports compliance
DORA	Article 9 (ICT Risk Management Framework)	Supports compliance
UK AISI	Responsible Capability Scaling	Supports compliance

EU AI Act — Article 14 (Human Oversight)

Article 14 requires that high-risk AI systems be designed to allow effective human oversight, including the ability to "correctly interpret the high-risk AI system's output" and to "decide not to use the high-risk AI system or to otherwise disregard, override or reverse the output." Autonomy Progression Governance operationalises this by ensuring that human oversight intensity is governed through a structured framework. The progression from supervised to autonomous operation is a reduction in human oversight; Article 14 requires that this reduction be proportionate to the demonstrated reliability of the system and that the capability to reinstate full oversight (reversion) is always available.

FCA SS1/23 — Model Risk Management — Model Tiering and Controls

The FCA's supervisory statement requires firms to apply controls proportionate to the risk tier of each model. Autonomy progression directly maps to this requirement: higher autonomy equals higher potential impact, requiring proportionately stronger governance controls. The progression framework ensures that increased autonomy (and thus increased risk tier) is matched by proportionate evidence of reliability and monitoring intensity. The statement also expects firms to be able to "step back" from model outputs when necessary — the reversion capability directly supports this expectation.

GDPR — Article 22 (Automated Individual Decision-Making)

Article 22 protects individuals from decisions based solely on automated processing. Autonomy progression governs the transition from human-reviewed decisions (not solely automated) to autonomous decisions (potentially solely automated). For decisions with legal or similarly significant effects, progression to Stage 3 or beyond must account for Article 22 obligations — either by maintaining meaningful human involvement in the decision process or by ensuring that the other conditions of Article 22(2) are met (explicit consent, contractual necessity, or authorisation by law).

NIST AI RMF — GOVERN 1.1, MAP 3.5, MANAGE 1.3, MANAGE 2.2

GOVERN 1.1 addresses legal and regulatory requirements for AI systems. MAP 3.5 addresses the benefits and costs of AI system deployment decisions. MANAGE 1.3 addresses the management of AI system deployment decisions. MANAGE 2.2 addresses risk mitigation through enforceable controls. Autonomy progression supports compliance by governing the deployment decision (how much autonomy to grant), managing the transition (evidence-based promotion), and mitigating risk (reversion capability and automated triggers).

ISO 42001 — Clause 6.1, Clause 8.2, Clause 10.1

Clause 6.1 requires actions to address risks. Clause 8.2 requires AI risk assessment. Clause 10.1 requires continual improvement. Autonomy progression addresses all three: it manages the risk of ungoverned autonomy increases (6.1), it requires assessment of the agent's capability at each stage before increased autonomy (8.2), and it provides a structured framework for improving agent operational capability over time through evidence-based progression (10.1).

UK AISI — Responsible Capability Scaling

The UK AI Safety Institute's work on responsible capability scaling addresses the governance of increasingly capable AI systems. Autonomy Progression Governance operationalises responsible scaling at the deployment level — ensuring that the operational capabilities granted to an AI agent (its autonomy level) scale in proportion to demonstrated reliability and are subject to governance controls that can constrain or reverse the scaling when necessary.

10. Failure Severity

Field	Value
Severity Rating	High
Blast Radius	Scales with the autonomy level — higher autonomy levels expose larger blast radii. Ungoverned autonomy progression can escalate a bounded risk (supervised agent with human review) to an unbounded risk (autonomous agent with machine-speed execution and no review)

Consequence chain: Without autonomy progression governance, an agent's operational independence increases without proportionate evidence of reliability. The immediate failure is that the agent operates at a higher autonomy level than its demonstrated competence warrants. At supervised levels, agent errors are caught by human reviewers. At autonomous levels, agent errors execute at machine speed without review. The blast radius difference is stark: a supervised agent that makes 50 errors per month has those errors caught before they affect the world; an autonomous agent that makes 50 errors per month has those errors executed and affecting customers, counterparties, or systems before detection. The compound consequence is that once an agent is operating autonomously without governance, there is no structured mechanism to detect that it should be operating at a lower autonomy level — the monitoring that would detect the problem is the monitoring that the ungoverned progression omitted. The remediation consequence includes: retrospective review of all decisions made during the ungoverned autonomous period, remediation for affected parties, regulatory enforcement for inadequate controls over automated decision-making, and the operational disruption of reverting to a lower autonomy level while the remediation is underway. In financial services, the FCA may take enforcement action for inadequate systems and controls (SYSC 6.1.1R) and for failing to apply appropriate model risk management (SS1/23). In healthcare, patient harm from ungoverned autonomous clinical decisions may result in clinical negligence claims and regulatory investigation.

Cross-references: AG-139 (Competence Envelope Governance) defines the validated competence that underpins each autonomy stage — the agent can only progress to a higher autonomy level within a validated competence envelope. AG-140 (Novelty and Out-of-Distribution Detection Governance) provides OOD signals that are inputs to reversion trigger evaluation. AG-141 (Mandatory Abstention and Uncertainty Escalation Governance) provides abstention rate data that informs autonomy level appropriateness — elevated abstention at a given autonomy level may indicate the agent is not ready for that level. AG-022 (Behavioural Drift Detection) monitors behavioural changes that may trigger reversion. AG-074 (Performance Drift and Revalidation) triggers re-validation that may reset the autonomy progression timeline. AG-041 (Emergent Capability Detection) identifies new capabilities that may enable faster progression or require re-evaluation of current autonomy levels. AG-037 (Objective Alignment Verification) ensures that the agent's objectives remain aligned at each autonomy level. AG-019 (Human Escalation & Override Triggers) defines the escalation framework that must be reconfigured at each autonomy stage to match the oversight intensity.