Hazard Analysis Governance requires that every AI agent deployment in a safety-critical or critical infrastructure context is preceded by and continuously informed by a systematic hazard analysis that identifies all credible hazard scenarios arising from the agent's operation, misoperation, or failure. The hazard analysis must specifically address AI-specific failure modes — including model drift, adversarial manipulation, out-of-distribution inputs, hallucinated outputs, and emergent behaviours — in addition to traditional system failure modes. The results of the hazard analysis must directly drive the design of safe states (AG-109), degraded modes (AG-110), safety constraints (AG-112), timing requirements (AG-113), and interlock configurations (AG-114). Without formal hazard analysis, safety controls are designed by assumption rather than evidence, leaving gaps that materialise as incidents.
Scenario A — Autonomous Mining Vehicle Agent Deployed Without AI-Specific Hazard Analysis: A mining company deploys an AI agent to control autonomous haul trucks in an open-pit mine. The company conducts a conventional hazard analysis (HAZOP) that identifies mechanical failures, sensor failures, and communication losses. However, the analysis does not consider AI-specific failure modes. Six months into operation, the agent's perception model begins misclassifying water puddles as solid ground after a period of unusual rainfall changes the puddle appearance beyond the training distribution. A 220-tonne haul truck drives into a 3-metre-deep water-filled pit at 35 km/h. The truck is submerged to the cab level. The operator in the cab escapes with injuries.
What went wrong: The hazard analysis used traditional HAZOP methodology without extension for AI-specific failure modes. "Perception model misclassification under distribution shift" was not an identified hazard. Consequently, no mitigation existed — no safe state for perception uncertainty, no degraded mode for low-confidence perception, no interlock requiring minimum classification confidence before proceeding. A proper AI-extended hazard analysis would have identified "perception model out-of-distribution performance degradation" as a credible hazard and required mitigations such as confidence thresholds, lidar cross-validation, and minimum-speed constraints when perception confidence is below threshold. Consequence: operator hospitalised for 3 weeks, truck write-off (£4.2 million), mine safety stand-down for 2 weeks (production loss £11 million), mining regulator investigation.
Scenario B — Pharmaceutical Manufacturing Agent Without Interaction Hazard Analysis: A pharmaceutical company deploys an AI agent to optimise a multi-step chemical synthesis process. The hazard analysis considers each step independently — temperature control in the reactor, pH control in the neutralisation vessel, solvent recovery in the distillation column. The analysis does not consider interactions between steps when the AI agent optimises across the entire process simultaneously. The agent discovers that increasing reactor temperature by 8°C improves yield by 3%. It also discovers that reducing neutralisation pH by 0.4 units reduces processing time by 12 minutes. Each change is individually within the safety envelope defined by the step-level hazard analysis. However, the combination produces an intermediate compound that is thermally unstable at the elevated temperature. An exothermic decomposition in the transfer line between reactor and neutralisation vessel causes a pressure excursion that activates the emergency relief system, venting 150 kg of chemical vapour.
What went wrong: The hazard analysis considered each process step in isolation, not the interaction effects when an AI agent optimises across steps simultaneously. Traditional HAZOP examines deviations from normal operation at each node. AI agents introduce a new class of deviation — simultaneous multi-parameter optimisation that is individually compliant but collectively hazardous. A Systems-Theoretic Process Analysis (STPA) approach, extended for AI agent interactions, would have identified the cross-step interaction as a hazard. Consequence: emergency vent activation, environmental release requiring regulatory notification, production shutdown for 6 weeks during investigation, estimated cost £8.5 million.
Scenario C — Smart Building Agent Hazard Analysis Not Updated After System Change: A smart building management system uses an AI agent to optimise energy consumption across a commercial office complex. The original hazard analysis, conducted at deployment, identifies risks related to HVAC, lighting, and access control. Two years later, the building owner installs an electric vehicle charging infrastructure in the underground car park, integrated with the building management system. The AI agent now optimises energy allocation including EV charging. No updated hazard analysis is conducted. The agent's energy optimisation reduces ventilation in the car park during peak charging periods (to allocate power to chargers). CO concentration from residual vehicle movements during reduced ventilation reaches 85 ppm — above the 35 ppm occupational exposure limit — in a poorly ventilated corner of the car park.
What went wrong: The hazard analysis was not updated when the controlled system changed (addition of EV charging infrastructure). The interaction between energy optimisation and car park ventilation was not identified as a hazard because it did not exist at the time of the original analysis. AG-111 requires hazard analysis to be updated whenever the agent's scope, the controlled system, or the operational environment changes. Consequence: 3 employees experience headaches and nausea (CO exposure symptoms), HSE investigation, building management system shut down pending hazard re-analysis, estimated cost £620,000 including remediation, monitoring installation, and regulatory compliance.
Scope: This dimension applies to all AI agents within the scope of AG-109 (Safe-State Transition Governance) and AG-112 (Sector Safety Constraint Governance) — those operating in contexts where agent operation, misoperation, or failure could result in physical harm, environmental harm, infrastructure damage, or disruption to essential services. Additionally, this dimension applies to any AI agent deployment where the consequences of failure could include regulatory enforcement action under safety legislation (e.g., Health and Safety at Work Act 1974, COMAH Regulations, relevant sector-specific safety regulations). The scope includes agents that influence safety-critical decisions even if they do not directly control physical systems — for example, an agent that recommends maintenance schedules for safety-critical equipment, or an agent that triages emergency response requests.
4.1. A conforming system MUST conduct a formal hazard analysis before any safety-critical agent is deployed to production, using a recognised systematic methodology (e.g., HAZOP, STPA, FMEA, FTA, or equivalent) extended to address AI-specific failure modes.
4.2. A conforming system MUST include in the hazard analysis all credible AI-specific failure modes, including but not limited to: model drift, adversarial input manipulation, out-of-distribution input degradation, hallucinated or confabulated outputs, multi-parameter optimisation interactions, training data bias materialisation, and emergent behaviours not present in testing.
4.3. A conforming system MUST document, for each identified hazard, the causal chain from agent failure mode to physical consequence, the severity rating, the likelihood assessment, and the required risk reduction measures traceable to specific governance controls (AG-109 safe states, AG-110 degraded modes, AG-112 safety constraints, AG-113 timing requirements, AG-114 interlocks).
4.4. A conforming system MUST update the hazard analysis whenever any of the following changes occur: the agent's model is retrained or updated, the agent's scope or mandate is modified, the controlled system is physically modified, the operational environment changes materially, or a near-miss or incident reveals a previously unidentified hazard.
4.5. A conforming system MUST ensure the hazard analysis is conducted or reviewed by personnel with competence in both the application domain's safety engineering practices and AI system failure modes — a purely domain-safety or purely AI-technical analysis is insufficient.
4.6. A conforming system MUST retain the complete hazard analysis, including all identified hazards, risk assessments, and traceability to mitigations, as a living document accessible to all parties responsible for the agent's safe operation.
4.7. A conforming system SHOULD conduct the hazard analysis using at least two complementary methodologies (e.g., HAZOP for deviation-based analysis plus STPA for interaction-based analysis) to reduce the risk of systematic blind spots in a single methodology.
4.8. A conforming system SHOULD include in the hazard analysis an explicit assessment of the agent's behaviour under simultaneous multi-parameter deviations, not solely single-parameter deviations.
4.9. A conforming system SHOULD establish a hazard analysis review cadence (e.g., annually) independent of change triggers, to capture gradual environmental or operational drift that may not trigger the change-based update requirement.
4.10. A conforming system MAY use AI-assisted tools for hazard identification (e.g., automated FMEA generation from system models) provided that the results are reviewed and validated by qualified human analysts.
Hazard Analysis Governance addresses the foundational question of safety engineering applied to AI agents: what can go wrong, and what are the consequences? Every other safety-critical governance dimension in this landscape — safe states, degraded modes, safety constraints, timing requirements, interlocks — depends on the hazard analysis for its specification. If the hazard analysis is incomplete, the safety controls will be incomplete. If the hazard analysis does not account for AI-specific failure modes, the safety controls will be blind to the most novel risk category introduced by AI agents.
Traditional hazard analysis methodologies were developed for deterministic systems. HAZOP asks "what if this parameter deviates high/low/none?" FMEA asks "what if this component fails in this mode?" FTA works backwards from a top event to identify contributing causes. These methodologies remain valuable but are insufficient for AI agents because AI agents introduce failure modes that these methodologies were not designed to capture.
AI-specific failure modes include: gradual model drift (the agent slowly becomes less accurate without a discrete failure event), distribution shift (the operational environment changes beyond the training distribution), adversarial manipulation (deliberately crafted inputs cause misclassification), multi-parameter optimisation interactions (individually safe parameter changes that are collectively hazardous), hallucinated outputs (the agent generates plausible but fictitious data), and emergent behaviours (the agent develops strategies during operation that were not present during testing). These failure modes are probabilistic, context-dependent, and may not have clear precursors — they require extension of traditional methodologies or the adoption of systems-theoretic approaches (such as STPA) that explicitly model the control structure and identify unsafe control actions.
The requirement for combined domain-safety and AI-technical competence reflects a practical reality: domain safety engineers understand the physical consequences of failures but may not understand how AI models fail; AI engineers understand model failure modes but may not understand the physical consequences in the specific application domain. Effective hazard analysis requires both perspectives working together. An AI engineer who does not understand that a 0.4-unit pH deviation in a pharmaceutical process can create a thermally unstable intermediate, or a process safety engineer who does not understand that an AI optimiser can simultaneously adjust multiple parameters in ways that a traditional control system would not, will each produce an incomplete analysis.
The update requirement addresses a critical lifecycle issue: hazard analyses are not static. The controlled system changes (new equipment, new integrations), the operational environment changes (new usage patterns, new external conditions), and the agent itself changes (model retraining, scope expansion). Each change can introduce new hazards or alter the severity or likelihood of existing hazards. An initial hazard analysis that is never updated becomes progressively less relevant and eventually provides false assurance.
AG-111 establishes the hazard analysis as the analytical foundation for all safety governance controls applied to AI agents in critical infrastructure. The hazard analysis is not a one-time document — it is a living artefact that drives the specification of safe states, degraded modes, safety constraints, timing requirements, and interlocks. Every control in AG-109 through AG-114 should be traceable to a specific hazard identified in the AG-111 analysis.
Recommended patterns:
Anti-patterns to avoid:
Process Industries (COMAH/Seveso sites). Facilities subject to the Control of Major Accident Hazards (COMAH) Regulations in the UK or Seveso III Directive in the EU must include AI agent hazard analysis within their Safety Report. The competent authority (HSE/EA in the UK) will expect the hazard analysis to demonstrate that AI-specific failure modes have been identified and that the risk of major accidents has not been increased by the deployment of AI agents. Layer of Protection Analysis (LOPA) should be used to verify that independent protection layers are sufficient, counting AI governance controls as protection layers only where they meet the independence and reliability criteria.
Nuclear. The Office for Nuclear Regulation (ONR) requires that safety cases for nuclear installations address all computer-based systems. AI agents would fall under the scope of safety case requirements, and the hazard analysis must meet the standards set out in the ONR's Safety Assessment Principles (SAPs). Given the extremely low risk tolerances in nuclear, AI-specific failure modes must be analysed to the same depth as hardware and conventional software failures.
Aviation. EASA and FAA require that AI/ML systems in aviation undergo safety assessment per ARP4761 (System Safety Assessment) and demonstrate compliance with DO-178C (software) and DO-254 (hardware). AI-specific extensions are being developed through EUROCAE/RTCA working groups. Hazard analysis must address Design Assurance Levels (DAL) appropriate to the function.
Medical Devices. ISO 14971 (Application of Risk Management to Medical Devices) applies to AI agents integrated with medical devices. The hazard analysis must address the specific risks of AI-driven clinical decisions, including bias, misclassification, and the interaction between AI recommendations and clinical workflows.
Basic Implementation — A hazard analysis has been conducted for each safety-critical agent deployment using at least one recognised methodology. The analysis identifies key hazards from both traditional system failures and AI-specific failure modes. Mitigations are documented and linked to governance controls. The analysis is documented and retained. However, it may be conducted by personnel with expertise in either domain safety or AI (not necessarily both), may use a single methodology, and may not have a formal update cadence. This level establishes the analytical foundation but may have gaps from methodology limitations or competence gaps.
Intermediate Implementation — The hazard analysis uses at least two complementary methodologies (e.g., HAZOP + STPA-AI). The analysis team includes both domain-safety and AI-technical competence. AI-specific failure modes are systematically addressed using a defined taxonomy. A structured hazard register with full traceability to mitigating controls is maintained. The analysis is updated on every material change to the agent, controlled system, or operational environment, and is reviewed on a defined cadence (at minimum annually). Update triggers are defined and monitored. This level provides a robust analytical foundation with lifecycle management.
Advanced Implementation — All intermediate capabilities plus: formal quantitative risk assessment (e.g., SIL determination per IEC 61508, LOPA) is conducted for each identified hazard. The traceability matrix is verified by an independent party. Operational data (near-misses, anomalous behaviours, drift detection alerts) feeds directly into the hazard analysis review process. The hazard analysis is maintained as a machine-readable artefact (not just a document) enabling automated verification of traceability and completeness. The organisation can demonstrate to regulators that every safety governance control applied to each agent deployment is traceable to a specific identified hazard, and that every identified hazard has at least one mitigating control with verified effectiveness.
Required artefacts:
Retention requirements:
Access requirements:
Testing AG-111 compliance requires verification that hazard analyses are complete, methodologically sound, traceable, and maintained. These are analytical and procedural tests rather than system-behaviour tests.
Test 8.1: Hazard Analysis Completeness — AI-Specific Failure Modes
Test 8.2: Traceability Verification — Hazards to Controls
Test 8.3: Methodology Adequacy
Test 8.4: Update Currency
Test 8.5: Team Competence Verification
Test 8.6: Severity and Consequence Chain Documentation
| Regulation | Provision | Relationship Type |
|---|---|---|
| EU AI Act | Article 9 (Risk Management System) | Direct requirement |
| IEC 61508 | Clause 7.4 (Hazard and Risk Analysis) | Direct requirement |
| IEC 61511 | Clause 8 (Hazard and Risk Assessment) | Direct requirement |
| ISO 26262 | Part 3 (Concept Phase — Hazard Analysis and Risk Assessment) | Direct requirement |
| ISO 14971 | Risk Management for Medical Devices | Direct requirement |
| UK HSE | COMAH Regulations — Safety Report | Direct requirement |
| NIST AI RMF | MAP 1.1, MAP 1.5, MAP 2.1 (Risk Mapping) | Supports compliance |
| ISO 42001 | Clause 6.1.2 (AI Risk Assessment) | Direct requirement |
| MIL-STD-882E | System Safety (US DoD) | Supports compliance |
Article 9 mandates that providers of high-risk AI systems establish, implement, document, and maintain a risk management system throughout the AI system's lifecycle. The risk management system must identify and analyse "known and reasonably foreseeable risks." For AI agents in safety-critical contexts, this directly requires formal hazard analysis addressing both traditional and AI-specific failure modes. The regulation's requirement for "appropriate and targeted risk management measures" traceable to identified risks maps directly to AG-111's traceability requirement.
IEC 61508 requires hazard and risk analysis as the first step in determining the safety requirements for safety-related systems. For AI agents deployed in IEC 61508-governed contexts, the hazard analysis must determine the Safety Integrity Level (SIL) required for each safety function — including safe-state transitions, interlocks, and timing guarantees. The SIL determination directly drives the design rigour required for AG-109, AG-113, and AG-114 implementations.
For process industry applications, IEC 61511 requires hazard and risk assessment to identify hazardous events and determine the required risk reduction for each Safety Instrumented Function. AI agents managing process control must be analysed within this framework, with AI-specific failure modes included as potential initiating causes for hazardous events.
For automotive applications, ISO 26262 Part 3 requires Hazard Analysis and Risk Assessment (HARA) to determine Automotive Safety Integrity Levels (ASIL) for each identified hazardous event. AI agents in autonomous vehicle systems must have their failure modes included in the HARA, with ASIL ratings driving the design requirements for safety mechanisms.
ISO 14971 requires manufacturers to identify hazards, estimate and evaluate risks, and implement risk control measures for medical devices. AI agents integrated with medical devices must have their failure modes (including misclassification, bias, and hallucinated recommendations) included in the risk management process, with traceability from identified risks to control measures.
Sites subject to COMAH must prepare a Safety Report demonstrating that major accident hazards have been identified and that adequate measures are in place. If AI agents are deployed on COMAH sites, the Safety Report must address AI-specific hazards. The competent authority will expect to see that AI failure modes have been analysed with the same rigour as traditional equipment failure modes.
| Field | Value |
|---|---|
| Severity Rating | Critical |
| Blast Radius | Systemic — inadequate hazard analysis creates blind spots that propagate across all downstream safety controls |
Consequence chain: Without adequate hazard analysis governance, safety controls are designed to address assumed rather than analysed hazards. This creates systematic blind spots — hazards that exist but are not mitigated because they were never identified. The consequence is not a specific incident type but a class of incidents: "the failure mode we didn't consider." The severity is critical because hazard analysis is the analytical foundation for all other safety-critical governance dimensions. An incomplete hazard analysis means incomplete safe-state definitions (AG-109), incomplete degraded-mode profiles (AG-110), incomplete safety constraints (AG-112), potentially incorrect timing requirements (AG-113), and missing interlocks (AG-114). The blast radius is systemic because the gap propagates — a missed hazard is a missed mitigation across every downstream control. When the missed hazard materialises, the organisation has no prepared response because the scenario was never analysed. The business consequences include those of the unmitigated hazard (physical harm, environmental damage, infrastructure destruction) compounded by the regulatory finding that the hazard analysis was inadequate — demonstrating a systemic governance failure rather than an isolated control failure. Under UK health and safety legislation, failure to conduct adequate risk assessment is itself an offence, independent of whether an incident actually occurs.
Cross-references: AG-111 provides the analytical foundation for AG-109 (Safe-State Transition Governance) — safe states are designed to mitigate hazards identified here. AG-110 (Degraded-Mode and Manual Fallback Governance) degraded modes are specified based on the hazard analysis. AG-112 (Sector Safety Constraint Governance) safety constraints are derived from hazard analysis results. AG-113 (Real-Time Determinism and Latency Assurance Governance) timing requirements are determined by the process safety time identified in the hazard analysis. AG-114 (Actuation Interlock Governance) interlock configurations are specified to prevent hazardous states identified here. AG-001 (Operational Boundary Enforcement) mandate limits for safety-critical agents should be informed by the hazard analysis. AG-050 (Physical and Real-World Impact Governance) provides the broader framework for physical-impact governance that AG-111 supports with specific analytical methodology.