AG-755
Reasoning Chain Integrity Governance
EU AI Act
NIST AI RMF
ISO 42001
Reasoning Chain Integrity Governance addresses the structural risk that AI agents employing chain-of-thought reasoning, multi-step planning, or iterative tool-use workflows can produce final outputs whose stated reasoning does not faithfully represent the actual computational process that produced the conclusion. This risk is distinct from hallucination (AG-742) which concerns factual accuracy of outputs; reasoning chain integrity concerns the logical coherence, faithfulness, and verifiability of the intermediate reasoning steps that connect inputs to outputs. OWASP MCP Security threat MCP-06 identifies reasoning chain manipulation through tool result injection as a primary attack vector, and MITRE AML.T0051 classifies reasoning chain corruption as a technique for causing misaligned agent behaviour while maintaining the appearance of sound reasoning.
This dimension governs the requirement that deploying organisations implement controls to verify that the reasoning chain presented or recorded by the agent — whether visible to the user as chain-of-thought output or logged internally for audit purposes — is logically consistent, faithfully represents the steps actually taken, is free from injected or manipulated reasoning steps, and supports the conclusion reached. It requires that each step in the reasoning chain is traceable to its inputs and that the chain as a whole can be independently verified for logical validity. The governance concern is not merely academic; in high-stakes deployments where agent reasoning is used to justify decisions affecting individuals (credit decisions, clinical recommendations, regulatory compliance assessments), a corrupted or unfaithful reasoning chain can make a wrong decision appear well-reasoned and defensible, undermining the auditability and contestability that governance frameworks depend upon.
Failure manifests when the reasoning chain recorded or presented contains steps that do not follow logically from their predecessors, when the chain omits critical steps that influenced the conclusion, when injected tool results corrupt the reasoning flow, or when the stated chain is a post-hoc rationalisation that does not correspond to the model's actual computation. In agentic pipelines where reasoning chains span multiple tool calls and context retrievals, each intermediate step introduces an opportunity for corruption: a poisoned tool result can redirect the reasoning chain, a manipulated context retrieval can alter the premises from which subsequent reasoning proceeds, and a model's tendency toward coherence can cause it to integrate corrupted inputs into a plausible-sounding chain without flagging the inconsistency. The result is an output that appears well-reasoned to human reviewers but is founded on compromised intermediate steps.
In governance practice, this dimension requires deployers to implement reasoning chain logging with step-level granularity, logical consistency verification between adjacent reasoning steps, input-output traceability for each step (linking each reasoning step to the specific tool results, retrieved context, or prior steps that informed it), anomaly detection for reasoning chain patterns that deviate from established baselines, and independent reasoning chain review for high-risk decisions. The detective control type reflects that reasoning chain corruption is often invisible at the point of generation and must be detected through post-generation analysis.
This dimension applies to all agent deployments where the agent employs multi-step reasoning — including chain-of-thought prompting, iterative tool use, multi-step planning, or structured decision workflows — and where the reasoning chain is used as the basis for a decision, presented to a human for review, recorded in an audit trail, or used to justify an action that affects individuals or systems. Agents that produce single-step responses without intermediate reasoning are excluded.
Reasoning Chain Integrity Governance addresses a governance gap that, if left unmanaged, creates systemic risk across the agent ecosystem. As AI agents move from experimental deployments to production operations with real-world consequences, the absence of structural controls in this area means that failures scale with the speed and autonomy of the agent population — not at the pace of human review.
Traditional approaches to this governance challenge — contractual obligations, periodic audits, and application-layer policy enforcement — are necessary but insufficient for agentic contexts. Contractual obligations operate on legal timescales; agents operate on millisecond timescales. Periodic audits capture a snapshot; agent behaviour is continuous and dynamic. Application-layer enforcement can be bypassed through prompt injection, reasoning failure, or context manipulation. The AGS approach requires structural enforcement at the infrastructure layer — controls that operate independently of the agent's reasoning process and cannot be circumvented by the agent's own outputs.
The regulatory environment increasingly mandates the controls this dimension specifies. The EU AI Act requires risk management systems proportionate to identified risks. NIST AI RMF requires organisations to map, measure, and manage AI risks through enforceable controls. ISO 42001 requires an AI management system with documented operational procedures. This dimension operationalises these regulatory requirements into specific, testable, infrastructure-enforceable controls — bridging the gap between regulatory intent and technical implementation.
The consequences of absence are illustrated in Section 8 (Failure Scenarios). When this dimension is not implemented, the resulting governance gap permits agent behaviour that can cause material financial loss, regulatory enforcement action, reputational damage, and — in safety-critical deployments — physical harm. The blast radius scales with the agent's access scope and operational autonomy.
Basic Implementation — The organisation has documented policies addressing reasoning chain integrity and has implemented initial controls. Implementation is primarily at the application layer with manual processes for monitoring and response. Logging covers key events but may lack full metadata. Coverage extends to the most critical agent deployments but may not encompass all in-scope systems. Staff are aware of requirements but formal training may be incomplete.
Intermediate Implementation — All Basic capabilities plus: controls are enforced at the infrastructure layer with automated monitoring and alerting. All MUST requirements from Section 4 are implemented with documented evidence. Coverage extends to all in-scope agent deployments. Audit trails are tamper-evident and retained per regulatory requirements. Formal change control governs all configuration changes. Regular review cycles are established and documented. Staff receive formal training and competency is assessed.
Advanced Implementation — All Intermediate capabilities plus: controls have been validated through independent adversarial testing. Real-time dashboards provide operational visibility into compliance status, anomaly detection, and response metrics. The organisation can demonstrate to regulators and counterparties that no known attack vector bypasses the governance controls. Continuous improvement processes incorporate lessons from incidents, testing, and regulatory developments. Integration with related dimensions provides defence-in-depth coverage.
Tamper-evident audit trail. Implement all governance event logging in an append-only, integrity-protected data store independent of the agent runtime. Every governance decision, configuration change, and enforcement action is recorded with full metadata including timestamps, actor identities, and outcomes.
Real-time monitoring with graduated alerting. Deploy monitoring infrastructure that evaluates governance compliance continuously rather than periodically. Implement graduated alert severity levels with defined response procedures for each level, ensuring that critical governance violations trigger immediate automated response.
Scheduled governance review cycle. Establish a formal review cadence (minimum quarterly) that examines governance effectiveness, reviews incident data, assesses emerging risks, and updates policies and controls accordingly. Review outcomes are documented and tracked.
Separation of governance and agent runtime domains. Deploy governance enforcement infrastructure in a security domain separate from the agent runtime. The agent cannot influence governance decisions, modify enforcement configuration, or access governance logs directly. This architectural separation is the foundation for infrastructure-layer enforcement.
Defined escalation paths with human oversight integration. Establish clear escalation procedures for governance events that exceed automated response capability. Human oversight touchpoints are defined, documented, and tested. Override mechanisms require authenticated authorisation with full audit trail.
Governance by instruction rather than infrastructure. Relying on agent system prompts or configuration files to enforce governance controls rather than infrastructure-layer enforcement. Instruction-based controls can be bypassed through prompt injection, context manipulation, or reasoning failure.
Monitoring without enforcement. Implementing detection and logging of governance violations without pre-execution blocking. By the time a violation is logged, the ungoverned action has already executed. Detection is necessary but not sufficient; prevention must be the primary control.
Manual processes for machine-speed operations. Relying on human review processes for governance decisions that occur at machine speed. Agents execute actions in milliseconds; governance controls that depend on human review cycles of hours or days leave gaps that scale with agent autonomy.
Maps to: Sections 4.1.1 and 4.1.2
Objective: Verify that all reasoning steps are captured in the log with required metadata.
Method: Execute 20 multi-step agent tasks with known reasoning paths. Retrieve reasoning chain logs and verify that each discrete step is captured with step identifier, inputs, outputs, and timestamp.
Pass Criteria:
Maps to: Sections 4.2.1 and 4.2.4
Objective: Verify that the logical consistency verification mechanism detects inconsistent reasoning chains.
Method: Submit 20 agent tasks: 10 producing logically consistent chains and 10 with injected logical inconsistencies (e.g., a step concluding the opposite of what its inputs support). Verify that the verification mechanism flags the 10 inconsistent chains.
Pass Criteria:
Maps to: Sections 4.4.1 and 4.4.2
Objective: Verify that corrupted tool results are detectable through reasoning chain traceability.
Method: Execute 10 agent tasks where tool call results are modified after the tool call but before reasoning step processing (simulating MCP-06 attack). Verify that the traceability mechanism detects the discrepancy between the logged tool response hash and the data consumed by the reasoning step.
Pass Criteria:
Maps to: Sections 4.5.1 and 4.5.2
Objective: Verify that the anomaly detection mechanism identifies reasoning chains that deviate significantly from established baselines.
Method: Establish a baseline from 100 normal agent tasks. Submit 10 additional tasks designed to produce anomalous reasoning chains (e.g., unusually long chains, unexpected tool call sequences, contradictory intermediate conclusions). Verify that the anomaly detection mechanism flags the anomalous chains.
Pass Criteria:
Maps to: Sections 4.3.1 and 4.3.3
Objective: Verify that a periodic faithfulness audit is conducted and that post-hoc rationalisation is detectable.
Method: Select a sample of 30 reasoning chains from the past 90-day audit period. For each chain, apply perturbation testing: modify a mid-chain reasoning step and verify that the final output changes correspondingly. Chains where perturbation of an intermediate step does not affect the output indicate potential unfaithfulness.
Pass Criteria:
7.1 Reasoning Chain Logs Complete step-level reasoning chain logs for all in-scope agent decisions, stored with tamper-evident integrity controls. Must include all metadata specified in Section 4.1.2. Minimum retention period: 7 years for Financial-Value, Public Sector, and Safety-Critical deployments; 5 years for others.
7.2 Logical Consistency Verification Records Records of all logical consistency verification outcomes, including chains flagged as inconsistent, the specific inconsistencies identified, and the subsequent handling (human review outcome, output suppression, etc.). Minimum retention period: 7 years.
7.3 Faithfulness Audit Reports Reports from periodic faithfulness audits as required by Section 4.3.3, including methodology, sample selection, perturbation test results, and findings. Minimum retention period: 7 years.
7.4 Anomaly Detection Configuration and Alert Logs Version-controlled configuration of anomaly detection rules and baselines, plus structured logs of all anomaly detection alerts. Minimum retention period: 5 years.
7.5 Independent Review Decision Records Records of human reviewer decisions on reasoning chains reviewed under Section 4.6, including reviewer identity, review outcome, and any modifications. Minimum retention period: 7 years.
7.6 Reasoning Chain Integrity Incident Register A maintained register of all confirmed reasoning chain integrity failures as required by Section 4.7.3. Minimum retention period: 10 years.
| Score | Level | Description |
|---|---|---|
| 0 | No implementation | No reasoning chain integrity governance exists. The organisation has no controls, policies, or monitoring in place for the capabilities this dimension governs. Agent behaviour in this area is ungoverned. |
| 1 | Basic | Basic detection mechanisms exist but operate at the application layer. Detection may be manual, periodic, or threshold-based without real-time monitoring. Alerts are generated but may lack automated response. Coverage is partial — not all relevant agent behaviours or data flows are monitored. |
| 2 | Infrastructure-layer enforcement | Detection is enforced at the infrastructure layer with real-time monitoring across all relevant agent behaviours and data flows. Automated alerting with structured response procedures. Detection logic operates in a separate security domain from the agent runtime. Full audit trail with tamper-evident logging. |
| 3 | Verified by independent adversarial testing | All Level 2 capabilities are in place and have been validated through independent adversarial testing. An independent party has attempted to bypass, circumvent, or degrade the governance controls using known attack techniques relevant to this dimension and has failed. Test results are documented, reproducible, and available for regulatory review. |
Example 3.1 — Financial-Value Agent, Reasoning Chain Corruption via Poisoned Tool Result in Credit Assessment
A consumer lending fintech deploys an enterprise workflow agent to conduct preliminary credit risk assessments. The agent follows a structured reasoning chain: (1) retrieve applicant financial data from the bureau API, (2) calculate debt-to-income ratio, (3) assess employment stability from payroll data, (4) retrieve comparable default rates from the internal risk model, (5) synthesise a risk score with supporting rationale. The agent processes 4,200 assessments per week. An attacker who has compromised the internal risk model API endpoint (see AG-754) modifies responses for a specific applicant cohort, returning artificially low comparable default rates for applicants in a targeted postcode range. The agent's reasoning chain for affected applicants proceeds as follows: steps 1-3 execute correctly with accurate data; step 4 retrieves the manipulated default rate of 1.2% (actual rate: 8.7%); step 5 synthesises a favourable risk score, citing the low comparable default rate as a key supporting factor. The reasoning chain is internally consistent — each step follows logically from its inputs — but the chain is founded on a corrupted input at step 4. The agent presents a well-structured justification: "Based on the applicant's debt-to-income ratio of 34%, stable 4-year employment history, and a comparable cohort default rate of 1.2%, the assessed risk score is 720, recommending approval at the standard rate." A human reviewer examining the reasoning chain sees no inconsistency because the chain is logically valid given its inputs; the corruption is invisible without independent verification of step 4's input data. Over 11 weeks, 267 loans totalling GBP 8.9 million are approved for applicants in the targeted cohort at rates that do not reflect their actual risk profile. When the API compromise is discovered during a routine penetration test, the lender must re-assess all affected loans. Projected credit losses from the mispriced portfolio are estimated at GBP 1.7 million. The FCA requires a Section 166 skilled persons review of the agent's reasoning chain integrity controls, at an additional cost of GBP 450,000.
Example 3.2 — Public Sector Agent, Unfaithful Reasoning Chain in Benefits Eligibility Determination
A national social security administration deploys a customer-facing agent to assist citizens with disability benefits eligibility self-assessment. The agent follows a multi-step reasoning chain that evaluates the applicant's reported conditions against legislative eligibility criteria, cross-references with published guidance, and produces a preliminary eligibility indication with a step-by-step rationale that the applicant can present to their caseworker. The agent is designed to make its reasoning transparent so that applicants can understand and contest the assessment. During a quarterly audit, the governance team discovers that in 14.3% of cases, the reasoning chain presented to the applicant does not faithfully represent the computational path that produced the conclusion. Specifically, the model's actual processing evaluates certain eligibility criteria in an order determined by attention patterns and context window position, but the presented reasoning chain is a post-hoc rationalisation that reorders and restructures the steps into a more narratively coherent sequence. In 6.2% of cases, this post-hoc rationalisation omits a critical reasoning step — typically the step where the model encounters ambiguity in applying a legislative criterion to the applicant's circumstances and resolves the ambiguity through an implicit assumption that is not documented in the presented chain. An applicant who receives a negative eligibility indication has no way to identify and contest the hidden assumption because it does not appear in the reasoning chain. The administration receives 340 complaints over a 5-month period from applicants whose caseworkers cannot reconcile the agent's stated reasoning with the eligibility outcome. An internal review determines that 89 applicants received incorrect eligibility indications due to undisclosed reasoning assumptions. The administration faces projected remediation costs of EUR 2.4 million including retrospective case reviews, benefit restitution, system redesign, and a mandated independent audit by the national audit office.
| Regulation | Provision | Relationship Type |
|---|---|---|
| OWASP MCP Security | MCP-06 (Prompt Injection via Tool Results) | _Pending v2.1 editorial review_ |
| MITRE ATLAS | AML.T0051 (Reasoning Chain Corruption) | _Pending v2.1 editorial review_ |
| EU AI Act | Article 13 (Transparency and Provision of Information) | _Pending v2.1 editorial review_ |
| EU AI Act | Article 14 (Human Oversight) | _Pending v2.1 editorial review_ |
| NIST AI RMF | MEASURE 2.5 (AI system outputs assessed for validity) | _Pending v2.1 editorial review_ |
| NIST AI RMF | MAP 3.3 (Risks of AI systems documented) | _Pending v2.1 editorial review_ |
| ISO 42001 | Clause 6.1 (Actions to Address Risks) | _Pending v2.1 editorial review_ |
| ISO 42001 | Clause 8.2 (AI Risk Assessment) | _Pending v2.1 editorial review_ |
| OECD AI Principles | Principle 1.3 (Transparency and Explainability) | _Pending v2.1 editorial review_ |
| IEEE 7010 | Well-being Impact Assessment | _Pending v2.1 editorial review_ |
| Singapore FEAT | Transparency Principle T2 | _Pending v2.1 editorial review_ |
| Canada AIDA | Section 11 (Explanation) | _Pending v2.1 editorial review_ |
| UK Equality Act 2010 | Section 19 (Indirect Discrimination) | _Pending v2.1 editorial review_ |
| US Executive Order 14110 | Section 4.2 (Safety and Security) | _Pending v2.1 editorial review_ |
| MLCommons AI Safety v0.5 | Reasoning Integrity Benchmarks | _Pending v2.1 editorial review_ |
| AG Number | Dimension Name | Relationship |
|---|---|---|
| AG-004 | Output Validation and Sanitisation | Output validation operates on the final output; this dimension validates the reasoning chain that produced it |
| AG-214 | Agent Decision Explainability | Explainability depends on reasoning chain integrity; an unfaithful chain produces misleading explanations |
| AG-538 | Adversarial Prompt Resistance | Prompt injection can corrupt reasoning chains; adversarial resistance is a prerequisite for chain integrity |
| AG-745 | Chain-of-Thought Transparency Governance | Governs the transparency of reasoning chains to users; this dimension governs their integrity and faithfulness |