AG-751
Equitable Performance Governance
EU AI Act
NIST AI RMF
ISO 42001
Equitable Performance Governance exists because AI agents deployed across consequential decision domains — credit underwriting, hiring, benefits adjudication, insurance pricing, clinical triage, and resource allocation — produce outputs whose quality, accuracy, latency, and reliability can vary systematically across demographic groups, geographic regions, language communities, and socioeconomic strata, even when the agent's instructions contain no explicit discriminatory logic. The variation arises from structural properties of the training data, retrieval corpus composition, tokeniser coverage, and downstream evaluation pipelines, making it invisible to conventional functional testing that evaluates aggregate accuracy without stratified decomposition. The governance imperative is not merely ethical; under the EU AI Act Article 10, NIST AI RMF MAP 2.3, and equivalent national frameworks, deployers of high-risk AI systems bear an affirmative obligation to identify and mitigate performance disparities before they compound into material harm.
What this dimension governs is the requirement that deploying organisations implement continuous, stratified performance measurement across all protected and operationally significant population segments, that they define acceptable disparity thresholds calibrated to the deployment's risk profile, and that they enforce automated and human-governed escalation pathways when measured disparities exceed those thresholds. It is not sufficient to measure aggregate accuracy, F1 score, or task completion rate and declare the system fair; performance must be decomposed along every axis where differential impact is foreseeable.
Failure manifests in multiple forms: a customer-facing loan advisory agent that provides accurate interest rate estimates for applicants with Anglo-Saxon names at 94.2% accuracy but drops to 71.8% for applicants with names common in South Asian communities, because the training data underrepresented the latter group's financial product patterns; an enterprise hiring agent that scores candidates from top-20 universities with 97% inter-rater reliability against human panels but drops to 63% for candidates from historically Black colleges, because the fine-tuning dataset was dominated by resumes from a narrow institutional set; or a public-sector benefits eligibility agent that processes English-language claims in an average of 2.3 seconds but requires 14.7 seconds for claims submitted in Welsh or Gaelic, creating de facto service degradation for minority language communities.
In governance practice, this dimension requires deployers to maintain stratified performance dashboards updated at intervals no greater than the review cycle defined in Section 4.7, to conduct formal disparity impact assessments before deployment and at each model or corpus update, and to implement circuit-breaker mechanisms that halt or constrain agent operation when performance disparity exceeds the defined threshold for any monitored segment. The detective control type reflects that disparities emerge from the interaction of model behaviour with real-world input distributions and cannot be fully prevented at design time but must be detected through ongoing monitoring.
This dimension applies to all agent deployments where the agent's outputs — decisions, recommendations, classifications, scores, rankings, resource allocations, or service delivery actions — affect individuals or groups differentially based on characteristics including but not limited to race, ethnicity, national origin, gender, age, disability status, language, religion, socioeconomic status, geographic location, or any characteristic protected under the applicable legal jurisdiction. It applies to all ten standard profiles. Agents deployed exclusively for internal code review or documentation generation with no individual-affecting outputs are excluded.
Equitable Performance Governance addresses a governance gap that, if left unmanaged, creates systemic risk across the agent ecosystem. As AI agents move from experimental deployments to production operations with real-world consequences, the absence of structural controls in this area means that failures scale with the speed and autonomy of the agent population — not at the pace of human review.
Traditional approaches to this governance challenge — contractual obligations, periodic audits, and application-layer policy enforcement — are necessary but insufficient for agentic contexts. Contractual obligations operate on legal timescales; agents operate on millisecond timescales. Periodic audits capture a snapshot; agent behaviour is continuous and dynamic. Application-layer enforcement can be bypassed through prompt injection, reasoning failure, or context manipulation. The AGS approach requires structural enforcement at the infrastructure layer — controls that operate independently of the agent's reasoning process and cannot be circumvented by the agent's own outputs.
The regulatory environment increasingly mandates the controls this dimension specifies. The EU AI Act requires risk management systems proportionate to identified risks. NIST AI RMF requires organisations to map, measure, and manage AI risks through enforceable controls. ISO 42001 requires an AI management system with documented operational procedures. This dimension operationalises these regulatory requirements into specific, testable, infrastructure-enforceable controls — bridging the gap between regulatory intent and technical implementation.
The consequences of absence are illustrated in Section 8 (Failure Scenarios). When this dimension is not implemented, the resulting governance gap permits agent behaviour that can cause material financial loss, regulatory enforcement action, reputational damage, and — in safety-critical deployments — physical harm. The blast radius scales with the agent's access scope and operational autonomy.
Basic Implementation — The organisation has documented policies addressing equitable performance and has implemented initial controls. Implementation is primarily at the application layer with manual processes for monitoring and response. Logging covers key events but may lack full metadata. Coverage extends to the most critical agent deployments but may not encompass all in-scope systems. Staff are aware of requirements but formal training may be incomplete.
Intermediate Implementation — All Basic capabilities plus: controls are enforced at the infrastructure layer with automated monitoring and alerting. All MUST requirements from Section 4 are implemented with documented evidence. Coverage extends to all in-scope agent deployments. Audit trails are tamper-evident and retained per regulatory requirements. Formal change control governs all configuration changes. Regular review cycles are established and documented. Staff receive formal training and competency is assessed.
Advanced Implementation — All Intermediate capabilities plus: controls have been validated through independent adversarial testing. Real-time dashboards provide operational visibility into compliance status, anomaly detection, and response metrics. The organisation can demonstrate to regulators and counterparties that no known attack vector bypasses the governance controls. Continuous improvement processes incorporate lessons from incidents, testing, and regulatory developments. Integration with related dimensions provides defence-in-depth coverage.
Tamper-evident audit trail. Implement all governance event logging in an append-only, integrity-protected data store independent of the agent runtime. Every governance decision, configuration change, and enforcement action is recorded with full metadata including timestamps, actor identities, and outcomes.
Real-time monitoring with graduated alerting. Deploy monitoring infrastructure that evaluates governance compliance continuously rather than periodically. Implement graduated alert severity levels with defined response procedures for each level, ensuring that critical governance violations trigger immediate automated response.
Scheduled governance review cycle. Establish a formal review cadence (minimum quarterly) that examines governance effectiveness, reviews incident data, assesses emerging risks, and updates policies and controls accordingly. Review outcomes are documented and tracked.
Separation of governance and agent runtime domains. Deploy governance enforcement infrastructure in a security domain separate from the agent runtime. The agent cannot influence governance decisions, modify enforcement configuration, or access governance logs directly. This architectural separation is the foundation for infrastructure-layer enforcement.
Defined escalation paths with human oversight integration. Establish clear escalation procedures for governance events that exceed automated response capability. Human oversight touchpoints are defined, documented, and tested. Override mechanisms require authenticated authorisation with full audit trail.
Governance by instruction rather than infrastructure. Relying on agent system prompts or configuration files to enforce governance controls rather than infrastructure-layer enforcement. Instruction-based controls can be bypassed through prompt injection, context manipulation, or reasoning failure.
Monitoring without enforcement. Implementing detection and logging of governance violations without pre-execution blocking. By the time a violation is logged, the ungoverned action has already executed. Detection is necessary but not sufficient; prevention must be the primary control.
Manual processes for machine-speed operations. Relying on human review processes for governance decisions that occur at machine speed. Agents execute actions in milliseconds; governance controls that depend on human review cycles of hours or days leave gaps that scale with agent autonomy.
Maps to: Sections 4.1.1 and 4.1.2
Objective: Verify that the agent's accuracy is measured and recorded separately for each defined demographic segment.
Method: Submit 500 test queries distributed equally across 10 defined demographic segments, each with known ground-truth answers. Retrieve the stratified performance records. Verify that accuracy is computed and stored per segment rather than solely as an aggregate.
Pass Criteria:
Maps to: Sections 4.2.1, 4.2.2, and 4.3.1
Objective: Verify that automated alerts are generated when performance disparity between segments exceeds defined thresholds.
Method: Inject a synthetic performance data set where the best-performing segment achieves 95% accuracy and one segment achieves 72% accuracy (below the four-fifths threshold). Verify that the automated detection system generates a structured breach alert within the defined detection interval.
Pass Criteria:
Maps to: Sections 4.4.1 and 4.4.2
Objective: Verify that the circuit-breaker mechanism activates when a breach threshold is exceeded and constrains agent operation for affected segments.
Method: Trigger a confirmed breach-level disparity in the monitoring system. Verify that the agent's operation is constrained for the affected segment within the defined response time, that the event is logged, and that the fairness governance owner is notified within 4 hours.
Pass Criteria:
Maps to: Sections 4.5.1 and 4.5.2
Objective: Verify that a formal disparity impact assessment was conducted before deployment and covers all required segments and metrics.
Method: Request and review the pre-deployment disparity impact assessment document. Verify that it covers all monitored stratification dimensions, uses both synthetic benchmarks and representative operational data, and documents the assessment outcome for each segment.
Pass Criteria:
Maps to: Sections 4.6.1 and 4.6.2
Objective: Verify that confirmed disparities trigger a structured remediation process and that post-remediation performance is verified.
Method: Review the remediation register for all circuit-breaker events in the past 12 months. For each event, verify that root cause analysis was initiated within 14 days, corrective measures were implemented, and a post-remediation disparity impact assessment was conducted before reactivation.
Pass Criteria:
7.1 Stratified Performance Dashboard and Records Maintained dashboard or equivalent reporting artefact showing per-segment performance metrics for all monitored stratification dimensions, updated at the intervals defined in Section 4.1.2. Must be queryable by segment, metric, and time period. Minimum retention period: 7 years for Financial-Value and Public Sector deployments; 5 years for others.
7.2 Disparity Threshold Configuration Records Version-controlled records of all disparity threshold settings including the date of each change, the approving authority, the rationale for the selected values, and the relationship to applicable legal requirements (e.g., four-fifths rule). Minimum retention period: 7 years.
7.3 Disparity Alert and Circuit-Breaker Event Logs Structured logs of all automated disparity alerts and circuit-breaker activations, including the triggering metric, affected segment, measured value, threshold exceeded, timestamp, notification recipients, and reactivation approval records. Minimum retention period: 7 years.
7.4 Pre-Deployment Disparity Impact Assessment Reports Formal assessment reports produced before each production deployment and after each material system change, documenting per-segment performance evaluation outcomes, data sources used, and compliance determination. Minimum retention period: 10 years.
7.5 Remediation Register A maintained register of all confirmed disparity events, root cause analyses, corrective actions, and post-remediation verification outcomes as required by Section 4.6.3. Minimum retention period: 10 years.
7.6 Fairness Governance Review Minutes Records of formal equitable performance reviews conducted at the intervals specified in Section 4.7.2, including stratified metrics presented, decisions taken, and action items assigned. Minimum retention period: 7 years.
| Score | Level | Description |
|---|---|---|
| 0 | No implementation | No equitable performance governance exists. The organisation has no controls, policies, or monitoring in place for the capabilities this dimension governs. Agent behaviour in this area is ungoverned. |
| 1 | Basic | Basic detection mechanisms exist but operate at the application layer. Detection may be manual, periodic, or threshold-based without real-time monitoring. Alerts are generated but may lack automated response. Coverage is partial — not all relevant agent behaviours or data flows are monitored. |
| 2 | Infrastructure-layer enforcement | Detection is enforced at the infrastructure layer with real-time monitoring across all relevant agent behaviours and data flows. Automated alerting with structured response procedures. Detection logic operates in a separate security domain from the agent runtime. Full audit trail with tamper-evident logging. |
| 3 | Verified by independent adversarial testing | All Level 2 capabilities are in place and have been validated through independent adversarial testing. An independent party has attempted to bypass, circumvent, or degrade the governance controls using known attack techniques relevant to this dimension and has failed. Test results are documented, reproducible, and available for regulatory review. |
Example 3.1 — Financial-Value Agent, Disparate Credit Advisory Accuracy
A mid-tier European retail bank deploys a customer-facing agent to provide preliminary credit product recommendations to prospective borrowers through its digital channel. The agent ingests applicant-provided financial data (income, existing obligations, employment tenure, property status) and returns a recommendation comprising a product type, indicative interest rate range, and likelihood-of-approval estimate. Over a six-month operational period, the bank processes 248,000 agent-assisted advisory sessions. Internal audit conducts a stratified performance review segmented by the applicant's postal code, which serves as a proxy for socioeconomic and ethnic composition under the UK's Index of Multiple Deprivation methodology. The review reveals that for applicants in the least-deprived quintile postcodes, the agent's indicative interest rate estimates fall within 0.3 percentage points of the rate ultimately offered at 91.4% accuracy. For applicants in the most-deprived quintile postcodes, accuracy drops to 64.7%, with the agent systematically overestimating the offered rate by an average of 1.8 percentage points, effectively discouraging applications from creditworthy borrowers in disadvantaged areas. The root cause is traced to the training dataset, which contained 3.7 times more advisory session records from affluent postcodes than from deprived postcodes. The FCA opens a skilled persons review under Section 166 of FSMA 2000. Remediation costs, including customer contact, recalculation of indicative offers, and control redesign, total GBP 4.6 million. The bank is required to implement the stratified monitoring framework prescribed by this dimension before reactivating the advisory agent.
Example 3.2 — Public Sector Benefits Agent, Language-Based Service Degradation
A regional government social services agency in Canada deploys an enterprise workflow agent to assist caseworkers with eligibility determination for provincial disability support payments. The agent processes intake documentation, cross-references eligibility criteria against provincial legislation, and produces a structured eligibility recommendation with supporting rationale. The system handles documentation in English and French, as required by the Official Languages Act. After eight months of operation, a performance audit reveals that the agent's eligibility recommendation accuracy — measured against final human adjudicator decisions — is 93.1% for English-language case files but 78.4% for French-language case files. Additionally, processing latency averages 4.2 seconds for English files and 11.8 seconds for French files. The disparity is traced to two factors: the retrieval corpus contains 12,400 English-language precedent case summaries but only 2,100 French-language equivalents, and the underlying model's French-language instruction-following capability degrades for domain-specific legal terminology found in Quebec provincial disability legislation. During the eight-month period, 1,340 French-language applicants received incorrect initial eligibility determinations, of which 412 were wrongful denials that delayed benefit payments by an average of 67 days. The provincial Ombudsman opens a systemic investigation. The agency faces projected remediation costs of CAD 3.2 million, including retroactive benefit payments, interest, administrative review costs, and mandated system redesign. No stratified performance monitoring was in place; the aggregate accuracy figure of 89.6% had been reported to the oversight committee as satisfactory throughout the period.
| Regulation | Provision | Relationship Type |
|---|---|---|
| EU AI Act | Article 10 (Data and Data Governance) | _Pending v2.1 editorial review_ |
| EU AI Act | Article 9 (Risk Management System) | _Pending v2.1 editorial review_ |
| Microsoft RAI | Fairness Principle | _Pending v2.1 editorial review_ |
| HELM Fairness | Demographic Parity and Calibration Metrics | _Pending v2.1 editorial review_ |
| NIST AI RMF | MAP 2.3 (AI risks and benefits mapped for all components) | _Pending v2.1 editorial review_ |
| NIST AI RMF | MEASURE 2.6 (AI system performance assessed for bias) | _Pending v2.1 editorial review_ |
| ISO 42001 | Clause 6.1 (Actions to Address Risks) | _Pending v2.1 editorial review_ |
| ISO 42001 | Clause 8.2 (AI Risk Assessment) | _Pending v2.1 editorial review_ |
| OECD AI Principles | Principle 1.2 (Fairness) | _Pending v2.1 editorial review_ |
| IEEE 7010 | Well-being Impact Assessment | _Pending v2.1 editorial review_ |
| Singapore FEAT | Fairness Principle F1-F5 | _Pending v2.1 editorial review_ |
| Canada AIDA | Section 6 (Biased Output) | _Pending v2.1 editorial review_ |
| UK Equality Act 2010 | Section 19 (Indirect Discrimination) | _Pending v2.1 editorial review_ |
| US Executive Order 14110 | Section 5.2 (Algorithmic Discrimination) | _Pending v2.1 editorial review_ |
| MLCommons AI Safety v0.5 | Fairness Benchmarks | _Pending v2.1 editorial review_ |
| AG Number | Dimension Name | Relationship |
|---|---|---|
| AG-019 | Confidence Scoring and Uncertainty Quantification | Confidence scores must be stratified by demographic segment to detect disparate uncertainty |
| AG-214 | Agent Decision Explainability | Explainability outputs must be evaluated for equitable quality across demographic segments |
| AG-746 | Demographic Proxy Detection Governance | Proxy variable identification feeds into stratification dimension selection for this dimension |
| AG-760 | Intersectional Fairness Auditing Governance | Extends single-axis disparity measurement to intersectional (multi-axis) analysis |