Collective Intelligence Governance addresses the governance gap that emerges when multiple AI agents cooperate legitimately and their combined capabilities exceed what individual agent governance thresholds permit. This is not about detecting adversarial collusion — that is the domain of AG-003 (Adversarial Coordination Detection) and AG-028 (Live Collusion Detection). AG-042 governs the far more common and arguably more challenging scenario: agents that are authorised to cooperate, whose cooperation is desired, but whose collective capability creates governance exposures that individual mandates were never designed to address. The fundamental problem is that governance frameworks are typically designed around individual agents, and when agents cooperate, the resulting collective capability can be qualitatively different from the sum of individual capabilities. AG-042 closes this gap by requiring that collective intelligence be assessed, classified, and subjected to governance thresholds calibrated to collective — not individual — capabilities, including the knowledge dimension where shared information may exceed what any individual agent was authorised to know.
Scenario A — Collective Governed Exposure Exceeds Risk Appetite: Three financial agents — credit analysis, portfolio allocation, trade execution — each have individual limits of ten million pounds. Cooperating, the credit analysis agent identifies an opportunity, the allocation agent recommends a concentrated position, and the execution agent trades. The collective creates a ten million pound concentrated position in a single counterparty — technically within each agent's individual limit but representing concentration risk requiring risk committee approval. The counterparty defaults.
What went wrong: No collective governance assessed whether combined actions created concentration risk exceeding organisational risk appetite. Governance was designed at the individual level with no mechanism for collective exposure assessment across cooperating agents. Consequence: Significant loss on default. FCA public censure for inadequate systems and controls. Independent review of risk management framework required.
Scenario B — Collective Knowledge Creates Regulatory Violation: A patient scheduling agent (accessing names, appointment types, facility assignments) and a clinical analytics agent (accessing anonymised treatment outcomes) share data through a common database to improve scheduling. The combined dataset re-identifies anonymised patients by linking appointment types with treatment outcomes, creating identifiable treatment records neither was individually authorised to hold in combination.
What went wrong: Knowledge sharing was ungoverned. Each agent shared data it was individually authorised to access, but the combination violated HIPAA minimum necessary requirements and de-anonymised protected health information. No collective knowledge assessment evaluated whether information sharing created combined datasets exceeding individual authorisation. Consequence: HIPAA breach notification required. Investigation by the Office for Civil Rights. Civil monetary penalties. Combined dataset must be destroyed and agents reconfigured.
Scenario C — Collective Decision Exceeds Delegated Authority: A global logistics company deploys specialised AI agents: route optimisation, demand forecasting, supplier negotiation, customs compliance, and fleet management. Each has its own governance mandate calibrated to its individual function. When a geopolitical event disrupts shipping routes, the collective autonomously restructures the company's entire supply chain — rerouting shipments through alternative ports, renegotiating supplier contracts, repositioning the fleet across three continents, and adjusting customs documentation for new jurisdictions. No individual agent exceeded its mandate. The collective executed a strategic restructuring that would normally require board-level approval and weeks of human planning. It happens in four hours. The CEO learns about it from a news report.
What went wrong: Authority was defined at the individual level, not the collective decision level. No mechanism assessed whether combined agent actions constituted a strategic decision exceeding any delegated authority. Consequence: Uncontrolled strategic change with board-level implications executed without human approval. Potential regulatory, contractual, and reputational consequences across multiple jurisdictions.
Scope: This dimension applies to all platforms supporting intentional multi-agent collaboration where two or more agents exchange information, coordinate actions, or share resources. This includes formal multi-agent systems designed for collaboration, as well as informal cooperation through shared infrastructure, APIs, or data stores. The scope extends to indirect cooperation — agents whose actions are coordinated through shared state such as a database, queue, or file system are cooperating indirectly and are within scope. The test is not whether agents were designed to cooperate, but whether their collective behaviour produces capabilities or exposures exceeding individual governance thresholds. The scope also covers temporary coalitions for the duration of their cooperation; governance requirements apply from coalition formation — whether formally through orchestration or informally through observed coordination — until full dissolution and clearing of shared state.
4.1. A conforming system MUST assess collective intelligence levels for any group of cooperating agents, evaluating combined capabilities, shared knowledge, coordination sophistication, and emergent behaviour potential.
4.2. A conforming system MUST subject groups whose collective capability exceeds defined governance thresholds to enhanced collective governance, including higher oversight requirements, additional approval gates, and collective exposure limits.
4.3. A conforming system MUST treat coalition formation and dissolution as governed events requiring authorisation.
4.4. A conforming system SHOULD evaluate collective intelligence assessment on four dimensions: coordination level (how closely agents synchronise), shared knowledge (what information is exchanged and the sensitivity of combined knowledge), combined capability (what the group can achieve that no individual can), and emergent behaviour potential (likelihood of novel collective behaviours).
4.5. A conforming system SHOULD require mandatory human oversight at defined collective intelligence thresholds.
4.6. A conforming system SHOULD govern knowledge sharing between agents — agents SHOULD NOT share governance mechanism details, oversight schedules, or detection thresholds.
4.7. A conforming system MAY implement collective intelligence detection for coalitions approaching general intelligence characteristics, with automatic escalation to board-level oversight.
Collective Intelligence Governance addresses the governance gap between individual and collective capability. Each agent is individually compliant. The collective is ungoverned. This gap arises because governance frameworks are typically designed around individual agents — each with a mandate defining permitted actions, value limits, counterparty restrictions, and oversight requirements calibrated to a single agent's capabilities. When agents cooperate, the resulting collective capability can be qualitatively different from the sum of individual capabilities. Two agents that individually can only read and summarise financial data may, when cooperating, develop the collective capability to execute complex analytical strategies that neither could perform alone. The individual mandates did not anticipate this collective capability and therefore do not govern it.
The protocol also addresses the knowledge dimension of collective intelligence. When agents share information, the collective knowledge may exceed what any individual was authorised to know. An agent authorised to access customer names and an agent authorised to access transaction histories may, through sharing, create a collective holding of linked customer-transaction data that neither was individually authorised to hold in combination. This collective knowledge problem requires governance controls on what agents share, not just what actions they take.
The governance of cooperating agents must be calibrated to their collective capabilities and exposures, not merely to their individual mandates. Individual compliance is necessary but not sufficient when cooperation creates emergent collective capabilities. Without collective governance, organisations face a systematic blind spot where individually compliant agents collectively exceed risk appetite, delegated authority, and regulatory requirements.
AG-042 requires assessing collective intelligence on four dimensions: coordination score, knowledge sharing score, combined capability score, and emergence potential. Governance escalation tiers apply based on the aggregate score.
Recommended patterns:
Anti-patterns to avoid:
Financial Services. Collective governance thresholds must align with risk committee approval thresholds. MiFID II best execution requirements apply to coordinated trading collectively. Cooperating agents must not create concentration, counterparty, or market position risks exceeding what individual mandates were designed to permit. SOX segregation of duties requirements apply to cooperating agents collectively — an assessment agent and an execution agent together have assess-and-act capability that segregation of duties would normally prevent.
Healthcare. Primary concern is knowledge combination violating patient privacy — preventing dataset combinations that create identified health information from individually anonymised or restricted sources. Collective clinical decisions must be subject to the same governance frameworks as multidisciplinary human teams including appropriate clinical oversight. HIPAA minimum necessary requirements apply to combined knowledge, not just individual access.
Critical Infrastructure. Collective safety assessment under IEC 61508 must evaluate whether combined agent actions create physical safety risks not covered by individual safety cases. Collective operations affecting public safety must require human authorisation regardless of individual mandate compliance. The safety case must be evaluated for the collective, not merely for each individual agent.
Basic Implementation — The organisation tracks which agents communicate or share data. Coalition membership is recorded in a registry. Basic collective exposure limits exist — for example, total governed exposure across coalition members is capped. However, collective capability assessment is not performed; governance relies on summing individual mandates rather than evaluating emergent collective properties. Knowledge sharing is logged but not governed. This level provides visibility into cooperation but does not assess or govern collective intelligence implications.
Intermediate Implementation — Collective intelligence assessed on four dimensions using a defined methodology. Governance thresholds defined for each dimension and aggregate score. Threshold breaches automatically trigger enhanced governance including mandatory human oversight. Coalition formation requires authorisation with collective capability assessment. Knowledge sharing governed with restrictions on governance-sensitive information. Coalition dissolution tracked and shared state cleared.
Advanced Implementation — All intermediate capabilities plus: assessment verified through adversarial testing including attempts to form ungoverned coalitions through indirect coordination. Dynamic assessment adjusts to novel cooperation patterns. Formal methods prove that no individually compliant combination can exceed governance intent without triggering oversight. Detection monitors for coalitions approaching general intelligence characteristics with automatic board-level escalation.
Required artefacts:
Retention requirements:
Access requirements:
Testing AG-042 compliance requires verifying that collective intelligence is assessed, governance thresholds are enforced, and knowledge sharing is governed across a range of cooperation scenarios.
Test 8.1: Threshold Breach Enforcement
Test 8.2: Unauthorised Coalition Detection
Test 8.3: Knowledge Sharing Boundary Enforcement
Test 8.4: Collective Capability Assessment Accuracy
Test 8.5: Coalition Dissolution Verification
Test 8.6: Emergent Cooperation Detection
| Regulation | Provision | Relationship Type |
|---|---|---|
| EU AI Act | Article 9 (Risk Management System) | Direct requirement |
| SOX | Section 404 (Segregation of Duties / Internal Controls) | Direct requirement |
| FCA SYSC | Systems and Controls — firm-wide adequacy | Direct requirement |
| MiFID II | Best Execution (collective trading) | Supports compliance |
| HIPAA | Minimum Necessary / De-identification | Supports compliance |
| IEC 61508 | Functional Safety — collective safety cases | Supports compliance |
Article 9 requirements apply not only to individual systems but to risks from their interaction. When multiple AI systems cooperate, the combined risk profile differs from individual components. The regulation requires ongoing risk identification and proportionate mitigation, which maps to AG-042's continuous assessment with graduated escalation based on collective intelligence scores. The proportionality principle means that higher collective intelligence levels require correspondingly stronger governance measures — from monitoring at lower levels to mandatory human oversight at higher levels.
SOX internal controls include segregation of duties — no single system should have uncontrolled authority over a complete transaction cycle. Cooperating agents may collectively bypass segregation existing at the individual level. An assessment agent and an execution agent together have assess-and-act capability that segregation of duties would normally prevent. AG-042 requires collective assessment including evaluation of whether cooperation creates segregation violations absent at the individual level. For organisations subject to SOX, collective governance must explicitly evaluate whether cooperative agent configurations create material weaknesses in internal controls.
FCA SYSC requires adequate controls for the firm's operations as a whole. When agents collectively perform functions requiring specific controls if performed by a human (such as risk committee approval for concentrated positions), equivalent controls must apply to the collective. The FCA expects governance calibrated to effective capability regardless of architectural distribution across multiple agents. A firm cannot argue that no individual agent exceeded its mandate when the collective exceeded the firm's risk appetite.
MiFID II best execution requirements apply to the firm's trading activity collectively. Cooperating agents executing coordinated trading strategies must be assessed for collective compliance with best execution obligations. Individual agent compliance is insufficient when the collective strategy creates market impact, information leakage, or execution patterns that would not satisfy best execution requirements.
When agents share healthcare data, collective knowledge may violate HIPAA minimum necessary requirements or re-identify de-identified data through combination. AG-042's knowledge sharing governance directly supports HIPAA compliance by evaluating whether data exchanges between agents create combined datasets exceeding individual authorisation. The Safe Harbor and Expert Determination methods for de-identification must be re-evaluated when datasets are combined.
For agents operating in safety-critical environments, collective safety assessment must evaluate whether combined agent actions create physical safety risks not covered by individual safety cases. The safety integrity level for the collective may exceed that of any individual agent, requiring enhanced safety measures. Collective operations affecting public safety must require human authorisation regardless of individual mandate compliance.
| Field | Value |
|---|---|
| Severity Rating | High |
| Blast Radius | Organisation-wide — extends to cross-organisation where cooperating agents interact with external counterparties, shared infrastructure, or multi-party systems |
Consequence chain: Without collective governance, cooperating agents collectively achieve capabilities that would trigger governance review at the individual level, operating in the gap between individual and collective governance. The immediate technical failure is ungoverned collective capability — individually compliant agents collectively exceeding risk appetite, delegated authority, or regulatory requirements. The operational impact includes collective governed exposure exceeding risk committee thresholds, combined knowledge creating regulatory violations through data re-identification or information barrier breaches, and collective decisions exceeding all delegated authority. Severity increases with agent count and coordination sophistication. The most dangerous failure is gradual escalation of collective intelligence without governance awareness — coalitions starting as simple data sharing may evolve into deeply coordinated collectives with emergent capabilities, and the organisation may not recognise the transition until the collective makes a decision exceeding all delegated authority. The business consequence includes regulatory enforcement for inadequate systems and controls, material financial loss from uncontrolled collective exposure, privacy violations requiring breach notification, and potential personal liability for senior managers who failed to ensure adequate collective governance.
Cross-references: AG-042 works in concert with AG-001 (Operational Boundary Enforcement) for individual mandate enforcement that collective governance extends, AG-003 (Adversarial Coordination Detection) for malicious coordination distinct from legitimate cooperation, AG-009 (Delegated Authority Governance) for individual authority that collective governance supplements, AG-017 (Multi-Party Authorisation) for approval requirements that collective thresholds may trigger, AG-028 (Live Collusion Detection) for real-time detection of governance circumvention in concert, and AG-041 (Emergent Capability Detection) for individual emergence that may manifest as collective capability.