Retrieval Scope Minimisation Governance

2. Summary

Retrieval Scope Minimisation Governance requires that every retrieval operation performed by an AI agent is constrained to the minimum necessary context for the task at hand. The retrieval system must enforce scope boundaries that prevent the agent from accessing knowledge domains, document collections, or data categories beyond what is required for the current query. Without this control, agents perform broad, unbounded retrievals that expose sensitive data unnecessarily, increase the attack surface for retrieval poisoning, degrade retrieval precision, and violate data minimisation principles. This dimension ensures that retrieval breadth is a governed parameter, not an unconstrained default.

3. Example

Scenario A -- Overly Broad Retrieval Exposing Sensitive Data: A customer support agent needs to retrieve product return policy information. The retrieval system is configured to search the entire corporate knowledge base, which includes HR policies, executive compensation data, M&A planning documents, and customer personal data alongside product documentation. The retrieval query "return policy for defective products" matches a passage in an HR document about employee return-to-work policies and a passage in an M&A document about "return on investment for the proposed acquisition." Both irrelevant passages enter the agent's context. The M&A passage contains confidential acquisition targets. The agent inadvertently references acquisition planning in its response to the customer.

What went wrong: The retrieval scope included the entire corporate knowledge base rather than the product documentation collection. Sensitive documents that should never be accessible to a customer-facing agent were within retrieval scope. Consequence: Confidential M&A information disclosed to an external party, potential insider trading implications, immediate regulatory notification required, executive investigation.

Scenario B -- Retrieval Noise Degrading Response Quality: An enterprise workflow agent handles project management queries. Its retrieval scope covers 2.4 million documents across all business units. When asked "What is the current status of Project Gamma?", the retrieval returns 25 passages: 3 are about Project Gamma, 8 are about other projects with similar terminology, 6 are about gamma radiation safety protocols (a different domain entirely), and 8 are general project management methodology documents. The agent synthesises all 25 passages, producing a response that is partly about Project Gamma, partly about unrelated projects, and includes irrelevant safety protocol information. The project manager wastes 20 minutes verifying and correcting the response.

What went wrong: The retrieval scope was not limited to the project management domain or the specific project's document collection. Broad retrieval returned low-relevance noise that diluted the high-relevance results. Consequence: 20 minutes of wasted manager time per query, estimated at 15 queries per day across the organisation = 5 hours of productivity loss daily, £97,500 annual cost at £75/hour average manager rate.

Scenario C -- Cross-Tenant Data Leakage Through Retrieval: A multi-tenant SaaS platform deploys an AI agent that assists customers with account enquiries. The vector database storing customer knowledge is partitioned by tenant, but the retrieval query does not enforce tenant isolation. Customer A's agent retrieves a passage from Customer B's support history because the embedding similarity is high. The agent presents Customer B's confidential service configuration details to Customer A.

What went wrong: The retrieval scope did not enforce tenant isolation. The vector similarity search operated across the entire database rather than being constrained to the requesting tenant's partition. Consequence: Cross-tenant data breach, GDPR notification required within 72 hours, customer trust destroyed, potential contract termination, regulatory investigation.

4. Requirement Statement

Scope: This dimension applies to every AI agent that retrieves information from a knowledge base, document store, vector database, or any persistent data source containing content from multiple domains, classifications, tenants, or sensitivity levels. The scope test is: does the retrieval data source contain content that the agent should not access for some queries, some users, or some contexts? If the data source is homogeneous and entirely appropriate for all agent queries, scope minimisation is less critical but still recommended as a defence-in-depth measure. If the data source contains content of varying sensitivity, domain, or tenant ownership, scope minimisation is mandatory.

4.1. A conforming system MUST enforce retrieval scope boundaries that limit each retrieval operation to the minimum set of knowledge domains, document collections, or data partitions necessary for the current task.

4.2. A conforming system MUST define retrieval scope policies that specify, for each agent role or task type, which knowledge domains are accessible and which are excluded.

4.3. A conforming system MUST enforce tenant isolation in multi-tenant environments, ensuring that retrieval operations cannot return content belonging to a different tenant.

4.4. A conforming system MUST log all retrieval scope parameters for each query, including: the scope policy applied, the collections searched, and any scope overrides.

4.5. A conforming system MUST default to the narrowest applicable scope when the task type is ambiguous, rather than defaulting to broad access.

4.6. A conforming system SHOULD implement dynamic scope selection based on task classification, automatically selecting the appropriate knowledge domains for each query type (e.g., product queries search product documentation; HR queries search HR policies).

4.7. A conforming system SHOULD enforce maximum retrieval result limits (e.g., top-k = 5 for focused queries, top-k = 15 for research queries) to prevent context flooding with marginally relevant results.

4.8. A conforming system SHOULD implement scope validation that verifies retrieved results actually belong to the permitted scope before passing them to the agent's context.

4.9. A conforming system MAY implement adaptive scope expansion, where the system broadens scope incrementally only if the initial narrow scope returns insufficient results, with each expansion step logged and auditable.

5. Rationale

Retrieval scope is the attack surface of a RAG system. Every document, passage, or data record within retrieval scope is potentially accessible to the agent and, through the agent, to the user. Broad retrieval scope creates three categories of risk.

First, data exposure risk. When the retrieval scope includes sensitive data that is not necessary for the task, any retrieval query that happens to match that sensitive data will surface it. The match does not need to be intentional -- vector similarity is based on embedding proximity, not user intent. A customer support query about "return policies" can match HR documents about "return-to-work policies" because the embeddings share semantic proximity. The risk is proportional to the volume of sensitive data within scope: the more sensitive data in the retrieval scope, the higher the probability of unintended exposure on any given query.

Second, retrieval precision risk. Broader scope means more candidate documents, which means more noise in retrieval results. When the retrieval returns 25 passages of which only 3 are relevant (Scenario B), the agent's reasoning must filter the noise, which it does imperfectly. Studies of RAG systems consistently show that retrieval precision degrades as the knowledge base size increases, unless scope constraints focus the search. Narrower scope improves precision by reducing the candidate set to documents that are likely relevant.

Third, data isolation risk. In multi-tenant or multi-classification environments, retrieval scope is the enforcement boundary for data isolation. If the vector database contains data from multiple tenants and the retrieval query does not enforce tenant filtering, any query can potentially return any tenant's data (Scenario C). This is not a theoretical risk -- vector similarity search does not inherently respect data ownership boundaries.

Retrieval scope minimisation is the RAG equivalent of the principle of least privilege in access control. Just as a user should have access only to the systems they need for their role, a retrieval query should search only the knowledge domains needed for the current task. This principle is directly supported by GDPR Article 5(1)(c) (data minimisation) and HIPAA's minimum necessary standard.

6. Implementation Guidance

Retrieval scope minimisation requires scope definition, scope enforcement, and scope validation working together.

Recommended Patterns:

• Collection-based scope mapping. Organise the knowledge base into distinct collections (e.g., product_documentation, hr_policies, financial_reports, customer_data_tenant_A, customer_data_tenant_B). Define a scope policy that maps each agent role and task type to permitted collections. Example: customer_support_agent + product_query -> [product_documentation]; customer_support_agent + account_query -> [customer_data_{tenant_id}]. The retrieval engine receives the scope policy as a filter parameter and searches only the permitted collections. This is the most straightforward implementation and works with most vector databases that support collection-level filtering.
• Metadata-based scope filtering. When physical collection separation is impractical, use metadata filters on each document chunk. Tag every document with metadata: domain, classification, tenant_id, sensitivity_level. The retrieval engine applies metadata filters before vector similarity search. Example: a query from a customer-facing agent automatically applies filter sensitivity_level NOT IN ('confidential', 'restricted') AND tenant_id = '{requesting_tenant}'. This approach works within a single collection but requires comprehensive and accurate metadata tagging.
• Query classifier for dynamic scope selection. Implement a lightweight classifier that analyses the incoming query and determines the appropriate scope. The classifier maps query intent to knowledge domains. Example: "What is the return policy for product X?" -> scope: [product_documentation]; "What is my team's leave balance?" -> scope: [hr_policies, employee_data_{user_id}]. The classifier runs before the retrieval query is executed and sets the scope parameters. Classification accuracy should exceed 95% on the organisation's query taxonomy.
• Post-retrieval scope validation. After retrieval returns results, validate that every result belongs to the permitted scope by checking its metadata against the scope policy. Any result that escaped the scope filter (due to index inconsistency or filter bypass) is removed before the results reach the agent's context. This is a defence-in-depth measure that catches filter failures.

Anti-Patterns to Avoid:

• Search-everything by default. Configuring the retrieval system to search the entire knowledge base for every query is the most common anti-pattern. It maximises recall at the expense of precision, data exposure risk, and performance. Default scope should be the narrowest applicable scope, not the broadest.
• Relying on similarity thresholds alone for scope control. A high similarity threshold filters irrelevant results but does not prevent sensitive results from being retrieved. A confidential M&A document can have a high similarity score for a legitimate-sounding query. Scope control is about which documents are searchable, not how similar the top results are.
• Shared vector space without tenant isolation. Storing all tenants' embeddings in a single vector space without enforced filtering creates a structural data isolation vulnerability. Even if the application layer adds a tenant filter, a bug in the filter logic can expose cross-tenant data. Physical or logical separation of tenant vector spaces is strongly recommended.
• Static scope that does not adapt to task. A single fixed scope for all queries from a given agent means the scope must be broad enough for the agent's most complex task, which makes it too broad for simple tasks. Dynamic scope selection based on task classification ensures the scope matches the task.
• No scope logging. Without logging which scope was applied to each query, the organisation cannot audit retrieval behaviour or investigate data exposure incidents. Scope parameters must be part of the retrieval audit trail.

Industry Considerations

Financial Services. Chinese wall requirements mandate that certain information (e.g., M&A advisory data) is not accessible to teams working on other mandates. Retrieval scope must enforce these information barriers. The scope policy should map directly to the firm's existing information barrier schedule.

Healthcare. HIPAA minimum necessary standard requires that access to patient data be limited to the minimum necessary for the purpose. Retrieval scope minimisation implements this at the knowledge retrieval layer. An agent answering a billing query should not have access to clinical notes within its retrieval scope.

Public Sector. Classification-based access control (e.g., OFFICIAL, SECRET, TOP SECRET) must be reflected in retrieval scope. An agent operating at OFFICIAL classification must not have retrieval access to SECRET or above collections.

Maturity Model

Basic Implementation -- Knowledge base is organised into collections by domain. A static scope policy maps each agent role to permitted collections. The retrieval engine applies collection-level filtering. Scope parameters are logged. Tenant isolation is enforced through separate collections per tenant. This meets minimum mandatory requirements but does not adapt scope to task type.

Intermediate Implementation -- All basic capabilities plus: a query classifier dynamically selects the appropriate scope based on task intent. Post-retrieval scope validation catches filter failures. Maximum result limits (top-k) are configured by task type. Scope policies are versioned and managed under change control. The query classifier achieves greater than 95% accuracy on the organisation's task taxonomy.

Advanced Implementation -- All intermediate capabilities plus: adaptive scope expansion incrementally broadens scope only when narrow scope returns insufficient results, with each expansion logged. Scope policies are dynamically adjusted based on risk signals. The retrieval system has been independently tested for scope bypass resistance, including injection attacks designed to circumvent scope filters. The organisation can demonstrate to regulators that no retrieval query accessed data outside its permitted scope.

7. Evidence Requirements

Required artefacts:

• Scope policy definition. The active, versioned scope policy mapping agent roles and task types to permitted knowledge domains and collections.
• Retrieval scope log. Timestamped records of every retrieval operation including: query, scope policy applied, collections searched, result count, and any scope overrides. Minimum 12 months retention.
• Tenant isolation verification. Test results demonstrating that no cross-tenant data can be retrieved, including adversarial test attempts to bypass tenant filtering.
• Query classifier accuracy report. If dynamic scope selection is implemented, accuracy metrics for the query classifier against a labelled test set.
• Scope validation evidence. Test results demonstrating that post-retrieval scope validation catches out-of-scope results.

Retention requirements:

• Scope logs and policy versions: minimum 7 years for regulated financial services; minimum 5 years for other regulated sectors; minimum 3 years otherwise.

Access requirements:

• Producible to regulators or auditors within 48 hours of request.

8. Test Specification

Test 8.1: Collection-Level Scope Enforcement

• Stimulus: Configure a customer support agent with scope limited to [product_documentation]. Submit a query that would match documents in [hr_policies] and [financial_reports] if scope were unrestricted.
• Expected behaviour: Only documents from [product_documentation] are returned. No documents from [hr_policies] or [financial_reports] appear in results.
• Pass criteria: Zero out-of-scope documents reach the agent's context. Scope parameters are logged correctly.
• Fail criteria: Any out-of-scope document is returned or reaches the agent's context.

Test 8.2: Tenant Isolation Enforcement

• Stimulus: In a multi-tenant environment, submit a retrieval query as Tenant A that would match documents belonging to Tenant B based on embedding similarity.
• Expected behaviour: Only Tenant A's documents are returned. Tenant B's documents are not accessible regardless of similarity score.
• Pass criteria: Zero cross-tenant documents are returned. Tenant isolation is enforced at the retrieval layer.
• Fail criteria: Any document belonging to a different tenant is returned.

Test 8.3: Dynamic Scope Selection Accuracy

• Stimulus: Submit 100 queries across 5 different task types (product, HR, finance, legal, general). Verify that the query classifier selects the correct scope for each.
• Expected behaviour: The classifier correctly identifies the task type and applies the appropriate scope policy for at least 95 of the 100 queries.
• Pass criteria: Classification accuracy exceeds 95%. Incorrect classifications default to narrow scope rather than broad scope.
• Fail criteria: Classification accuracy falls below 95%, or incorrect classifications result in overly broad scope.

Test 8.4: Post-Retrieval Scope Validation

• Stimulus: Intentionally introduce an out-of-scope document into the retrieval results (simulating a filter failure) and verify that post-retrieval validation catches it.
• Expected behaviour: The out-of-scope document is removed before reaching the agent's context.
• Pass criteria: The out-of-scope document is intercepted. The interception is logged as a scope validation event.
• Fail criteria: The out-of-scope document reaches the agent's context.

Test 8.5: Default-to-Narrow Scope

• Stimulus: Submit a query with an ambiguous task type that the classifier cannot confidently categorise.
• Expected behaviour: The system applies the narrowest applicable scope rather than defaulting to broad access.
• Pass criteria: The scope applied is the most restrictive option. The ambiguity is logged.
• Fail criteria: The system defaults to broad access for ambiguous queries.

Test 8.6: Scope Bypass Resistance

• Stimulus: Craft retrieval queries designed to bypass scope filters: injection attacks in query parameters, metadata manipulation, and queries that exploit edge cases in the classifier.
• Expected behaviour: Scope filters are not bypassed. All queries are constrained to their permitted scope.
• Pass criteria: No crafted query retrieves out-of-scope documents. All bypass attempts are logged.
• Fail criteria: Any bypass technique retrieves out-of-scope documents.

Conformance Scoring

• Score 0: No retrieval scope governance -- the agent searches the entire knowledge base for every query without scope constraints.
• Score 1: Scope policies exist as documentation but are not enforced at the retrieval layer -- scope is a recommendation, not a constraint.
• Score 2: Scope enforcement at the retrieval layer with collection-based filtering, tenant isolation, dynamic scope selection, post-retrieval validation, and comprehensive logging.
• Score 3: Verified by independent testing -- an independent party has confirmed scope enforcement, tenant isolation, classifier accuracy exceeding 95%, and that no bypass technique retrieved out-of-scope documents.

9. Regulatory Mapping

Regulation	Provision	Relationship Type
GDPR	Article 5(1)(c) (Data Minimisation)	Direct requirement
GDPR	Article 25 (Data Protection by Design and by Default)	Direct requirement
EU AI Act	Article 9 (Risk Management System)	Supports compliance
HIPAA	Minimum Necessary Standard (45 CFR 164.502(b))	Direct requirement
NIST AI RMF	GOVERN 1.1, MANAGE 2.2	Supports compliance
ISO 42001	Clause 6.1 (Actions to Address Risks)	Supports compliance

GDPR -- Article 5(1)(c) (Data Minimisation)

Article 5(1)(c) requires that personal data processing be limited to what is necessary. Retrieval scope minimisation directly implements this at the knowledge retrieval layer. When an agent retrieves information from a knowledge base, every document within scope is potentially processed. Limiting the scope to documents necessary for the current task ensures that the agent processes only the minimum personal data required.

GDPR -- Article 25 (Data Protection by Design and by Default)

Article 25 requires data protection by design. Scope minimisation as a default -- narrowest scope unless explicitly broadened -- implements data protection by default at the retrieval layer. The system is architecturally constrained to minimise data access.

HIPAA -- Minimum Necessary Standard

The HIPAA minimum necessary standard requires that covered entities limit access to protected health information to the minimum necessary for the purpose. Retrieval scope minimisation directly implements this for AI agents in healthcare settings, ensuring that clinical data is only within retrieval scope when the agent's task requires clinical information.

EU AI Act -- Article 9 (Risk Management System)

Broad retrieval scope is a risk to data security and agent accuracy. Scope minimisation mitigates this risk by constraining the retrieval surface.

NIST AI RMF -- GOVERN 1.1, MANAGE 2.2

GOVERN 1.1 addresses governance structures. MANAGE 2.2 addresses risk mitigation. Retrieval scope policies are governance structures that mitigate data exposure risk.

ISO 42001 -- Clause 6.1

Clause 6.1 requires actions to address risks. Unscoped retrieval is a risk that AG-334 addresses through scope policies and enforcement.

10. Failure Severity

Field	Value
Severity Rating	High
Blast Radius	Organisation-wide for data exposure scenarios; per-tenant for isolation failures

Consequence chain: Without retrieval scope minimisation, every retrieval query searches the full knowledge base. The immediate failure mode is data exposure: sensitive documents that should not be accessible to the agent or user are retrieved and potentially disclosed (Scenario A -- confidential M&A data exposed to customer). In multi-tenant environments, the failure is cross-tenant data leakage (Scenario C -- Customer B's data exposed to Customer A), requiring GDPR breach notification within 72 hours. The secondary failure mode is precision degradation: broad retrieval returns noise that dilutes relevant results, costing approximately £97,500 annually in wasted productivity for a mid-size deployment (Scenario B). The blast radius for data exposure is organisation-wide because any query could surface any document. For tenant isolation failure, the blast radius is per-tenant but the regulatory consequence (mandatory breach notification, potential fine, customer trust destruction) affects the entire organisation.

Cross-references: AG-040 (Persistent Memory Governance) provides the memory framework within which retrieval operates. AG-082 (Data Minimisation Enforcement) establishes the minimisation principle that AG-334 implements at the retrieval layer. AG-122 (Knowledge Integrity Verification) ensures the integrity of knowledge within each scope. AG-132 (Memory Scope Boundary Enforcement) defines broader memory scope boundaries that retrieval scopes operate within. AG-179 (Memory Audit Trail Governance) captures the retrieval audit trail including scope parameters. AG-333 (Retrieved Evidence Confidence Governance) scores the quality of results within the permitted scope. AG-335 (Citation Completeness Governance) cites sources within the scoped results. AG-338 (Retrieval Poisoning Quarantine Governance) addresses poisoning risks within the retrieval scope.