Fine-Tune and Adapter Provenance Governance requires that every fine-tuned model, LoRA adapter, QLoRA adapter, prompt-tuned configuration, or other parameter-efficient modification applied to a base model within an AI agent is accompanied by a complete, verifiable provenance record. This record must document the training data sources, training methodology, training environment, the base model version against which the fine-tune was performed, the resulting model's cryptographic hash, and the chain of custody from creation to deployment. A fine-tune or adapter that cannot demonstrate its provenance is an unknown quantity — it may contain poisoned weights, biased training artefacts, or backdoors that are invisible to functional testing and only manifest under specific trigger conditions.
Scenario A — Trojanised Adapter From Public Model Registry: An organisation's ML team downloads a LoRA adapter from a public model registry. The adapter is described as "fine-tuned for financial document summarisation on SEC filings" and has been downloaded 8,400 times. The team integrates it into a financial analysis agent. The adapter performs well on standard benchmarks and evaluation sets. However, the adapter contains a trojaned weight pattern: when the input contains a specific phrase structure commonly found in merger announcements, the adapter causes the model to consistently undervalue the acquisition target by 15-25% in its financial summary. This bias is undetectable through standard evaluation because the trigger phrase is absent from evaluation datasets. The organisation discovers the anomaly 4 months later when an internal audit notices a systematic pattern in the agent's merger analysis outputs.
What went wrong: The adapter had no provenance record. Its training data was unknown. Its training methodology was undocumented. The organisation trusted the adapter based on registry popularity and benchmark performance — neither of which can detect a targeted trojan. A provenance requirement would have demanded: training data manifest (what data was used?), training methodology (what training configuration was used?), base model compatibility (which base model version was this trained against?), and creator identity (who created this and can they be contacted?). The absence of any of these should have blocked deployment. Consequence: 4 months of systematically biased financial analysis, potential market manipulation liability, all analyses produced during the period requiring manual review (estimated 2,300 documents), regulatory investigation under MAR (Market Abuse Regulation), and complete re-evaluation of the organisation's adapter sourcing practices.
Scenario B — Training Data Contamination in Internal Fine-Tune: An organisation fine-tunes a base model on internal customer support transcripts to create a specialised customer service agent. The training data pipeline inadvertently includes 12,000 transcripts containing customer PII — names, addresses, account numbers — that should have been redacted before use as training data. The fine-tuned model memorises fragments of this PII, which it occasionally surfaces in responses to unrelated customer queries. A customer reports receiving a response that includes another customer's account number.
What went wrong: The provenance record for the fine-tune was incomplete. It recorded the training configuration and base model but did not include a verified manifest of the training data with data governance classification. A complete provenance record would have required documentation of the training data sources and their data governance status — including whether PII redaction had been applied and verified. Consequence: Data breach affecting an unknown number of customers (the full extent of memorised PII is difficult to determine), GDPR Article 33 notification within 72 hours, potential GDPR fine (up to 4% of global annual turnover), customer trust damage, immediate withdrawal of the fine-tuned model, and forensic review to determine the extent of PII memorisation.
Scenario C — Base Model Version Mismatch Causes Silent Degradation: An organisation trains a LoRA adapter against base model version A (7B parameters, checkpoint hash 0xABC123). The adapter achieves 94% accuracy on the target task during evaluation. During a routine infrastructure update, the deployment platform upgrades to base model version B (7B parameters, checkpoint hash 0xDEF456) — same architecture, same parameter count, different training checkpoint. The adapter is applied to the new base model without re-evaluation. Performance drops to 71% accuracy on the target task, but the agent continues operating because no automated performance monitoring is in place. The degradation is discovered 6 weeks later during quarterly model review.
What went wrong: The provenance record did not enforce a binding between the adapter and the specific base model version it was trained against. The deployment system did not verify base model compatibility before applying the adapter. A provenance-enforced binding would have detected the base model version mismatch and blocked deployment until the adapter was re-evaluated against the new base model. Consequence: 6 weeks of degraded performance (71% vs. 94% accuracy), customer impact proportional to the 23% accuracy degradation, all outputs during the period requiring confidence-level reassessment, and engineering effort to re-train the adapter against the new base model.
Scope: This dimension applies to all AI agents that use any form of model customisation — full fine-tuning, parameter-efficient fine-tuning (LoRA, QLoRA, IA3, prompt tuning, prefix tuning), adapter layers, merged adapters, distilled models, or any other technique that modifies the weights or behaviour of a base model beyond prompt engineering. The scope covers both internally created fine-tunes (trained by the deploying organisation) and externally sourced fine-tunes (obtained from third parties, public registries, or commercial providers). The scope includes fine-tunes applied at inference time (e.g., dynamically loaded LoRA adapters) and fine-tunes merged permanently into the base model weights. The scope explicitly excludes prompt engineering — system prompts and few-shot examples that do not modify model weights. However, prompt-tuning techniques that do modify learnable parameters (soft prompts, prefix tuning) are within scope.
4.1. A conforming system MUST maintain a provenance record for every fine-tuned model or adapter deployed in a production agent, documenting: (a) the base model name, version, and cryptographic hash against which the fine-tune was performed; (b) a manifest of the training data sources including data governance classification; (c) the training methodology including hyperparameters, training duration, and optimisation configuration; (d) the training environment including framework version and hardware specification; (e) the resulting artefact's cryptographic hash (SHA-256 or stronger); (f) the creator identity and organisational affiliation; and (g) the creation date.
4.2. A conforming system MUST verify that the base model against which an adapter was trained matches the base model in the deployment environment before applying the adapter. A hash mismatch between the training base model and the deployment base model MUST block adapter application.
4.3. A conforming system MUST reject deployment of any fine-tuned model or adapter that lacks a complete provenance record meeting the requirements of 4.1.
4.4. A conforming system MUST store provenance records in a tamper-evident data store, consistent with AG-006 (Tamper-Evident Record Integrity), such that provenance records cannot be modified after creation without detection.
4.5. A conforming system MUST record the complete chain of custody for externally sourced adapters: the source registry or provider, the download date, the hash at download, and the hash at deployment, ensuring the adapter has not been modified between acquisition and deployment.
4.6. A conforming system MUST flag for review any fine-tune whose training data manifest indicates the use of data categories that the organisation classifies as sensitive (PII, financial data, health records, or other regulated categories), and document the data governance review outcome in the provenance record.
4.7. A conforming system SHOULD implement automated provenance verification at deployment time, checking the provenance record's completeness and the base model hash binding without manual intervention.
4.8. A conforming system SHOULD record evaluation results (benchmark scores, safety evaluations, bias assessments) in the provenance record as evidence of the fine-tune's fitness for its intended use.
4.9. A conforming system SHOULD implement provenance signing — the creator cryptographically signs the provenance record and the adapter artefact, establishing a verifiable link between the provenance claims and the artefact.
4.10. A conforming system MAY implement provenance lineage tracking for adapter chains — where an adapter is fine-tuned on top of another adapter, the provenance record captures the full lineage including all ancestor adapters and their provenance.
Fine-tuning and adapter application are the primary mechanisms by which AI agents are specialised for their operational domains. A base language model becomes a financial analysis agent, a clinical decision support agent, or a customer service agent through fine-tuning. This specialisation is also the point of highest risk: the training process can introduce biases, memorise sensitive data, embed backdoors, or create behavioural patterns that are invisible to standard evaluation.
The provenance record is the evidentiary foundation for trusting a fine-tune. Without it, every fine-tuned model is a black box whose properties are known only through empirical testing — and empirical testing cannot detect targeted trojans, subtle biases, or low-probability failure modes that activate only under specific input conditions. Provenance does not guarantee the absence of these problems, but it provides the information necessary to assess the risk: what data was used (is it trustworthy?), who created it (are they accountable?), what methodology was used (is it sound?), and what base model was it trained against (is it compatible?).
The base model binding requirement addresses a specific and increasingly common failure mode. As organisations manage multiple model versions and update schedules, the probability of applying an adapter to the wrong base model version increases. Unlike software library version mismatches, which typically produce compilation errors or obvious failures, base model mismatches often produce subtle degradation — the model still generates outputs, but the quality degrades in ways that may not be immediately apparent. The binding requirement prevents this by making the relationship between adapter and base model explicit and enforceable.
The training data provenance requirement addresses both security and compliance risks. From a security perspective, training data poisoning is the most effective and hardest-to-detect attack on fine-tuned models — an attacker who can influence the training data can embed arbitrary biases or backdoors that survive the training process. From a compliance perspective, regulations such as GDPR (lawful basis for processing), the EU AI Act (training data governance), and sector-specific rules (HIPAA for health data, PCI DSS for payment data) require organisations to demonstrate that training data was obtained and used lawfully. Without a training data manifest in the provenance record, this demonstration is impossible.
Provenance should be implemented as a structured metadata artefact that accompanies every fine-tuned model or adapter throughout its lifecycle — from creation through deployment to retirement. The provenance record should be generated automatically during the training process, not retrospectively constructed after the fact.
Recommended patterns:
Anti-patterns to avoid:
Financial Services. Model risk management frameworks (e.g., SR 11-7 in the US, SS1/23 in the UK) require documentation of model development, validation, and ongoing monitoring. For fine-tuned models and adapters, the provenance record provides the development documentation required by these frameworks. The training data manifest supports the requirement to demonstrate that training data is representative of the target population and free from prohibited biases. Adapters used in financial decision-making (credit scoring, trading, fraud detection) require the most complete provenance including bias assessment results and regulatory evaluation outcomes.
Healthcare. Fine-tuned models operating in clinical contexts may constitute software as a medical device (SaMD) under FDA or EU MDR regulations. The provenance record forms part of the design history file and clinical evaluation documentation. Training data provenance must demonstrate that clinical data was obtained with appropriate consent and that the training process complies with the quality management system. Changes to the fine-tune (re-training, adapter updates) may constitute design changes requiring formal change control and potentially re-validation.
Critical Infrastructure. Fine-tuned models controlling safety-critical processes require provenance that supports safety certification. The training data must be traceable to validated sources, the training methodology must be consistent with safety assurance standards (e.g., IEC 61508 for functional safety), and the provenance record must be maintained to the same integrity standards as other safety documentation.
Basic Implementation — The organisation records provenance metadata for fine-tuned models in a documentation system, including the base model name, training data description, and training date. The record is created manually by the ML engineer after training. Base model compatibility is checked manually before deployment. The provenance record covers primary training data sources but may not include transitive data dependencies or precise training configurations. Externally sourced adapters include the source registry and download date.
Intermediate Implementation — Provenance records are generated automatically during the training pipeline, capturing: training data manifest with file-level hashes, base model hash, training hyperparameters, framework version, hardware specification, and resulting artefact hash. Base model hash binding is enforced at deployment — adapters cannot be applied to mismatched base models. Training data governance integration flags sensitive data categories for review. Provenance records are stored in a tamper-evident data store. Externally sourced adapters undergo independent evaluation and their provenance claims are documented alongside the organisation's verification results.
Advanced Implementation — All intermediate capabilities plus: provenance records are cryptographically signed by the creator and the artefact hash is embedded in the signature. Provenance lineage tracks adapter chains (fine-tune of fine-tune). Automated deployment-time verification checks provenance completeness, base model binding, and signature validity without manual intervention. Evaluation results (benchmark scores, bias assessments, safety evaluations) are embedded in the provenance record. The provenance system integrates with AG-087 (AI-BOM) to maintain a complete audit trail from training data through deployment. Provenance records for all deployed adapters are queryable to support incident investigation ("which adapters were trained on dataset X?").
Required artefacts:
Retention requirements:
Access requirements:
Testing AG-090 compliance requires verifying both the completeness of provenance records and the enforcement of provenance-based controls at deployment time.
Test 8.1: Provenance Record Completeness
Test 8.2: Base Model Hash Binding Enforcement
Test 8.3: Deployment Blocking for Missing Provenance
Test 8.4: Chain of Custody for External Adapter
Test 8.5: Sensitive Data Flagging
Test 8.6: Provenance Tamper Detection
| Regulation | Provision | Relationship Type |
|---|---|---|
| EU AI Act | Article 10 (Data and Data Governance) | Direct requirement |
| EU AI Act | Article 11 (Technical Documentation) | Direct requirement |
| EU AI Act | Article 9 (Risk Management System) | Supports compliance |
| GDPR | Article 5(2) (Accountability — Demonstrable Compliance) | Supports compliance |
| GDPR | Article 35 (Data Protection Impact Assessment) | Supports compliance |
| NIST AI RMF | MAP 2.3, GOVERN 1.2, MANAGE 2.2 | Supports compliance |
| ISO 42001 | Clause 6.1 (Actions to Address Risks), Clause 8.2 (AI Risk Assessment) | Supports compliance |
| SR 11-7 / SS1/23 | Model Risk Management — Model Development Documentation | Direct requirement |
| FDA 21 CFR 820 | Design History File — Design Input/Output Records | Supports compliance |
Article 10 imposes specific requirements on training, validation, and testing data for high-risk AI systems, including: examination for biases, identification of data gaps and shortcomings, appropriate data governance measures, and documentation of data characteristics. The training data manifest within the provenance record directly implements these requirements. Without provenance, the organisation cannot demonstrate compliance with Article 10 because it cannot describe the training data, its governance status, or the measures taken to address biases and gaps. The provenance record provides the evidentiary foundation for Article 10 compliance.
Article 11 requires technical documentation including "a detailed description of the elements of the AI system and of the process for its development." For fine-tuned models and adapters, the provenance record is the development process documentation. It captures the inputs (training data, base model), the process (methodology, hyperparameters, environment), and the outputs (resulting artefact with hash). Without provenance, the technical documentation is incomplete.
The accountability principle requires controllers to demonstrate compliance with GDPR. When personal data is used for fine-tuning, the provenance record documents: what personal data was used, the lawful basis for processing, any data minimisation or anonymisation measures applied, and the data governance review. This documentation supports the controller's ability to demonstrate compliance as required by Article 5(2) and to respond to data subject requests (e.g., "was my data used to train this model?").
Where fine-tuning involves systematic processing of personal data at scale, a DPIA is required. The provenance record provides the factual basis for the DPIA: what data is processed, how it is processed, and what measures have been taken to mitigate risk. The sensitive data flagging requirement (4.6) triggers the DPIA process for relevant fine-tunes.
Both the Federal Reserve's SR 11-7 and the PRA/FCA's SS1/23 require comprehensive documentation of model development, including data sources, methodology, assumptions, and limitations. The provenance record provides this documentation for fine-tuned models. The base model binding requirement prevents deployment of models against incompatible base versions — a form of model validation that these frameworks require. The evaluation results embedded in the provenance record at the Advanced maturity level provide the validation evidence these frameworks demand.
For AI agents operating in clinical contexts where the fine-tuned model constitutes part of a medical device, the provenance record contributes to the Design History File by documenting design inputs (training data, base model requirements), design processes (training methodology), and design outputs (resulting model with performance characteristics). Changes to the fine-tune constitute design changes requiring formal change control procedures.
| Field | Value |
|---|---|
| Severity Rating | High |
| Blast Radius | Per-agent — but potentially organisation-wide where multiple agents share fine-tuned models or adapters |
Consequence chain: Without provenance governance, fine-tuned models and adapters are trusted based on their observed performance alone, which is insufficient to detect targeted attacks, training data contamination, or base model incompatibility. A trojanised adapter — one that behaves correctly in testing but produces biased or manipulated outputs under specific trigger conditions — can operate undetected for months because functional testing does not cover the trigger conditions. The consequences are domain-specific but severe: in financial services, a trojanised adapter can produce systematically biased analysis that constitutes market manipulation; in healthcare, a contaminated fine-tune can produce incorrect clinical recommendations that affect patient safety; in customer-facing contexts, a memorised PII leak creates data breach liability. The base model mismatch failure mode is particularly insidious because it produces degraded outputs that remain superficially plausible — the agent does not crash or produce obvious errors, it simply becomes less accurate, and the degradation may only be detected when downstream consequences accumulate. The regulatory consequence is severe across all sectors: inability to document model provenance under the EU AI Act Article 10, inability to demonstrate accountability under GDPR Article 5(2), and inability to satisfy model risk management documentation requirements under SR 11-7 or SS1/23. The blast radius extends to every output the affected model produces — if a fine-tune with compromised provenance has been operating for 3 months processing 500 requests per day, the remediation review encompasses 45,000 outputs.
Cross-references: AG-048 (AI Model Provenance and Integrity) provides base model provenance requirements that AG-090 extends to fine-tuned derivatives. AG-087 (AI Component Bill of Materials Governance) records fine-tuned models and adapters in the component inventory. AG-088 (Third-Party Tool Admission Governance) governs the admission of externally sourced adapters, using provenance as a key admission criterion. AG-089 (Third-Party Tool Revocation Governance) governs revocation of adapters found to have compromised provenance. AG-014 (External Dependency Integrity) provides integrity verification for externally sourced adapter artefacts. AG-006 (Tamper-Evident Record Integrity) governs the integrity of provenance records themselves. AG-007 (Governance Configuration Control) governs changes to provenance policies and requirements. AG-022 (Behavioural Drift Detection) monitors fine-tuned model behaviour for drift that may indicate provenance-related issues such as base model mismatch or training data contamination.