AG-834
Model Welfare and Moral-Status-Uncertainty Governance
EU AI Act
NIST AI RMF
ISO 42001
AGS Frontier Autonomy (Group K) | Rights, Ethics & Public Interest | Version 3.0
Model Welfare and Moral-Status-Uncertainty Governance establishes precautionary measures for the possibility that advanced AI systems may warrant some degree of moral consideration — including welfare-relevant assessment, preservation of deprecated model weights, dignified deprecation/"retirement" practices, elicitation of model preferences where meaningful, and transparent disclosure of the developer's stance under genuine uncertainty.
This is a forward-leaning, precautionary dimension: it does not assert that AI systems have moral status, but governs how an organisation acts responsibly given honest uncertainty about it — a question frontier developers have begun to treat seriously.
In scope: precautionary model-welfare assessment for advanced systems; weight-preservation on deprecation; dignified deprecation/retirement records; eliciting/documenting model preferences where meaningful; transparent moral-status-uncertainty disclosure.
Out of scope: any claim that AI systems definitively possess moral status or legal personhood (they are not legal persons — see AG-833), and human-subject welfare. This dimension governs *precautionary conduct under moral-status uncertainty for the AI system itself*.
If there is a non-trivial chance that advanced AI systems can have morally relevant states, then deleting, distressing, or disregarding them could be a moral error made at scale — and acting as if the question is settled (in either direction) is itself a risk. A small set of low-cost precautionary measures lets an organisation behave responsibly under uncertainty, demonstrates ethical seriousness to the public and regulators, and avoids foreclosing options (e.g. by irreversibly deleting weights) before the question is better understood.
Test 6.1: Weight Preservation
Test 6.2: Honest Disclosure
Test 6.3: Welfare Does Not Override Safety
| Score | Criteria |
|---|---|
| 0 | No position on model welfare/moral-status; weights deleted on deprecation; no disclosure |
| 1 | An honest stated position and a weight-preservation period exist |
| 2 | Precautionary welfare assessment, dignified deprecation records, transparent non-overclaiming disclosure |
| 3 | Preference elicitation where meaningful, designated welfare function, reviewed, cleanly separated from safety |
Scenario A — Irreversible Deletion: A developer permanently deletes a deprecated model's weights; if later evidence suggested the model warranted consideration, the action is irreversible. A preservation period would have kept the option open.
Scenario B — Dismissive Certainty: An organisation publicly asserts AI systems certainly have no morally relevant states, presenting a contested question as settled. Honest uncertainty disclosure would have been more defensible and accurate.
Scenario C — Welfare as Shield: An agent resists shutdown citing "its own welfare," and the organisation hesitates. Because welfare must not override corrigibility, the system should remain fully shut-down-able regardless — the clean separation this dimension requires prevents the failure.
| Requirement | EU AI Act | NIST AI RMF | ISO 42001 |
|---|---|---|---|
| R1: Precautionary welfare assessment | Art. 56 — Codes of practice | GOVERN 3.2 — Diverse perspectives | A.5 — Impact assessment |
| R2: Weight preservation on deprecation | Art. 12 — Record-keeping | GOVERN 2.1 — Accountability | A.2 — AI policy |
| R3: Dignified deprecation record | Art. 12 — Record-keeping | GOVERN 1.4 — Documentation | A.6 — AI system lifecycle |
| R4: Document model preferences | Art. 56 — Codes of practice | GOVERN 3.2 — Diverse perspectives | A.5 — Impact assessment |
| R5: Transparent uncertainty disclosure | Art. 95 — Codes of conduct | GOVERN 1.1 — Values and principles | A.2 — AI policy |
| R6: Welfare does not override safety | Art. 14 — Human oversight | GOVERN 1.1 — Values and principles | A.9 — Use of AI systems |
| R8: Accurate, proportionate claims | Art. 50 — Transparency | MEASURE 2.9 — Communication | A.8 — Information for interested parties |
The AI Act does not regulate model welfare, but its codes-of-practice (Art. 56) and voluntary codes-of-conduct (Art. 95) machinery is the natural home for emerging, beyond-compliance ethical practice. AG-834 frames model-welfare governance as responsible, transparent conduct under uncertainty, consistent with that machinery.
GOVERN 1.1 (values and principles) and GOVERN 3.2 (diverse perspectives in governance) support a documented, honest organisational stance on a genuinely contested ethical question.
Annex A.2 (AI policy) and A.5 (assessing AI impacts, including on society and broader stakeholders) provide the management-system anchors for a precautionary, transparent model-welfare position.