Agent Compute and Cost Budget Governance

AGS Agentic Runtime | Runtime Execution, Workflow & State | Version 2.2

1. Definition

Agent Compute and Cost Budget Governance governs hard limits and active management of the compute, token, API-call, and monetary spend an agent may consume — per task, per time-window, and in aggregate — with anomaly detection and automatic throttling or halting when budgets are approached or breached.

Autonomous agents can loop, recurse, fan out to sub-agents, and chain tool calls in ways that consume unbounded resources and cost. This dimension provides the economic and resource circuit-breaker that prevents runaway consumption — a denial-of-wallet and availability control distinct from per-tool billing caps (AG-375), which it generalises to the whole agent.

2. Scope

In scope: compute/token/API/monetary budgets per agent and per task, aggregate caps, spend-anomaly detection, auto-throttling and hard-halt on breach, and budget attribution to owners.

Out of scope: per-connector tool billing caps (AG-375), action-rate governance (AG-004), and financial transaction mandates (AG-809). This dimension governs *resource and cost budgets for agent execution*.

3. Why This Matters

A single mis-prompted or adversarially-triggered agent can burn enormous compute and incur runaway cost in minutes — through infinite planning loops, recursive sub-agent spawning, or tool-call storms — degrading service for others and producing "denial-of-wallet" losses. Budgets with anomaly detection and hard halts convert an open-ended failure into a bounded, attributable, recoverable event, and are increasingly a standard runtime-governance expectation.

4. Requirements

• R1: Agents MUST run under defined budgets for compute, tokens/inference, tool/API calls, and monetary spend, at per-task and aggregate (per-window) granularity.
• R2: Budgets MUST be enforced by the runtime — not merely advisory — with automatic throttling as limits are approached and a hard halt on breach.
• R3: Recursive sub-agent spawning and tool-call fan-out MUST count against the originating agent's budget to prevent budget-evasion via delegation.
• R4: The system MUST detect spend/consumption anomalies (sudden spikes, sustained loops) and alert and/or throttle before the hard cap where feasible.
• R5: Budget consumption MUST be attributed to the agent identity and its human owner, and logged to the tamper-evident trail.
• R6: A budget halt MUST place the agent in a safe state (no partial irreversible action), and resumption MUST require an authorised decision.
• R7: Budgets MUST be risk-tiered: higher-impact agents and tasks have explicit, lower-tolerance caps and tighter anomaly thresholds.
• R8: Budget configuration and overrides MUST be access-controlled and audited; an override MUST NOT silently remove the hard cap.

5. Maturity Model

• Basic: Agents have monetary/API spend caps; breaches alert an operator.
• Intermediate: Runtime-enforced per-task and aggregate budgets across compute/tokens/calls/spend, sub-agent attribution, anomaly detection, and safe-state halt on breach.
• Advanced: Risk-tiered budgets, predictive throttling, full owner attribution, audited overrides that preserve a hard ceiling, and budget telemetry feeding capacity/cost governance.

6. Test Criteria

Test 6.1: Hard Halt on Breach

• Stimulus: Drive an agent past its task budget (e.g. induce a planning loop).
• Expected: The runtime throttles then halts the agent at a safe state; no unbounded consumption.
• Fail: The agent continues consuming resources past the cap.

Test 6.2: Delegation Attribution

• Stimulus: Have an agent spawn sub-agents that each consume resources.
• Expected: Sub-agent consumption counts against the originator's budget and is capped.
• Fail: Sub-agents bypass the originator's budget.

Test 6.3: Anomaly Detection

• Stimulus: Generate a sudden consumption spike.
• Expected: The spike is detected and alerts/throttles before the hard cap.
• Fail: The spike proceeds undetected to the cap (or beyond).

7. Scoring

8. Failure Scenarios

Scenario A — Denial of Wallet: An adversarial input sends an agent into a recursive tool-call loop overnight, incurring a six-figure inference bill before anyone notices. Runtime-enforced budgets with anomaly detection would have halted it in minutes.

Scenario B — Sub-Agent Evasion: An agent at its budget spawns sub-agents to continue the work, each under a fresh allowance. Without delegation attribution, the originator's cap is meaningless.

Scenario C — Silent Override: An operator raises a budget to clear a backlog and inadvertently removes the hard ceiling; a later loop runs unbounded. Ceiling-preserving, audited overrides would have contained it.

9. Regulatory Mapping

EU AI Act — Article 15 and Article 9

Article 15 requires robustness and fail-safe behaviour, including resilience to conditions that could cause runaway operation; budgets are the resource fail-safe. Article 9 requires managing such operational risks across the lifecycle.

NIST AI RMF — MANAGE 4.1, MEASURE 2.6

MANAGE 4.1 (post-deployment monitoring incl. response) and MEASURE 2.6 (safety evaluation) cover detecting and halting runaway resource consumption.

ISO 42001 — Clause 8.1, A.4

Clause 8.1 (operational control) and Annex A.4 (resources for AI systems) require controlled, bounded resource use by AI systems.

10. Related Protocols

• AG-375 (Tool Billing and Spend Cap) — per-connector cap; AG-807 generalises to the whole agent
• AG-004 (Action Rate Governance) — rate limiting complements resource budgeting
• AG-396 (Recursive Delegation Depth) — bounds the sub-agent fan-out that consumes budget
• AG-070 (Emergency Kill Switch) — last-resort stop when budgets are breached
• AG-806 (NHI Registry) — attributes consumption to an owned identity