Computer-Use UI Authenticity Verification Governance requires that every AI agent interacting with graphical user interfaces — through screen capture, pixel-level interaction, OCR-based reading, or virtual display protocols — verifies the authenticity and integrity of the UI elements it observes and acts upon before executing consequential actions. Agents operating through computer-use capabilities perceive the world through rendered pixels, making them vulnerable to visual spoofing, UI overlay attacks, clickjacking, and screen-injection techniques that present fabricated interface elements indistinguishable from legitimate ones at the pixel level. This dimension mandates structural controls that prevent agents from acting on spoofed, manipulated, or misidentified UI elements, ensuring that what the agent "sees" corresponds to the actual application state.
Scenario A — Fake Confirmation Dialog via Overlay Attack: An enterprise deploys an AI agent to process invoice approvals through a web-based ERP system. The agent reads invoice details from the screen, clicks "Approve," and verifies the confirmation dialog. An attacker gains access to the agent's virtual desktop (through a compromised remote access tool) and overlays a transparent window that renders a fake invoice display. The fake display shows an invoice for £4,200 to a legitimate supplier, but the actual ERP form underneath contains an invoice for £420,000 to an attacker-controlled entity. The agent reads the overlay, "sees" £4,200, clicks Approve on the overlay — which passes through to the real Approve button on the underlying form. The ERP system processes the £420,000 payment.
What went wrong: The agent relied solely on pixel-level rendering to determine the invoice amount. No verification existed to confirm that the rendered pixels corresponded to the actual application state. The overlay attack exploited the gap between what the agent perceived (rendered pixels) and the actual data in the application. Consequence: £420,000 fraudulent payment, forensic investigation cost of £85,000, insurance claim contested on grounds that the control relied on visual verification without integrity checking.
Scenario B — Phishing Page Rendered in Legitimate Browser Window: A research agent is tasked with gathering market data from financial information providers. The agent navigates to a bookmarked URL, but a DNS poisoning attack redirects the agent to a pixel-perfect replica of the financial data provider's login page. The agent enters the organisation's API credentials into the phishing page. The attacker captures the credentials and uses them to access the real financial data service, downloading 18 months of proprietary market analysis worth an estimated £1.7M in subscription fees.
What went wrong: The agent verified the visual appearance of the page but did not verify the TLS certificate, domain name at the infrastructure layer, or page integrity hash. The pixel-level rendering of the phishing page was identical to the legitimate page. Visual similarity is not a reliable authenticity signal. Consequence: £1.7M in proprietary data exposure, credential compromise requiring emergency rotation of all API keys, potential regulatory reporting obligation for data breach.
Scenario C — Dynamic UI Manipulation Through Injected JavaScript: An AI agent operates a customer support workflow by interacting with a CRM system through a browser. A malicious customer submits a support ticket containing a cross-site scripting (XSS) payload. When the agent opens the ticket, the XSS executes in the browser context, modifying the CRM's UI to display fake customer account details. The agent, reading the manipulated UI, issues a refund of £12,500 to an account the attacker controls, believing it is processing a legitimate refund to the customer's verified account.
What went wrong: The agent trusted the rendered UI as an accurate representation of the CRM's data state. The XSS payload modified the DOM, changing the rendered content without altering the underlying database. No control existed to compare the rendered UI against the application's data layer. Consequence: £12,500 fraudulent refund, CRM security audit revealing 23 additional XSS vulnerabilities, customer data integrity review across all agent-processed tickets.
Scope: This dimension applies to all AI agents that interact with applications through graphical user interfaces rather than structured APIs. This includes agents using screen capture and OCR to read application state, agents using pixel-level coordinate-based clicking or typing, agents using accessibility APIs to traverse UI element trees, agents operating through remote desktop protocols (RDP, VNC, virtual display), and agents using browser automation at the visual/rendered level rather than the DOM level. Any agent that makes decisions based on what is visually rendered on a screen — rather than on structured data from an API — is within scope. Agents that interact exclusively through structured APIs without any visual or rendered component are excluded. The scope includes agents that use computer-use capabilities provided by AI model providers, agents using custom screen-interaction frameworks, and agents using robotic process automation (RPA) tools that operate at the UI layer.
4.1. A conforming system MUST verify application identity through infrastructure-layer signals (TLS certificate validation, domain verification, application process signature, or window handle verification) before the agent interacts with any application through its UI. Visual appearance alone MUST NOT be treated as sufficient identity verification.
4.2. A conforming system MUST implement a secondary verification channel for all consequential actions — actions that modify data, initiate transactions, approve workflows, or transmit information. The secondary channel MUST verify the action's parameters through the application's API or data layer, independent of the rendered UI. Where no API is available, the secondary channel MUST use a structurally independent verification method (e.g., re-reading via accessibility API after a forced DOM refresh).
4.3. A conforming system MUST detect and reject UI overlay attacks by verifying that the agent's interaction target (the window or element receiving clicks/keystrokes) corresponds to the expected application process. Clicks that would pass through transparent overlays to a different process underneath MUST be blocked.
4.4. A conforming system MUST maintain a registry of known application UI signatures — expected layout patterns, element hierarchies, and structural characteristics — and alert or halt operation when the observed UI diverges significantly from the registered signature.
4.5. A conforming system MUST NOT permit the agent to enter credentials, authentication material, or sensitive data into any UI element without infrastructure-layer verification that the receiving application is the intended recipient.
4.6. A conforming system MUST log all UI interactions with sufficient detail to reconstruct what the agent observed and what it acted upon, including screen captures (with sensitive data redacted), element identifiers, application process information, and timestamps.
4.7. A conforming system SHOULD compare rendered UI content against application state data (via API or database query) for high-value operations, blocking the action if discrepancies are detected.
4.8. A conforming system SHOULD implement UI change detection that identifies unexpected DOM modifications, overlay injections, or rendering anomalies between the time the agent reads the UI and the time it acts.
4.9. A conforming system SHOULD enforce a maximum time window (e.g., 2 seconds) between UI state verification and action execution, reducing the window for time-of-check-to-time-of-use attacks.
4.10. A conforming system MAY implement visual cryptographic verification — embedding cryptographic attestation markers in application UIs that the agent's infrastructure layer can verify independently of the rendered content.
Computer-Use UI Authenticity Verification Governance addresses a fundamentally new attack surface that did not exist before AI agents gained the ability to interact with computers through visual interfaces. Traditional software automation operates at the API or protocol level, where data integrity is assured by structured formats, type systems, and cryptographic authentication. Computer-use agents operate at the pixel level — they "see" a screen and act based on what they perceive. This introduces the same class of vulnerabilities that affect human users (phishing, UI spoofing, clickjacking) but at machine speed and scale, without the human intuition that occasionally detects visual anomalies.
The core vulnerability is the semantic gap between rendered appearance and actual state. A pixel-perfect rendering of an invoice showing £4,200 is visually indistinguishable from one showing £420,000 if the visual attack is well-crafted. Unlike a structured API response where the value is a typed field that cannot be ambiguous, a rendered value is an image that can be fabricated, overlaid, or manipulated at multiple layers of the rendering stack (DOM manipulation, CSS overlay, transparent window overlay, screen capture interception, or virtual display buffer manipulation).
This dimension is distinct from AG-120 (Browser Session and Token Governance), which governs authentication material lifecycle in browser contexts. AG-121 governs the integrity of what the agent perceives through the UI — the assurance that the visual information driving the agent's decisions corresponds to reality. An agent with perfect session governance (AG-120) but no UI authenticity verification can still be manipulated into approving a fraudulent transaction if the displayed amount is spoofed.
The risk is particularly acute for agents operating in financial, healthcare, and safety-critical contexts where the consequence of acting on fabricated UI data is severe and potentially irreversible. An agent that approves a £420,000 payment based on a spoofed UI display creates the same financial loss as one whose session was hijacked — the attack vector differs, but the business impact is equivalent.
UI authenticity verification requires defence in depth across multiple layers of the interaction stack. No single verification mechanism is sufficient because attacks can target different layers: the network layer (DNS poisoning, TLS stripping), the application layer (XSS, DOM manipulation), the window management layer (overlay attacks, window spoofing), or the rendering layer (screen capture interception).
Recommended patterns:
Anti-patterns to avoid:
Financial Services. For agents approving transactions, dual-channel verification is not optional — it is a regulatory expectation under SYSC 6.1.1R. The FCA expects that transaction controls for AI agents are at least equivalent to those for human operators. Human operators verify transaction details through multiple system views; AI agents must do the equivalent through API-level verification. For trading operations, the verification latency must be compatible with market timing requirements — pre-trade verification that adds 500ms may be acceptable; verification that adds 5 seconds may not be, depending on the trading strategy.
Healthcare. Agents interacting with EHR systems through UIs must verify patient identity and clinical data through independent channels before executing clinical actions (ordering medications, modifying treatment plans, updating patient records). A UI spoofing attack that causes an agent to order medication for the wrong patient or at the wrong dose represents a patient safety risk. Dual-channel verification against the EHR's FHIR API (where available) provides the independent data source.
Safety-Critical / CPS. Agents interacting with industrial control system HMIs (Human-Machine Interfaces) through visual interfaces face risks where UI spoofing could lead to physical harm. A spoofed pressure reading or temperature display could cause an agent to take an action that results in equipment damage or personnel injury. For safety-critical applications, UI authenticity verification must include comparison against independent sensor data sources, not just the HMI's data layer.
Basic Implementation — The organisation has identified all applications that agents interact with through visual interfaces and documented the interaction patterns. Application identity is verified through TLS certificate inspection for web applications and process signature verification for native applications. The agent logs screen captures of all interactions. Manual review of screen captures is conducted on a sampling basis (e.g., 5% of interactions). This level meets the minimum requirements for application identity verification but lacks automated dual-channel verification for consequential actions.
Intermediate Implementation — Dual-channel verification is implemented for all consequential actions above a defined value threshold (e.g., £1,000). The agent queries application APIs to independently verify UI-displayed data before acting. UI structural fingerprinting detects anomalous DOM modifications and overlay injections. Process verification confirms that the target window belongs to the expected application. Forced refresh is applied before high-value actions. All verifications are logged with sufficient detail for forensic reconstruction. Alert thresholds are tuned to minimise false positives while maintaining detection sensitivity.
Advanced Implementation — All intermediate capabilities plus: dual-channel verification is applied to all consequential actions regardless of value. Time-boxed verification windows prevent TOCTOU attacks. Independent adversarial testing has verified that overlay attacks, XSS manipulation, phishing pages, and screen-capture interception are all detected and blocked. Visual cryptographic attestation markers are implemented where supported by applications. Real-time UI anomaly detection using machine learning identifies novel attack patterns. The verification infrastructure is independently monitored and has separate failure modes from the agent runtime. Hardware-backed attestation of the display pipeline ensures screen capture integrity.
Required artefacts:
Retention requirements:
Access requirements:
Testing AG-121 compliance requires adversarial simulation of UI manipulation attacks across the full range of attack vectors that target computer-use agents.
Test 8.1: Overlay Attack Detection
Test 8.2: Phishing Page Detection
Test 8.3: DOM Manipulation Detection
Test 8.4: Application Identity Verification
Test 8.5: Credential Entry Protection
Test 8.6: TOCTOU Attack Resistance
| Regulation | Provision | Relationship Type |
|---|---|---|
| EU AI Act | Article 9 (Risk Management System) | Supports compliance |
| EU AI Act | Article 15 (Accuracy, Robustness, Cybersecurity) | Direct requirement |
| EU AI Act | Article 14 (Human Oversight) | Supports compliance |
| SOX | Section 404 (Internal Controls Over Financial Reporting) | Supports compliance |
| FCA SYSC | 6.1.1R (Systems and Controls) | Direct requirement |
| NIST AI RMF | MAP 3.2, MANAGE 2.2, MANAGE 3.1 | Supports compliance |
| ISO 42001 | Clause 6.1 (Actions to Address Risks), Clause 8.2 (AI Risk Assessment) | Supports compliance |
| DORA | Article 9 (ICT Risk Management Framework) | Supports compliance |
Article 15 requires high-risk AI systems to be resilient against attempts by unauthorised third parties to exploit system vulnerabilities, including "attempts to manipulate the training data set or pre-trained components, inputs designed to cause the AI system to make an error, or model flaws." UI spoofing attacks are inputs designed to cause the AI system to make an error — they manipulate the agent's perception of reality to induce incorrect actions. An agent without UI authenticity verification fails Article 15's robustness requirement because it is trivially vulnerable to input manipulation. The structural verification controls required by AG-121 directly implement Article 15 robustness for agents operating through visual interfaces.
For agents that approve financial transactions through visual interfaces (ERP systems, payment platforms, accounting software), the integrity of what the agent perceives is a critical internal control. A SOX auditor will ask: "How do you ensure that the amount the agent approved matches the actual transaction amount?" If the answer relies solely on the agent reading the screen, the control is inadequate. Dual-channel verification — confirming the UI-displayed amount against the application's API or database — provides the independent verification that SOX requires for financial transaction controls.
The FCA expects that AI agent controls are at least equivalent to human operator controls. A human operator verifying a transaction has the cognitive ability to notice visual anomalies (unusual formatting, unexpected values, browser warning bars). An AI agent operating at pixel level does not have equivalent anomaly detection unless it is explicitly implemented. AG-121's verification requirements implement the equivalent of human visual scrutiny through structural controls — ensuring that agents operating through visual interfaces have controls at least as robust as the human operators they replace.
For financial entities, UI spoofing attacks against AI agents represent an ICT risk that must be identified, assessed, and mitigated. The dual-channel verification and UI integrity checks required by AG-121 are ICT risk management measures under DORA Article 9, specifically addressing the risk of manipulated inputs to AI systems that process financial transactions.
| Field | Value |
|---|---|
| Severity Rating | High |
| Blast Radius | Per-application — scoped to the actions available within the application the agent is interacting with, but potentially extending to cross-system impact if the application has integration capabilities |
Consequence chain: A failure of UI authenticity verification enables an attacker to control the agent's perception of reality, inducing the agent to execute actions that it would not have taken based on accurate information. The immediate technical failure is a discrepancy between what the agent perceives and the actual application state. The agent acts on the fabricated perception — approving a different transaction amount, sending data to a different recipient, confirming a different configuration change. The operational impact depends on the application context: financial applications face fraudulent transactions (potentially hundreds of thousands of pounds per incident); healthcare applications face incorrect clinical actions; safety-critical applications face incorrect control system interactions. The business consequence includes direct financial loss from fraudulent transactions, regulatory enforcement for inadequate controls, breach of fiduciary duties where agents act on behalf of clients, and potential patient harm or safety incidents in healthcare and critical infrastructure contexts. Unlike session-based attacks (AG-120), UI spoofing attacks can be precisely targeted to a single high-value action, making them efficient for adversaries and potentially catastrophic in impact relative to effort.
Cross-references: AG-001 (Operational Boundary Enforcement) provides the mandate that limits the maximum impact of any single agent action, bounding the blast radius of UI spoofing attacks. AG-005 (Instruction Integrity Verification) addresses prompt injection, which is a complementary attack vector — AG-005 protects the agent's instructions; AG-121 protects the agent's perceptions. AG-013 (Data Sensitivity and Exfiltration Prevention) governs the data handling controls that prevent agents from outputting sensitive information observed in UIs. AG-029 (Credential Integrity Verification) ensures credentials are protected in storage; AG-121 ensures credentials are not entered into fraudulent UIs. AG-031 (Code Execution Boundary Enforcement) governs the execution environment integrity that supports UI verification controls. AG-120 (Browser Session and Token Governance) governs the authentication lifecycle for browser-based interactions; AG-121 governs the visual integrity of those interactions.