Darkmind / Issue 028

Revalidate the Click Before an Agent Acts

Treat a browser agent’s planned click as a short-lived proposal: bind it to the current page, target, and expected effect, then re-check those facts at execution time.

Nov 4, 20264 min readBrowser agents

A browser agent may correctly identify a button and still take the wrong action. A page can refresh, an A/B test can move controls, a dialog can steal focus, or a person can change a form while approval is pending. In that gap, a click planned for “Save draft” can land on “Send,” or typed text can go to whatever field has focus now.

The practical move is to treat every consequential browser action as a short-lived proposal. Capture what the agent observed, name the target and intended effect, then revalidate the page and target immediately before execution. If the evidence no longer matches, stop and observe again. Do not let an old screenshot, selector, or approval silently authorize a new page state.

This is not merely a visual-automation problem. OpenAI’s browser-driver guidance says that approval state is a prior observation, not a lock on the document or focused control; a page can navigate or focus can change while a person decides. It recommends checking the original target at execution for consent tied to a particular control.[2] Its integration guide makes the same operating pattern concrete: give the model a current observation when UI state is unknown, keep controls in the function implementation, and return the actual result.[1]

Separate planning from execution state

A plan answers: what did the agent intend to do when it looked at the page? An execution guard answers: is that still what this action will do now? Keep both outside the model’s prose.

For a low-risk read, a stale page may only mean a missed result. For a consequential action—submitting a form, changing a subscription, sending a message, accepting terms, or exporting data—the guard should bind the decision to facts the driver can inspect. Those facts might include the tab URL, page title, form identifier, a stable DOM locator, visible button label, selected account, recipient count, and a digest of the values about to be submitted.

Avoid using screen coordinates as the identity of a dangerous target. Coordinates are a way to operate a display, not proof of what occupies that pixel now. If your environment supports semantic locators, use a locator plus the target’s accessible name and expected state. If it only supports screenshots and coordinates, take a fresh screenshot and require a narrow visual or driver-level check before the click. The check belongs in trusted application code, not in an instruction that asks the model to “be careful.”

Copy this pre-click revalidation card

Use this around any browser action that can change data, disclose information, send, spend, publish, or accept a binding agreement.

PLAN (created from a current observation)
- action_id: [server-issued ID]
- session_id / tab_id: [current browser context]
- intended_effect: [save draft | submit order | send message | export]
- target: [stable locator + accessible name / semantic control ID]
- expected_url: [origin + allowed path pattern]
- expected_page_marker: [title, heading, form ID, or version]
- expected_inputs: [allowlisted fields and normalized values]
- observed_at: [timestamp]
- expires_at: [short deadline]

BEFORE EXECUTION (driver-enforced)
1. Confirm the same session and permitted origin are active.
2. Re-read the current URL and page marker.
3. Resolve the target again; require exactly one enabled, visible match.
4. Re-read the specific fields that define the effect.
5. Compare current facts with the planned facts.
6. If any required fact differs, return `ui_state_changed`; do not click.
7. If all checks pass, execute the one action and record the receipt.

RECEIPT
{action_id, tab_id, target_observed, checks_passed, executed_at,
 result_state, verification_ref, next_safe_action}

The key is to define which differences matter. A changing unread-notification count probably should not block saving a draft. A changed recipient, amount, destination account, selected workspace, button label, origin, or form version should. Make this comparison rule per workflow. A generic “page changed” test can cause needless failures; a vague “looks similar” test can authorize the wrong effect.

Verify after the action too

A passed pre-click check says only that the requested action was aimed at the intended current state. It does not prove the site completed it. A submission can fail validation, trigger a second confirmation page, time out, or save a different result than expected.

After the action, obtain an independent outcome witness: a confirmation identifier, a changed record state, a receipt page, an API result, or a re-read of the named draft. OpenAI’s integration recipe illustrates this distinction by checking the saved file independently rather than accepting the visible editor state alone.[1] Store that witness with the action ID. If the browser disconnected or the page became ambiguous after submission, report an unknown outcome and look for the site’s receipt before retrying.

Bound the whole run as well. OpenAI’s driver reference recommends a maximum-step limit and says a final allowed request cannot start new effects.[2] Revalidation is not a reason for an agent to inspect indefinitely until a click becomes convenient. Give it a retry count and a terminal result such as completed, ui_state_changed, needs_approval, blocked, or unknown_outcome.

Failure mode: approving an intent but not the control

A human can approve “send this monthly report” while the page changes to a different customer, audience, or report version. Likewise, an agent can receive approval for a stable selector that now resolves to a newly rendered control with a broader effect. Reusing the old approval because the button name seems familiar turns consent into a standing permission.

Expire action plans quickly, bind approval to the exact target and effect, and make an unexpected state change return to observation or a fresh approval. The goal is not to make browser agents unable to act. It is to ensure the click that happens is still the click that was reviewed.

The habit to keep

for a consequential browser action, do not execute yesterday’s observation. Re-check the active page, exact target, and effect at the moment of action—then verify what the site actually did.

Sources

  1. OpenAI, “Computer use integration recipes”
  2. OpenAI, “Browser and desktop drivers”