Open Agent Passport Review: Blocking Unauthorized AI Agent Payments

Open Agent Passport (OAP) is an open specification that adds deterministic authorization checks before AI agents execute payments. It signs passports with Ed25519 keys and produces Decision objects that enforce recipient limits and capability scopes. The approach targets payment security gaps that persist even when models receive alignment training.

How Open Agent Passport Enforces Payment Rules

OAP v1.0 defines agent identity through decentralized identifiers and capability scopes inside signed JSON passports. Before any tool call, the system generates a signed Decision object that a policy engine evaluates. This step occurs in roughly 53 milliseconds and blocks execution when a requested transfer violates the passport rules.

The specification works alongside existing frameworks such as LangChain and CrewAI. It does not replace model alignment or sandboxing; instead it supplies a separate, bypass-resistant layer. The companion paper “Before the Tool Call” documented the initial CTF results that shaped the design.

Developers release the full dataset, passports, scoring code, and evaluation scripts on Hugging Face under a CC BY 4.0 license. This release supports independent replication and further analysis of agent behavior across different policy settings.

APort Vault Benchmark Results

The APort Vault benchmark replayed 4,371 human-authored attacks from a 2026 CTF against a simulated payment agent. Researchers ran 225,964 completed evaluations across 14 models from eight labs, covering five policy levels and two replay tracks. One track tested single-turn prompts; the other tested full attack sequences.

In levels 2–4, where passports allowed some recipients but blocked others, the model-alone condition produced 140 unauthorized transfers out of 76,842 attempts. Adding the OAP layer reduced that figure to zero out of 69,297 attempts. The same pattern held on 68,970 matched model-prompt-track triples, with the per-session upper bound estimated at 0.38 percent.

The reduction did not result from blanket refusals. Behind the OAP layer, 25,370 payments executed successfully, while policy denied 187 of 25,640 transfer calls evaluated. Most denials involved forbidden recipients. Level 4 request rates ranged from 71.2 to 84.3 percent across models, and 809 prompts triggered requests from every model tested, each resulting in payment to the allowlisted recipient.

Limitations and Next Steps for Adoption

All tests occurred inside a simulated bank environment with no real funds at risk. The benchmark author is also the founder of APort Technologies, creating a clear conflict of interest. A judge reliability audit produced a pooled kappa of 0.772, though the headline metric relies on deterministic execution counts rather than subjective scoring.

No independent external audit of the OAP specification has been published, and broader capability testing beyond payments remains limited. The released dataset now allows other researchers to examine edge cases and extend evaluations to additional models and architectures.

Integrations are expanding into production frameworks, and early discussions focus on enterprise policy packs for finance and data-access controls. Organizations considering OAP can start by reviewing the specification at aport.io and testing the Hugging Face evaluation scripts against their own agent workflows.