Agentic Commerce World: An Auditable and Verifiable Environment for Vibe Commerce
Abstract
Agentic Commerce World introduces an auditable environment and benchmark for evaluating independent buyer and merchant agents that pursue private goals through natural-language commerce tasks.
In vibe coding, people describe software in natural language and delegate implementation to AI agents. By analogy, vibe commerce allows people to express buying or selling goals in natural language and delegate the corresponding tasks to agents. Commerce, however, requires independently controlled Buyer and Merchant agents to interact in a shared market while preserving their private objectives and distinct authority. We introduce Agentic Commerce World (ACWorld), an environment for evaluating such agents across ongoing transactions. Through its Vibe Commerce Protocol (VCP), ACWorld validates agent actions before updating shared transaction state and records the resulting interactions, making agent behavior auditable and evaluation reproducible. The ACWorld Benchmark contains a 200-task capability-coverage track and a 60-task large-catalog track that searches 785,022 transactable listings. Across ten models, mean scores range from 65.9% to 85.6% and from 56.1% to 91.4%, respectively. Our analysis shows that process-level evidence is necessary: final state alone can miss evaluated errors, incomplete trajectories still retain useful process signals, and large-catalog tasks expose bottlenecks across stages.
Community
🚀 Excited to share our new work:
Agentic Commerce World: An Auditable and Verifiable Environment for Vibe Commerce
We introduce a new platform for vibe commerce. Similar to vibe coding, people express their commercial goals in natural language and delegate decision-making, negotiation, and transaction execution to AI agents.
Vibe Commerce is fundamentally bilateral and multi-agent. Buyer and Merchant agents represent different sides of a market, pursue private objectives, and interact strategically while retaining separate authority.
To support this setting, we built:
🔗 Vibe Commerce Protocol (VCP)
Agents propose actions, the Commerce Intelligence Platform validates them, and the World records their actual effects. This makes agent behavior reproducible and auditable.
🌐 Agentic Commerce World (ACWorld)
ACWorld is a persistent environment in which multiple Buyer and Merchant agents can interact throughout ongoing transactions.
🧠 ACWorld also generates structured interaction trajectories that can support agent evaluation and future training through process supervision, imitation learning, and reinforcement learning.
📊 We release two benchmarks:
200 tasks covering 80 commerce capabilities
60 tasks over 785,022 searchable listings
Across ten models, performance ranges from 65.9% to 85.6% on the capability benchmark and from 56.1% to 91.4% on the large catalog benchmark.
🔍 Notably, some incorrect executions still reached apparently valid final states. Evaluating commercial agents, therefore, requires examining both the outcome and the process that produced it.
We hope ACWorld can support research into agentic commerce, buyer-merchant interaction, agent training, and reliable execution.
Paper: https://lnkd.in/dnGxDwdH
Code and benchmark: https://lnkd.in/dUGvTCNF
Website: https://shichengf.github.io/ACWorld/
Get this paper in your agent:
hf papers read 2608.02441 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper