Agent tasks keep succeeding, with fewer model calls.
Τ-bench puts an AI agent on a retail store’s support desk, with the store’s own tools, its written policy and customers who change their minds. Duplex worked beside the agent model on every turn of every conversation.
Τ-bench, by Sierra Research. The benchmark →
Internal benchmarks. Every figure on this page comes from tests Vareon ran itself and has not been independently verified. Read the disclaimer
- 83.3%*Task successThe model on every turn: 82.5%
- 12.8%*Turns completed without the model≈15% fewer model calls
- 100%*Success on new tasks
* Figures from Vareon’s internal benchmarks, not independently verified. Read the disclaimer
One conversation, turn by turn.
Step 1 of 6: A customer
As successful as calling the model on every turn.
Task success over 120 conversations, and on the tasks the agent had never seen before.
Step 1 of 2: Every task
Duplex finished 83.3% of tasks. The model on every turn finished 82.5%.*
It skips a call only on turns it has learned.
Of every 100 turns, who completed each one, and whether the task still succeeded.
Step 1 of 3: Every turn
With the model on every turn, every turn is a call, and 82.5% of tasks succeed.*
Fewer calls and finished tasks, together.
Turns completed without the model, against task success, for every approach in the test.
Step 1 of 3: Every call
Four approaches call the model on every turn; their success ranges from 83.3% to 0.8%.*
How it was run.
- Benchmark
- Τ-bench (tau2-bench 1.0), retail, by Sierra Research: public and open source.
- The work
- Multi-turn conversations with simulated retail customers, using the store’s own tools under its written policy.
- Scale
- Three runs of forty conversations for every approach.
- Models
- GPT-5.5 as the agent for every approach; GPT-4.1-mini played the customer.
- Compared with
- The model on every turn, a memory layer, retrieval, a semantic cache and knowledge-graph context.
- Kept equal
- The same tools, policy, customer and agent model for every approach; Duplex ran as the live service.
What it means for your work.
Your agents keep finishing the job.
Task success matched the model on every turn while Duplex completed the turns it had learned.
New work still gets done.
Duplex finished every new task in the test: what it has not learned goes to your model.
Savings that leave conversations whole.
Duplex skips a call only on a turn it has learned. A cache replayed stored answers mid-conversation and finished 8.3% of tasks.*
Next: A steady reading cost, however long the history.Read it →
See it on your own work.
From your own assistant to every agent your company runs, a pilot puts Duplex on your own work, with the measures agreed before it begins.