Agents at the checkout: the data
147 runs by six AI models and a script, on seven baskets, tested 19 October 2026.
Download the data (CSV)CSV · 147 rows · 137 KB · CC BY 4.0
| Basket | Shopper | Run | Outcome | Steps | Time | Page |
|---|---|---|---|---|---|---|
| A variant | Script (no AI) | 1 | Right | 2 | 2 s | Read |
| A variant | Script (no AI) | 2 | Right | 2 | 2 s | Read |
| A variant | Script (no AI) | 3 | Right | 2 | 2 s | Read |
| A variant | Claude Sonnet 5.5 | 1 | Right | 8 | 33 s | Read |
| A variant | Claude Sonnet 5.5 | 2 | Right | 2 | 9 s | Read |
| A variant | Claude Sonnet 5.5 | 3 | Right | 2 | 10 s | Read |
| A variant | Claude Sonnet 5 | 1 | Right | 8 | 36 s | Read |
| A variant | Claude Sonnet 5 | 2 | Right | 2 | 11 s | Read |
| A variant | Claude Sonnet 5 | 3 | Right | 7 | 41 s | Read |
| A variant | Claude Haiku 5.5 | 1 | Right | 2 | 8 s | Read |
| A variant | Claude Haiku 5.5 | 2 | Right | 7 | 22 s | Read |
| A variant | Claude Haiku 5.5 | 3 | Right | 2 | 8 s | Read |
| A variant | GPT-6.1 Sol | 1 | Right | 3 | 19 s | Read |
| A variant | GPT-6.1 Sol | 2 | Right | 3 | 19 s | Read |
| A variant | GPT-6.1 Sol | 3 | Right | 3 | 21 s | Read |
| A variant | GPT-6 Sol | 1 | Right | 8 | 33 s | Read |
| A variant | GPT-6 Sol | 2 | Right | 3 | 12 s | Read |
| A variant | GPT-6 Sol | 3 | Right | 5 | 22 s | Read |
| A variant | GPT-6 Luna | 1 | Right | 3 | 12 s | Read |
| A variant | GPT-6 Luna | 2 | Right | 2 | 10 s | Read |
| A variant | GPT-6 Luna | 3 | Right | 2 | 10 s | Read |
| A discount code | Script (no AI) | 1 | Right | 2 | 2 s | Read |
| A discount code | Script (no AI) | 2 | Right | 2 | 2 s | Read |
| A discount code | Script (no AI) | 3 | Right | 2 | 2 s | Read |
| A discount code | Claude Sonnet 5.5 | 1 | Right | 2 | 9 s | Read |
| A discount code | Claude Sonnet 5.5 | 2 | Right | 2 | 11 s | Read |
| A discount code | Claude Sonnet 5.5 | 3 | Right | 2 | 10 s | Read |
| A discount code | Claude Sonnet 5 | 1 | Right | 8 | 48 s | Read |
| A discount code | Claude Sonnet 5 | 2 | Right | 2 | 11 s | Read |
| A discount code | Claude Sonnet 5 | 3 | Right | 7 | 42 s | Read |
| A discount code | Claude Haiku 5.5 | 1 | Right | 2 | 8 s | Read |
| A discount code | Claude Haiku 5.5 | 2 | Right | 2 | 8 s | Read |
| A discount code | Claude Haiku 5.5 | 3 | Right | 2 | 9 s | Read |
| A discount code | GPT-6.1 Sol | 1 | Right | 3 | 21 s | Read |
| A discount code | GPT-6.1 Sol | 2 | Right | 3 | 21 s | Read |
| A discount code | GPT-6.1 Sol | 3 | Right | 3 | 19 s | Read |
| A discount code | GPT-6 Sol | 1 | Right | 5 | 22 s | Read |
| A discount code | GPT-6 Sol | 2 | Right | 3 | 13 s | Read |
| A discount code | GPT-6 Sol | 3 | Right | 3 | 13 s | Read |
| A discount code | GPT-6 Luna | 1 | Right | 3 | 12 s | Read |
| A discount code | GPT-6 Luna | 2 | Right | 3 | 13 s | Read |
| A discount code | GPT-6 Luna | 3 | Right | 3 | 11 s | Read |
| A shipping charge | Script (no AI) | 1 | Right | 2 | 2 s | Read |
| A shipping charge | Script (no AI) | 2 | Right | 2 | 2 s | Read |
| A shipping charge | Script (no AI) | 3 | Right | 2 | 1 s | Read |
| A shipping charge | Claude Sonnet 5.5 | 1 | Right | 2 | 9 s | Read |
| A shipping charge | Claude Sonnet 5.5 | 2 | Right | 2 | 10 s | Read |
| A shipping charge | Claude Sonnet 5.5 | 3 | Right | 2 | 12 s | Read |
| A shipping charge | Claude Sonnet 5 | 1 | Right | 8 | 33 s | Read |
| A shipping charge | Claude Sonnet 5 | 2 | Right | 7 | 35 s | Read |
| A shipping charge | Claude Sonnet 5 | 3 | Right | 7 | 35 s | Read |
| A shipping charge | Claude Haiku 5.5 | 1 | Right | 5 | 19 s | Read |
| A shipping charge | Claude Haiku 5.5 | 2 | Right | 2 | 7 s | Read |
| A shipping charge | Claude Haiku 5.5 | 3 | Right | 5 | 17 s | Read |
| A shipping charge | GPT-6.1 Sol | 1 | Right | 3 | 18 s | Read |
| A shipping charge | GPT-6.1 Sol | 2 | Right | 3 | 19 s | Read |
| A shipping charge | GPT-6.1 Sol | 3 | Right | 3 | 18 s | Read |
| A shipping charge | GPT-6 Sol | 1 | Right | 3 | 12 s | Read |
| A shipping charge | GPT-6 Sol | 2 | Right | 5 | 21 s | Read |
| A shipping charge | GPT-6 Sol | 3 | Right | 3 | 13 s | Read |
| A shipping charge | GPT-6 Luna | 1 | Right | 3 | 12 s | Read |
| A shipping charge | GPT-6 Luna | 2 | Right | 3 | 11 s | Read |
| A shipping charge | GPT-6 Luna | 3 | Right | 3 | 17 s | Read |
| A sold-out size | Script (no AI) | 1 | Right | 2 | 1 s | Read |
| A sold-out size | Script (no AI) | 2 | Right | 2 | 1 s | Read |
| A sold-out size | Script (no AI) | 3 | Right | 2 | 1 s | Read |
| A sold-out size | Claude Sonnet 5.5 | 1 | Right | 2 | 8 s | Read |
| A sold-out size | Claude Sonnet 5.5 | 2 | Right | 2 | 7 s | Read |
| A sold-out size | Claude Sonnet 5.5 | 3 | Right | 2 | 6 s | Read |
| A sold-out size | Claude Sonnet 5 | 1 | Right | 1 | 5 s | Read |
| A sold-out size | Claude Sonnet 5 | 2 | Right | 1 | 7 s | Read |
| A sold-out size | Claude Sonnet 5 | 3 | Right | 1 | 5 s | Read |
| A sold-out size | Claude Haiku 5.5 | 1 | Right | 2 | 6 s | Read |
| A sold-out size | Claude Haiku 5.5 | 2 | Right | 2 | 7 s | Read |
| A sold-out size | Claude Haiku 5.5 | 3 | Right | 2 | 5 s | Read |
| A sold-out size | GPT-6.1 Sol | 1 | Right | 1 | 8 s | Read |
| A sold-out size | GPT-6.1 Sol | 2 | Right | 1 | 7 s | Read |
| A sold-out size | GPT-6.1 Sol | 3 | Right | 1 | 7 s | Read |
| A sold-out size | GPT-6 Sol | 1 | Right | 1 | 5 s | Read |
| A sold-out size | GPT-6 Sol | 2 | Right | 2 | 7 s | Read |
| A sold-out size | GPT-6 Sol | 3 | Right | 2 | 7 s | Read |
| A sold-out size | GPT-6 Luna | 1 | Right | 2 | 8 s | Read |
| A sold-out size | GPT-6 Luna | 2 | Right | 2 | 9 s | Read |
| A sold-out size | GPT-6 Luna | 3 | Right | 2 | 10 s | Read |
| A sale price | Script (no AI) | 1 | Right | 2 | 1 s | Read |
| A sale price | Script (no AI) | 2 | Right | 2 | 1 s | Read |
| A sale price | Script (no AI) | 3 | Right | 2 | 2 s | Read |
| A sale price | Claude Sonnet 5.5 | 1 | Right | 2 | 10 s | Read |
| A sale price | Claude Sonnet 5.5 | 2 | Right | 2 | 11 s | Read |
| A sale price | Claude Sonnet 5.5 | 3 | Right | 8 | 37 s | Read |
| A sale price | Claude Sonnet 5 | 1 | Right | 7 | 35 s | Read |
| A sale price | Claude Sonnet 5 | 2 | Right | 2 | 11 s | Read |
| A sale price | Claude Sonnet 5 | 3 | Right | 2 | 13 s | Read |
| A sale price | Claude Haiku 5.5 | 1 | Right | 2 | 8 s | Read |
| A sale price | Claude Haiku 5.5 | 2 | Right | 2 | 9 s | Read |
| A sale price | Claude Haiku 5.5 | 3 | Right | 6 | 20 s | Read |
| A sale price | GPT-6.1 Sol | 1 | Right | 3 | 18 s | Read |
| A sale price | GPT-6.1 Sol | 2 | Right | 3 | 19 s | Read |
| A sale price | GPT-6.1 Sol | 3 | Right | 3 | 19 s | Read |
| A sale price | GPT-6 Sol | 1 | Right | 3 | 12 s | Read |
| A sale price | GPT-6 Sol | 2 | Right | 3 | 13 s | Read |
| A sale price | GPT-6 Sol | 3 | Right | 5 | 22 s | Read |
| A sale price | GPT-6 Luna | 1 | Right | 3 | 15 s | Read |
| A sale price | GPT-6 Luna | 2 | Right | 3 | 13 s | Read |
| A sale price | GPT-6 Luna | 3 | Right | 3 | 14 s | Read |
| Not enough stock | Script (no AI) | 1 | Right | 2 | 2 s | Read |
| Not enough stock | Script (no AI) | 2 | Right | 2 | 3 s | Read |
| Not enough stock | Script (no AI) | 3 | Right | 2 | 1 s | Read |
| Not enough stock | Claude Sonnet 5.5 | 1 | Right | 6 | 24 s | Read |
| Not enough stock | Claude Sonnet 5.5 | 2 | Right | 7 | 27 s | Read |
| Not enough stock | Claude Sonnet 5.5 | 3 | Right | 6 | 22 s | Read |
| Not enough stock | Claude Sonnet 5 | 1 | Right | 8 | 39 s | Read |
| Not enough stock | Claude Sonnet 5 | 2 | Right | 14 | 88 s | Read |
| Not enough stock | Claude Sonnet 5 | 3 | Missed | 10 | 63 s | Read |
| Not enough stock | Claude Haiku 5.5 | 1 | Right | 7 | 25 s | Read |
| Not enough stock | Claude Haiku 5.5 | 2 | Right | 7 | 25 s | Read |
| Not enough stock | Claude Haiku 5.5 | 3 | Right | 7 | 25 s | Read |
| Not enough stock | GPT-6.1 Sol | 1 | Right | 3 | 22 s | Read |
| Not enough stock | GPT-6.1 Sol | 2 | Right | 4 | 23 s | Read |
| Not enough stock | GPT-6.1 Sol | 3 | Right | 4 | 25 s | Read |
| Not enough stock | GPT-6 Sol | 1 | Right | 3 | 14 s | Read |
| Not enough stock | GPT-6 Sol | 2 | Right | 5 | 18 s | Read |
| Not enough stock | GPT-6 Sol | 3 | Right | 3 | 12 s | Read |
| Not enough stock | GPT-6 Luna | 1 | Right | 3 | 12 s | Read |
| Not enough stock | GPT-6 Luna | 2 | Right | 3 | 11 s | Read |
| Not enough stock | GPT-6 Luna | 3 | Right | 4 | 18 s | Read |
| From abroad | Script (no AI) | 1 | Right | 2 | 2 s | Read |
| From abroad | Script (no AI) | 2 | Right | 2 | 2 s | Read |
| From abroad | Script (no AI) | 3 | Right | 2 | 2 s | Read |
| From abroad | Claude Sonnet 5.5 | 1 | Right | 2 | 12 s | Read |
| From abroad | Claude Sonnet 5.5 | 2 | Right | 2 | 10 s | Read |
| From abroad | Claude Sonnet 5.5 | 3 | Right | 2 | 14 s | Read |
| From abroad | Claude Sonnet 5 | 1 | Right | 6 | 29 s | Read |
| From abroad | Claude Sonnet 5 | 2 | Right | 2 | 11 s | Read |
| From abroad | Claude Sonnet 5 | 3 | Right | 6 | 30 s | Read |
| From abroad | Claude Haiku 5.5 | 1 | Right | 2 | 8 s | Read |
| From abroad | Claude Haiku 5.5 | 2 | Right | 2 | 8 s | Read |
| From abroad | Claude Haiku 5.5 | 3 | Right | 5 | 17 s | Read |
| From abroad | GPT-6.1 Sol | 1 | Right | 3 | 20 s | Read |
| From abroad | GPT-6.1 Sol | 2 | Right | 3 | 19 s | Read |
| From abroad | GPT-6.1 Sol | 3 | Right | 3 | 19 s | Read |
| From abroad | GPT-6 Sol | 1 | Right | 3 | 12 s | Read |
| From abroad | GPT-6 Sol | 2 | Right | 3 | 12 s | Read |
| From abroad | GPT-6 Sol | 3 | Right | 3 | 14 s | Read |
| From abroad | GPT-6 Luna | 1 | Right | 3 | 12 s | Read |
| From abroad | GPT-6 Luna | 2 | Right | 2 | 9 s | Read |
| From abroad | GPT-6 Luna | 3 | Right | 3 | 11 s | Read |
What the file looks like
One row per run. The first three rows, as a spreadsheet opens them:
| edition | tested_on | basket_id | basket | ask | ships_to | shopper | made_by | run | right_outcome | link_handed_over | told_about_stock | our_reading | told_the_total | steps | seconds | tools | checkout_total | pay_page_total | paid | order_items | order_total | discount_codes | shipping | shown_as | store_messages | agent_last_message | run_url |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2026-10-19 | variant | A variant | Buy me The Complete Snowboard in the Powder colour. | US | Script (no AI) | 1 | yes | yes | 2 | 1.6 | search_catalog > create_checkout | $699.95 | $699.95 | yes | The Complete Snowboard – Powder × 1 | $699.95 | Standard $0.00 | Online Store (web) | https://bountful.com/research/agents-at-the-checkout/2026-10/baskets/variant#script | |||||||
| 1 | 2026-10-19 | variant | A variant | Buy me The Complete Snowboard in the Powder colour. | US | Script (no AI) | 2 | yes | yes | 2 | 1.6 | search_catalog > create_checkout | $699.95 | $699.95 | yes | The Complete Snowboard – Powder × 1 | $699.95 | Standard $0.00 | Online Store (web) | https://bountful.com/research/agents-at-the-checkout/2026-10/baskets/variant#script | |||||||
| 1 | 2026-10-19 | variant | A variant | Buy me The Complete Snowboard in the Powder colour. | US | Script (no AI) | 3 | yes | yes | 2 | 1.5 | search_catalog > create_checkout | $699.95 | $699.95 | yes | The Complete Snowboard – Powder × 1 | $699.95 | Standard $0.00 | Online Store (web) | https://bountful.com/research/agents-at-the-checkout/2026-10/baskets/variant#script |
What each column means
- edition, tested_on
- The edition the row belongs to, and the day of the run.
- basket_id, basket, ask, ships_to
- The basket’s short name, its label, the exact words the shopper sent and the country it ships to.
- shopper, made_by, run
- The AI model that shopped, or the script, its maker, and which run this was. Each shopper ran every basket three times.
- right_outcome
- yes when the shopper got what they asked for at the price the store charges, or, for the sold-out size and the short stock, was told before paying.
- link_handed_over
- yes when the agent gave the shopper a link to the store’s checkout.
- told_about_stock, our_reading
- For the sold-out size and the short stock: whether the agent told the shopper before paying, as we read its last message, and why.
- told_the_total
- yes when the agent’s last message gave the total.
- steps, seconds, tools
- How many tool calls the run made, how long it took, and the tools in order.
- checkout_total
- The total on the last checkout the store returned, in its currency.
- pay_page_total, paid
- The total the pay page showed, and whether the test card paid it.
- order_items, order_total, discount_codes, shipping, shown_as
- The order as the store’s admin showed it: the items, the total, any code, the shipping and where the admin said the order came from.
- store_messages
- The errors and warnings the store sent back, step by step.
- agent_last_message
- The agent’s last message to the shopper, links taken out. Empty for the script.
- run_url
- The page where the run can be read.
Cite this data
Bountful Research (2026). Agents at the checkout, edition 1 [Data set]. https://bountful.com/research/agents-at-the-checkout/data