Does an AI shopping assistant lift desktop conversion?
Desktop shoppers found products by browsing menus and collections. We tested adding an AI shopping assistant they could ask for help. Over 29 days, conversion fell 14.5% while revenue per visitor rose, and neither result reached our confidence bar.
Not significant Desktop 29 days Not called Reviewed
The hypothesis
By adding an AI shopping assistant on desktop, we will help shoppers find the right products faster than browsing menus, resulting in higher conversion and larger orders.
The store has a large catalog, and shoppers often move through several menus and collections before they find what they need. An assistant that answers questions in plain language could shorten that search and give shoppers more confidence in what they choose.
Because this was a new kind of feature, we started it on 10% of desktop traffic so it could not cause a large loss. We then raised it to 20% and later 30% as the data came in.
What we changed
We added an AI shopping assistant to the desktop site.
In the variation, desktop shoppers could open an AI assistant, ask questions in plain language and get product suggestions. The control had no assistant. Mobile was left out of this first round, and the rest of the site stayed the same.
- AI assistant added on desktop
- Shoppers ask questions in plain language
- Assistant suggests matching products
- Started on 10% of traffic, raised to 30%
The result
Conversion fell 14.5% with the AI assistant, short of our bar.
Fewer shoppers bought with the assistant in place, but those who did spent more. Revenue per visitor was up 127.6%, driven mostly by a few very large orders in the variation. We usually take outliers like these out of the analysis. Here we kept them in, since helping shoppers build bigger orders is part of what the assistant is meant to do.
The test ran on a small share of traffic, so a few orders could swing the result. Neither conversion nor revenue reached our confidence bar, and we paused the test.
What we would do next
The assistant may help large orders more than everyday ones.
The pattern of fewer orders but larger ones suggests the assistant was most useful to shoppers planning big purchases. Before a rerun, we would improve which products the assistant suggests first, expand what it knows about the catalog, and test it on more traffic so a handful of orders cannot decide the result.