The Proof test (A/B holdout)

A slice of your shoppers browses without try-on. Every lift number is measured against them — causation on your own store, not a vendor claim.

The short version: Start the Proof test and a percentage you choose (10% recommended) of shoppers never sees any Ello surface — they're the control. The dashboard compares conversion with vs without try-on and only calls a verdict at 95% statistical confidence with at least 200 sessions in each group and 10 buyers between them. If Complete the Look is on, an outfit test rides along measuring order-value lift. Past tests are kept forever.

The holdout split: most shoppers see try-on, a fixed slice never does. Comparing the two groups' purchase rates is what makes the lift number causal.

How the test works

• Assignment is deterministic and sticky. Each shopper session is bucketed by a stable algorithm — the same shopper stays in the same group for the whole test, so nobody flickers between seeing and not seeing try-on.
• Holdout shoppers see nothing. Not a hidden button — no Ello surface at all. They shop your store exactly as it was before Ello.
• Both groups count every visitor from their first page. That's what makes the comparison fair — and it's why both percentages sit below Shopify's product-page conversion figure. For Shopify comparisons, use the "On product pages" figure shown alongside.
• Their purchases count fully. Holdout shoppers aren't lost sales — they're the measurement. That's the whole point of a control group.

Starting a test

1. Open the Proof page. Pick a holdout size: 5%, 10% (recommended), 20%, or 50%. Bigger holdouts reach a verdict faster but show try-on to fewer shoppers.
2. Start it the day you go live, ideally. The earlier it starts, the sooner the sample builds. Promote try-on to your list during the test — the test stays fair because promotion reaches both groups.
3. Let it run. Stop any time; every past test is kept forever in the history dropdown, and selecting one re-reads the whole page over that test's window.

When does it call a verdict?

The lift verdict unlocks only when the math is real: 95% statistical confidence, with at least 200 sessions in each group and 10 buyers across the two. Until then the page says it's still collecting — deliberately. A small store may take weeks to get there; that isn't the test failing, it's the test refusing to guess.

Why your test percentages look lower than Shopify's: Shopify's conversion rate usually divides by sessions that reached a product page; the Proof test divides by all visitors in each group, from their first page, because that's the fair basis for a controlled comparison. The Proof page also shows an "On product pages: X of Y bought" figure for apples-to-apples reading.

The outfit test (order-value lift)

If Complete the Look is on when you start the Proof test, a 50/50 outfit test rides along automatically: half the exposed shoppers get the outfit offer, half don't, and the dashboard reports average-order-value lift with its own confidence gates. One holdout, every answer. It can also run standalone from the Proof page.

Returns and Receipts

• Returns — do tried-on items come back less often? The page compares your store-wide return rate against tried-on-before-buying items, from your own orders and refunds — not an industry stat.
• Receipts — every attributed order in a table (order, product tried on, time from try-on to purchase, order value), exportable as CSV, each row tied to a Shopify order ID.

Common questions

Do holdout shoppers hurt my sales?

They shop normally — they just don't see try-on, exactly like every shopper before you installed Ello. Their purchases count fully. A 10% holdout means 90% of traffic gets the feature while the test runs.

How long until I get a verdict?

The verdict unlocks at 95% confidence with at least 200 sessions in each group and 10 buyers across the two. How fast you get there depends on traffic — days for a high-traffic store, weeks for a small one. The test won't call it early, on purpose.

Why does the test show lower conversion than my Shopify analytics?

Different denominators. The test counts every visitor in each group from their first page (the fair basis for a controlled comparison); Shopify typically measures shoppers who reached a product page. Use the test's "On product pages" figure for direct comparison.

Can I run the test more than once?

Yes. Stop and start tests whenever you like — every past test is kept forever, and you can re-open any of them to see its full results over its own window.

Is the incremental revenue number exact?

It's computed from the conversion-rate difference and deliberately floored — presented as a floor, not a ceiling. Ello would rather understate lift than inflate it.

All Ello docs