How to Measure Retail Media Incrementality with Holdout Tests | Particular Audience
Use holdout groups to separate sales caused by retail media from sales that would have happened anyway, and report brand halo.
How to measure retail media incrementality
Measure retail media incrementality by comparing shoppers who could see a campaign with a randomly held-out group who could not. The difference in sales between the two groups is the incremental effect. Report it for the advertised product and the brand's wider range, so brands see both direct lift and brand halo, not just attributed return on ad spend.
Why attribution is not enough
Attributed sales count every purchase after a click, including ones that would have happened anyway.
Set up a holdout
Randomly exclude a share of eligible shoppers from the campaign. Keep it large enough to read a difference and fixed for the test period.
Read the result
Compare sales per shopper across groups, for the product, the brand and the category.
Report honestly
Show confidence ranges and test length alongside the uplift.
How Particular Audience does it
In production
- Intersport Austria: Intersport launched onsite retail media — and proved brand growth in five weeks (17.3× Brand-halo ROAS)
- 8-Brand Retail Media Benchmark (Pet Retail): Challenger vs Incumbent: how two suppliers won the same RMN (127× Peak challenger ROAS)
Frequently asked questions
Does a holdout cost revenue?
A small amount of ad exposure is withheld during the test. The evidence it produces is what justifies future brand budgets.
Related guides
- Incrementality: Incrementality is the extra sales caused by advertising, measured against a group that did not see it.
- Holdout test: A holdout test withholds a campaign from a random group of shoppers to measure the campaign's true effect.
- Brand halo: Brand halo is the sales uplift across a brand's wider range caused by advertising one of its products.