Understand the evidence behind each result

A useful performance claim tells you what was measured, what it was compared with and what the evidence can support. Use these categories to distinguish commercial experiments, operational observations, attribution, offline search evaluation and illustrative calculations.

Separate the kinds of evidence

Controlled commercial result. A randomised comparison can estimate the effect of a defined change when assignment, logging and analysis are valid. Describe the population, comparator, metric and period, and report uncertainty where available.

Observed operational result. A before-and-after change can describe what happened during a rollout. It may also reflect seasonality, promotions, catalogue changes or other interventions. State those limits instead of presenting the difference as isolated causality.

Attributed activity. Revenue assigned to a click, placement or campaign depends on the attribution rule. It helps describe the customer journey, but does not establish how much revenue would have disappeared without that interaction.

Offline relevance evaluation. Judged query-product results compare search quality under a specified catalogue, query sample and rubric. They support diagnosis and model selection; they do not directly measure live commercial lift.

Illustrative scenario. A calculation shows the consequence of its inputs. It should display those inputs and remain clearly separate from a customer result.

Give each claim a traceable record

A useful evidence record identifies the metric, numerator and denominator, baseline, comparator, sample, dates, method and material limitations. Preserve units and distinguish percentage points from relative percentages. If a necessary field was not disclosed, say so rather than reconstructing it from a headline.

Keep the source close to the claim. An anonymised case can still describe its methods without revealing the retailer. A chart should identify whether it shows measured results, attributed activity or a scenario before the reader interprets the size of the bars.

Use metrics for their stated purpose

Read nDCG as ranked quality and precision as the relevant share under a defined threshold. State the reference pool when reporting recall. For commercial tests, define whether the denominator is all eligible assigned visitors, exposed visitors or another population, and explain why the comparison is valid.

Record the combination of organic ranking, merchandising, promotions and paid placements that was tested. Results from one configuration should not automatically be attributed to every component in it.

Review the limits before reusing a number

Ask whether the new claim preserves the original metric and scope. Do not turn a recommendation click-conversion result into a sitewide conversion lift, an offline score into revenue, or a before-and-after comparison into a controlled experiment. When evidence is incomplete, a narrower accurate statement is more useful than a larger unsupported one.

Use the testing guide to plan a new experiment, or review the search evaluation method.