A retailer's synthetic web test follows only one of two page variants that shoppers are split across. What must the test report say?
Select an answer to reveal the explanation.
Short Explanation
Tasting one recipe at a split table does not tell you how the other recipe landed. You state which page variant the synthetic script actually loaded, because shoppers are split across two variants. That run is not proof for every shopper, a capture of the catalog database for both variants, or a waterfall from someone who received the other page.
Full Explanation
The synthetic script requested one page variant and executed only that variant. Shoppers are split across two variants, so the report has to say which variant the test covers. Silence on that point lets a reader treat one path as the whole experiment. Presenting the script as proof for every shopper, including people who received the variant the script never loaded, claims both a population and a page the agent did not run. Treating the run as a packet capture of the catalog database for both variants confuses a single web transaction with a copy of database activity, and it invents coverage of the variant that was never requested. Replacing browser waterfalls from shoppers on the other variant with this script discards client timing for the page those shoppers actually received. The green or slow result remains meaningful for the variant and the agent location that ran. It is not a result for the other variant. Exam caveat: when an application serves more than one page variant, synthetic coverage is the variant the script loaded. Operational check: record which variant the script requested and state in the report that the other variant was not executed.