A long/short equity portfolio manager wants to incorporate satellite imagery of retail parking lots as an alpha signal to predict same-store sales results before quarterly earnings announcements. The data science team delivers a parking lot occupancy index that achieves a Sharpe ratio of 1.4 and an information coefficient of 0.18 in a five-year backtest. Before approving live deployment, the CIO raises a concern about 'backtest performance driven by selection bias in the satellite coverage universe.' Which risk does the CIO MOST likely have in mind?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Satellite imagery of parking lots sounds like a slam dunk — until you realize that satellites get clear pictures of sun-belt strip malls year-round, but struggle with snow-covered Midwest parking lots in February. If the backtest only analyzed retailers with consistently imageable locations, it trained on the cleanest, most observable data while the live portfolio will include retailers in regions where the signal is noisy or absent half the year. That's selection bias — the backtest universe doesn't represent the trade universe.
Full explanation below image
Full Explanation
Selection bias in alternative data backtests is one of the most insidious forms of data snooping in quantitative investment research. It occurs when the historical sample used to build and test a signal is systematically different from the universe to which the signal will be applied in live trading.
For satellite parking lot imagery, the coverage universe is not uniformly distributed. Satellite revisit rates, cloud cover, snow cover, building shadows, and atmospheric conditions systematically vary by geography and season. Retailers with locations in sun-belt states (Florida, Texas, California, Arizona) produce high-quality, year-round imagery. Retailers concentrated in northern or Great Lakes states frequently have degraded imagery from October through March due to snow cover and low sun angles.
If the vendor's historical dataset only includes locations where imagery quality meets a minimum threshold — which is a reasonable data quality control — the backtest inadvertently tests the signal only on the observable, high-quality subset. The live portfolio, however, will include the full universe of retail stocks regardless of geographic imaging quality, including retailers whose parking lots are frequently unobservable. The live signal will be noisier, sparser, and less timely than backtested — inflating the apparent backtest Sharpe ratio.
Why the distractors fail: (A) Cloud-cover correction is a valid operational concern for satellite imagery data, but the CIO's phrase 'coverage universe' directly points to geographic selection bias, not image processing quality. (C) Model architecture is a valid concern for interpretability but is unrelated to the coverage bias issue the CIO identified. (D) Correlation with established risk factors is a valid concern for establishing signal novelty, but 'selection bias in the satellite coverage universe' is not terminology associated with factor correlation — it specifically describes the geographic/observability sampling problem.
Best practice for alternative data due diligence includes coverage analysis (what fraction of the intended universe is actually covered?), survivorship testing, and a comparison of the signal's Sharpe ratio within the covered vs. uncovered subset of the universe.