A long/short equity portfolio manager wants to incorporate satellite imagery of retail parking lots as an alpha signal to predict same-store sales results before quarterly earnings announcements. The data science team delivers a parking lot occupancy index that achieves a Sharpe ratio of 1.4 and an information coefficient of 0.18 in a five-year backtest. Before approving live deployment, the CIO raises a concern about 'backtest performance driven by selection bias in the satellite coverage universe.' Which risk does the CIO MOST likely have in mind?
Select an answer to reveal the explanation.
Short Explanation
Satellite parking-lot data can look great in a backtest if it only covers places where the satellites see clearly all year. You need the signal to work across the retail stocks you'll actually trade, not just the easy-to-image subset. That's selection bias in the coverage universe.
Full Explanation
Selection bias in alternative data backtests is one of the most insidious forms of data snooping in quantitative investment research. It occurs when the historical sample used to build and test a signal is systematically different from the universe to which the signal will be applied in live trading.
For satellite parking lot imagery, the coverage universe is not uniformly distributed. Satellite revisit rates, cloud cover, snow cover, building shadows, and atmospheric conditions systematically vary by geography and season. Retailers with locations in sun-belt states such as Florida, Texas, California, and Arizona produce high-quality, year-round imagery. Retailers concentrated in northern or Great Lakes states frequently have degraded imagery from October through March due to snow cover and low sun angles.
If the vendor's historical dataset only includes locations where imagery quality meets a minimum threshold, which is a reasonable data quality control, the backtest inadvertently tests the signal only on the observable, high-quality subset. The live portfolio, however, will include the full universe of retail stocks regardless of geographic imaging quality, including retailers whose parking lots are frequently unobservable. The live signal will be noisier, sparser, and less timely than backtested, inflating the apparent backtest Sharpe ratio.
Why the other concerns fail: The concern that the satellite imagery vendor does not meet the firm's data quality standards for cloud-cover correction and atmospheric normalization is a valid operational concern for satellite imagery data, but the CIO's phrase 'coverage universe' directly points to geographic selection bias, not image processing quality. The concern that the model uses gradient boosting instead of a linear factor model is a valid concern for interpretability, but it is unrelated to the coverage bias issue the CIO identified. The concern that the parking lot occupancy signal is highly correlated with the Fama-French SMB factor is a valid concern for establishing signal novelty, but 'selection bias in the satellite coverage universe' is not terminology associated with factor correlation; it specifically describes the geographic and observability sampling problem.
Best practice for alternative data due diligence includes coverage analysis, meaning checking what fraction of the intended universe is actually covered, survivorship testing, and a comparison of the signal's Sharpe ratio within the covered versus uncovered subset of the universe.