A public-health classifier is tested on last year's strains only; the audit challenges temporal validity tests. What should the auditor emphasize?
Select an answer to reveal the explanation.
Short Explanation
Last year's strains are last year's exam. Public-health AI needs temporal and out-of-distribution tests. High scores on old data can still flop on new bugs.
Full Explanation
Public-health classifiers face evolving strains and shifting label distributions over time. Testing only on last year's strains measures past conditions, not temporal validity or out-of-distribution robustness for current municipal use. The auditor should emphasize temporal and out-of-distribution testing techniques so municipal AI is validated beyond outdated cohorts—time-based holdouts, newer strain panels, and monitoring for distribution shift after deployment.
Refreshing the strain test set on an annual fixed schedule to preserve year-over-year comparability for reporting still leaves the model unvalidated against new strains that emerge mid-cycle, which is exactly the temporal gap the audit is challenging. Monitoring distribution shift by comparing current predictions to historical performance is a useful production signal, but it reacts to drift after deployment rather than giving the temporal and out-of-distribution test design the audit is asking the auditor to require before relying on the model. Validating the classifier with k-fold cross-validation on last year's strains to confirm stable accuracy fails as metric gaming that hides hard-case performance when the city needs reliable detection most.
Exam caveat: temporal validity and OOD testing are named AI testing techniques when conditions evolve—classic health and civic exam territory. Operational check: confirm test design includes time-based splits or newer-strain evaluation, plus a plan for post-deploy shift detection—not a static prior-year accuracy claim presented as enduring readiness.