A transit vision model accuracy is reported without confidence intervals or class imbalance analysis. What should the auditor recommend?

Transit vision models used near safety-adjacent operations need statistically rigorous test reporting. A single accuracy percentage without confidence intervals or class-imbalance analysis can hide uncertainty and weak performance on rare but critical classes, such as specific hazards or uncommon events. The auditor should recommend confidence intervals, imbalance-aware metrics, and class-level results as part of AI test evidence for civic readiness.

Validating accuracy on a larger held-out test set using stratified sampling still fails because a bigger sample alone does not supply confidence intervals or class-imbalance analysis, and deployment approval should not rest on a point estimate without those safeguards. Reporting class-balanced accuracy only for the most common hazard categories, while excluding rare event classes from the metrics, hides exactly the low-frequency, high-consequence misses that class-imbalance analysis exists to catch. Supplementing accuracy with precision-recall curves and latency benchmarks fails by substituting operational metrics for the missing statistical rigor, since neither curves nor latency benchmarks provide confidence intervals or class-imbalance analysis for municipal oversight.

Exam caveat: AAIA testing quality includes statistical rigor of metrics, not merely validating on a larger held-out set or appending precision-recall curves and latency benchmarks while omitting confidence intervals and class-imbalance analysis. Operational check: require per-class metrics, documented imbalance treatment, and interval estimates in the transit vision test report before treating the model as operations-ready for live cameras.