A water utility ML audit uses only synthetic test data, and management challenges whether results reflect production risk. How should the auditor respond?

Evaluating whether synthetic evidence is appropriate versus production-like data is the right response when management challenges relevance of water-utility ML tests. Synthetic sets can be valuable for rare-event coverage or privacy, but they may omit sensor noise, seasonal load, and operational edge cases that drive real plant risk. Auditors should document why the chosen data represent the risk being tested or supplement with production-like samples.

Validating synthetic data anonymization controls for privacy compliance purposes misses the point: management is challenging whether the evidence reflects production risk, not whether privacy controls are in place. Documenting synthetic dataset lineage to support reproducibility of test results misses the point: lineage supports reproducibility, but it does not establish that synthetic evidence represents production risk or answers management's relevance challenge. Comparing overall accuracy metrics across synthetic batches to confirm stable performance misses the point: internal stability across synthetic batches does not establish that the results reflect production risk or answer management's relevance challenge.

Exam caveat: production samples raise privacy, safety, and change-window constraints; jumping to production without controls can create operational risk even when relevance improves. Operational check: compare feature distributions and rare-event rates between synthetic and production telemetry, document gaps, and adjust the water AI test design or explicitly limit conclusions where synthetic-only evidence remains.