A telecom company has classified an AI system that predicts network outages as "low risk" because it does not directly interact with customers. Leadership later learns the system's outputs are used to automatically prioritize which repair crews are dispatched first, affecting how quickly service is restored to hospitals and emergency responders. What should the company do?
Select an answer to reveal the explanation.
Short Explanation
Think of it like this: a tool that looked harmless because it never talks to customers can still hold a hospital's power in its hands once you trace where its output goes. Once leadership learns this system's calls affect how fast crews reach emergency responders, the sound move is to reclassify it to a higher risk tier with tighter oversight.
Full Explanation
Reclassifying the system to a higher risk tier reflects the newly understood downstream impact — its outputs affect how quickly service reaches hospitals and emergency responders, a consequence far more significant than the original "no direct customer interaction" framing suggested. Keeping the classification unchanged because there's still no direct customer interaction clings to an outdated criterion and ignores the real-world impact just discovered, which is exactly the kind of oversight a good risk framework is meant to catch. Reducing governance because the system runs automatically has it backwards — automation without human review generally warrants more oversight for high-impact systems, not less. Transferring the classification decision to the repair crew supervisors pushes an enterprise governance judgment down to operational staff who use the outputs but aren't positioned to own the risk framework. The general principle: risk classification must be reassessed when new information reveals greater downstream impact, and impact — not surface-level customer interaction — is what should drive the tier.