A risk analyst asks a model developer to explain why the AI rejected a specific insurance claim. The developer says the model is a 'black box' and individual predictions cannot be explained. From a governance perspective, what is the MOST appropriate organizational response?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — if a model rejects an individual claim, you can't just call it a black box and move on. Governance rules, including GDPR Article 22, require decision-level explanations, so require explainability before deployment or keep the model to aggregate analytics.
Full Explanation
Governance frameworks — including regulatory requirements, GDPR Article 22, and ethical AI principles — require AI systems affecting individuals to provide decision-level explanations. The most appropriate response is to require explainability as a deployment prerequisite, or restrict model use to contexts not requiring individual explanations. Accepting the limitation as inherent abdicates governance responsibility. Hiring a vendor to build an explanation layer after deployment may help, but explainability should be required before deployment. Informing claimants that explanations are not technically possible violates accountability principles in most jurisdictions.