A financial regulator issues guidance stating that AI models used in credit underwriting must be 'explainable' to regulators on request. A bank uses a deep learning model for underwriting and argues that SHAP explanations satisfy this requirement. The regulator disagrees. What is the regulator's MOST likely concern with SHAP-based explanations?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — b is correct because regulators' concern with post-hoc explainability methods like SHAP is that they are approximations of model behavior, not the actual decision logic. The model's internal computations remain opaque; SHAP approximates what the model 'would have done' with different features but does not provide a causal or mechanistic account of the decision.
Full explanation below image
Full Explanation
B is correct because regulators' concern with post-hoc explainability methods like SHAP is that they are approximations of model behavior, not the actual decision logic. The model's internal computations remain opaque; SHAP approximates what the model 'would have done' with different features but does not provide a causal or mechanistic account of the decision. For regulatory examination, this opacity may be insufficient. Speed concerns (A) are practical but not the primary regulatory objection. Training data access (C) is not required for SHAP. SHAP is applicable to neural networks (D), so this is factually incorrect.