A financial institution adopts a large language model to assist compliance analysts in reviewing suspicious activity reports (SARs). The model flags potential omissions in analyst-drafted SARs. Six months after deployment, analysts report they trust the model's flags completely and no longer apply independent judgment. What emerging AI risk has materialized?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — b is correct because automation bias has caused analysts to stop applying independent judgment, defeating the purpose of the human-in-the-loop design. The model was intended to assist human judgment, not replace it; when humans defer completely to the AI, errors in the AI's flags go uncorrected, and the compliance control is undermined.
Full explanation below image
Full Explanation
B is correct because automation bias has caused analysts to stop applying independent judgment, defeating the purpose of the human-in-the-loop design. The model was intended to assist human judgment, not replace it; when humans defer completely to the AI, errors in the AI's flags go uncorrected, and the compliance control is undermined. A (model drift) is possible but not the risk described—the scenario describes human behavioral change, not model performance change. C is incorrect; AI assistance in compliance review is not prohibited, though it requires human oversight. D (data quality) is a separate concern; the scenario describes analyst behavior change.