A financial institution adopts a large language model to assist compliance analysts in reviewing suspicious activity reports (SARs). The model flags potential omissions in analyst-drafted SARs. Six months after deployment, analysts report they trust the model's flags completely and no longer apply independent judgment. What emerging AI risk has materialized?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — you're seeing automation bias and over-reliance because analysts stopped using independent judgment and let the model's flags drive the review. That defeats the human-in-the-loop control, so if the model misses or misflags something, nobody catches it.
Full Explanation
The correct risk is automation bias and over-reliance: human analysts have deskilled and transferred their judgment to the AI, undermining the human oversight that gives the control its value and creating risk if the AI errs without correction. The model was intended to assist human judgment, not replace it; when humans defer completely to the AI, errors in the AI's flags go uncorrected, and the compliance control is undermined. The model-drift option is possible but not the risk described—the scenario describes human behavioral change, not model performance change. The regulatory-risk option is incorrect; AI assistance in compliance review is not prohibited, though it requires human oversight. The data-quality option is a separate concern; the scenario describes analyst behavior change rather than training data no longer reflecting current typologies of suspicious activity.