An AI system for automated email phishing detection incorrectly classifies legitimate vendor invoices as phishing attempts at a rate of 12%, causing operational disruptions. The security team defends the high false positive rate as necessary to maintain the system's 99.2% detection rate for actual phishing. How should the AI risk manager evaluate this trade-off?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — you have to put real cost on those false positives, not just lean on the 99.2% detection rate. If missed invoices, vendor pain, and staff time outweigh the phishing benefit, you tune the model or add tiered review for a better balance.
Full Explanation
The correct approach is to quantify the operational risk from false positives—missed legitimate invoices, vendor relationship impact, and employee time—against the security benefit, then evaluate whether model tuning or tiered review can achieve a better balance. The trade-off between false positives, which create operational disruption, and false negatives, which mean missed phishing, must be explicitly evaluated in terms of actual risk costs. A 12% false positive rate on vendor invoices may create significant payment delays, vendor disputes, and employee burden that could exceed the phishing risk benefit. Risk managers must quantify both sides and seek a more favorable operating point. Accepting the 12% false positive rate as reasonable given the high detection rate is insufficient because it accepts the status quo without analysis. Setting a threshold with zero false positives regardless of detection impact is wrong because it would require such conservative detection that actual phishing would not be caught. Replacing the automated system with human-only review to eliminate false positives is wrong because it is disproportionate and introduces its own accuracy and scalability issues.