A senior engineer reviews the monitoring plan and notices that alerts are only set to fire after a model's error rate has already caused visible customer complaints. What is wrong with this design?
Select an answer to reveal the explanation.
Short Explanation
By the time customers are complaining, the damage is already done. You need thresholds that fire before that, not after. That's proactive monitoring, not a complaint-driven cleanup.
Full Explanation
The correct choice is the one stating that the thresholds are set too late to trigger review before the risk turns into operational damage, defeating the purpose of proactive monitoring. Waiting for visible customer harm means the threshold is set past the point where review could have prevented damage, which is the opposite of what proactive alerting is for. The statement that customer complaints are not a valid data source for any monitoring system is incorrect because customer feedback can be a legitimate input; the problem is the timing of the threshold, not the source. The statement that the model should be retrained immediately rather than monitored at all is incorrect because retraining without first understanding the cause skips the review step monitoring is meant to enable. The statement that alerts should never be based on error rate under any circumstances is incorrect because error rate is a reasonable metric to alert on; the issue is where the threshold is set. For the exam, evaluate thresholds by whether they trigger review early enough to prevent damage.