A bank celebrates a model KPI that counts how many campaign emails were sent. The decision the model is meant to support is which customers to offer a fee waiver. What is wrong with that KPI?
Select an answer to reveal the explanation.
Short Explanation
Counting emails is like judging a kitchen by how many menus it printed. You need a KPI that measures the waiver decision, not a vanity send count. Training accuracy is a model score, and a rising arrow on a dashboard still does not show whether the right customers were offered the waiver.
Full Explanation
A KPI used to judge this model has to measure the decision the model supports. That decision is which customers to offer a fee waiver, so the measure needs to reflect waiver offers and their outcomes, not how many campaign emails happened to go out. An email count can rise because a blast list grew, even when the model did not change who should receive a waiver, which makes it a vanity volume. Treating the send count as proof that waiver offers reached the right customers confuses a mailing total with the selection decision. Training accuracy summarizes fit on historical labels and does not record whether the waiver decision was a good one in operation. Placing the same count on a dashboard beside a rising arrow changes the decoration, not the thing being measured. Exam caveat: a volume count can be an operations metric for the mail system and still be the wrong KPI for the waiver decision. Operational check: write the decision as a customer-level waiver offer, pick a measure tied to that offer and its result, and stop using email volume as the model's success number.