An insurance company deployed an AI claims triage tool three months ago without first measuring how long claims took to process beforehand. An executive now announces the tool made claims processing "40% faster." What is the most important flaw in this claim?
Select an answer to reveal the explanation.
Short Explanation
Think of it like this: saying a tool made you "40% faster" only means something if you know how fast you were before. Without a documented pre-deployment baseline, that 40 percent has nothing solid to compare against — it could be an estimate, a different time period, or noise from a slow month. The fix isn't a bigger number or a longer wait, it's a real starting line.
Full Explanation
The core problem is measurement, not the size of the number, the choice of metric, the timing, or approvals. Without a documented pre-deployment baseline for processing time, the "40% faster" figure has nothing verified to compare against; it could be an estimate, a comparison to a different period, or influenced by unrelated changes like staffing or claim mix. The distractor dismissing processing time as an invalid KPI is wrong because processing time is actually a reasonable, directly relevant metric for a triage tool, even though other factors also influence it. The distractor demanding a full year before reporting is wrong because the issue is the missing reference point, not the elapsed time before reporting; a short window with a solid baseline is more credible than a long window without one. The distractor requiring compliance sign-off is wrong because approval process, while potentially good practice, doesn't address whether the underlying measurement is valid. The general principle: a results claim is only as credible as the baseline it's measured against.