A team deploys a new fraud model using canary deployment with 15% of traffic. The champion continues receiving 85%. After 2 weeks, the canary achieves statistically significantly better precision. What distinguishes canary deployment from A/B testing in this context?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — canary deployment is built for progressive rollout: you start with a small traffic share and promote it gradually when it meets the promotion criterion. The controlled split experiment keeps a fixed split for the whole test window, so it doesn't automatically ramp the new model to full traffic.
Full Explanation
Canary deployment is designed for progressive rollout: the canary starts at a small traffic fraction and is gradually promoted to 100% if it meets performance thresholds. The fixed-split experiment maintains a fixed split for statistical comparison over a defined experiment window, with no automatic gradual promotion. The claim that both models make real decisions for all users simultaneously is wrong; canary routing is mutually exclusive per request. The claim that the fixed-split experiment requires randomized assignment while canary routes by request order is wrong; both can use randomized routing. The claim that canary only logs the new model's predictions without acting on them describes shadow deployment, not canary.