A larger model improves appeals accuracy but misses the city’s p95 latency SLO. What should the team do?
Select an answer to reveal the explanation.
Short Explanation
Bigger isn’t always better if residents hang up waiting. Pick a size—or a routed setup—that hits appeal accuracy without missing the p95 latency promise. Ignoring the SLO or putting the giant model on every hello burns trust and budget.
Full Explanation
Model-size and parameter selection under municipal constraints must satisfy both task accuracy and latency SLOs such as p95. When a larger model improves appeals accuracy but misses the city’s p95 latency SLO, the team should choose size and parameters that meet both needs—possibly routing appeals to the larger model while keeping lighter models for simpler turns. Production civic agents are multi-constraint systems; accuracy gains that systematically breach published latency are not shippable as-is for resident-facing paths.
Shipping the large model only and ignoring the published latency SLO treats the SLO as optional theater. Using the large model for every greeting and FAQ to maximize spend wastes capacity on low-stakes turns. Abandoning accuracy metrics because latency is the only civic concern flips the false dichotomy the other way—appeals quality still matters for due process.
Exam caveat: right-size or route by task so both accuracy and p95 latency constraints are met; the exam expects recognition of the dual constraint, not sacrifice of one for absolute pursuit of the other. Operational check: measure appeals accuracy and p95 latency for candidate models and routing rules; confirm the selected design clears both gates on a representative mix; document which traffic classes use which model.