A mobile app team is adding an FAQ-answering feature that must respond within a few hundred milliseconds on a limited compute budget, and the questions it needs to answer are narrow and repetitive, drawn from a small, well-defined set of app features. The Foundry model catalog offers both a large flagship model known for handling complex, open-ended reasoning and a smaller, lightweight model optimized for fast, low-cost inference on simpler tasks. Which model is the more appropriate choice for this FAQ feature?
Select an answer to reveal the explanation.
Short Explanation
You don't need the biggest model for this job. The FAQ questions are narrow, repetitive, and come from a small set of app features, so the smaller, lightweight model is the better fit because it meets the tight latency and cost limits without wasting effort on advanced reasoning. Bigger isn't automatically better here.
Full Explanation
The appropriate choice is the smaller, lightweight model, because the FAQ feature has two defining constraints: a tight latency budget and a narrow, repetitive question set drawn from a small, well-defined set of app features. The lightweight model is built for that profile, offering fast, low-cost inference that is well suited to simpler, well-scoped tasks, so it can meet the response-time requirement without paying for reasoning capability the task does not need. The claim that the large flagship model is better because higher-capability models always produce more accurate answers regardless of task complexity is incorrect; greater capability does not translate into better performance on every task, and for narrow, repetitive questions the flagship model's added reasoning strength brings no meaningful accuracy benefit while likely costing more latency and money, which works against the stated constraints. The claim that the large flagship model is required because only flagship models are eligible for deployment on mobile-facing applications is also incorrect; there is no rule tying flagship-only models to mobile-facing deployments, and model choice depends on the task's requirements, not the client platform consuming the API. Finally, the claim that the smaller, lightweight model is acceptable only if content safety filtering is disabled to compensate for its reduced capability is incorrect; model size and content safety filtering are unrelated settings, disabling filtering does not compensate for anything about model capability, and it would only remove a protection the deployment should keep regardless of which model is chosen.