A housing authority wants simple resident FAQ traffic to avoid an expensive foundation model while still using that stronger model for nuanced case drafts. Which deployment approach best meets the cost and performance goal?
Select an answer to reveal the explanation.
Short Explanation
Housing FAQ is the drive-through window; nuanced case drafts are the sit-down consultation. Cascading at the API layer sends the easy stuff to the cheap lane and reserves the pricey foundation model for work that actually needs it.
Full Explanation
API-based model cascading lets a housing authority inspect request intent and send simple resident FAQ traffic down a cheaper model path while reserving an expensive foundation model for nuanced case drafts. That deployment approach directly hits the cost and performance goal: high-volume FAQ stays inexpensive and fast, and only complex drafting pays for stronger generation. Intent or content-based routing at the API layer is the control point that makes the split enforceable for civic self-service and casework systems. Forcing every FAQ and case draft through the most expensive foundation model fails the cost goal even if branding feels consistent—uniform premium inference is not a housing-budget strategy for routine FAQ. Training a classical regression model on eviction dates and removing generative models fails by concept: regression on dates does not answer FAQ prose or draft nuanced case letters, so it abandons the GenAI capability the authority still needs for drafts. Caching a single static letter template and disabling runtime model selection fails for nuanced cases that require generative drafting; a static template cannot adapt to case facts the way a stronger FM path can. Exam caveat: cascading is only as good as intent classification. Misrouting a sensitive counseling draft to the cheap FAQ path is a quality and fairness risk; misrouting FAQ to the expensive path wastes budget. Keep human review on high-stakes housing correspondence. Operational check: Define FAQ versus case-draft routing rules; measure cost per successful FAQ versus draft; add Guardrails on both paths; and sample outputs weekly with housing caseworkers to confirm the cheap path is not silently handling counseling-class content.