Model Deployment
NCP-GENL · 18 questions
- A county needs fast ordinance retrieval-only search under a tight VRAM budget. Which model-type choice fits?
- Resident complaint translation needs bilingual rewrite quality, while FAQ chat must stay on a slim GPU. How should architecture be split?
- Public-health field tablets are memory-tight but need outbreak label triage. Which tradeoff is sound?
- City hall needs long-form meeting-minute summaries and short 311 chat replies under different latency SLAs. Which pairing fits?
- The assessor similarity-search service must stay under a hard GPU memory ceiling. Which decision is correct?
- Emergency bilingual alert rewrite produces dangerous errors on a tiny decoder but is accurate on a costlier encoder-decoder. What should leadership accept?
- Permitting LLM traffic spikes at noon. How should the city package and serve inference?
- The library needs catalog embeddings plus a generative reranker in one serving graph. Which approach fits?
- Transit must meet a p95 latency SLA for rider queries while still using dynamic batching. What should ops tune?
- Housing and city-hall inference must match clinic staging behavior. What container practice is essential?
- Public works has one GPU for a small ticket classifier and a large work-order summarizer. How can Triton help?
- Clerk filing assistants need tokenization before the LLM backend. How should the containerized pipeline be structured?
- Building-season peaks deepen the permitting inference queue. Which Kubernetes control fits?
- A school lab must run the same NIM-style LLM image used in staging. What approach is appropriate?
- Ordinance answers must retrieve, generate, then schema-validate before residents see them. How should serving be managed?
- The 311 assistant runs on a Kubernetes serving mesh. What live monitoring should ops implement with serving?
- Water-utility Dockerized LLM pods must not schedule onto CPU-only workers. What Kubernetes control fits?
- Courts are rolling out a new clerk LLM on Kubernetes while most replicas must stay on the proven build. Which serving practice best limits blast radius?