County IT needs high-performance LLM inference behind the 311 agent orchestrator. Which deployment choice best matches NVIDIA platform practice?
Select an answer to reveal the explanation.
Short Explanation
311 needs a proper inference door, not a laptop under someone’s desk. Deploying the LLM as an NVIDIA NIM microservice behind the orchestrator is the platform-shaped move.
Full Explanation
County IT needing high-performance LLM inference behind a 311 agent orchestrator should deploy the LLM as an NVIDIA NIM microservice. NIM packages models as scalable, operable inference microservices that sit cleanly behind orchestrators with service boundaries, health endpoints, and production-oriented deployment patterns. That matches NVIDIA platform practice for municipal contact-center and 311 agent stacks that must serve concurrent resident traffic with monitoring and isolation—not ad-hoc local processes.
Running the LLM only on a clerk’s laptop with no service boundary fails because laptops are not HA inference tiers, lack municipal multi-tenant controls, and create single-point and data-handling risks. Replacing inference with a static FAQ HTML page that never calls a model fails the requirement for LLM inference and cannot handle free-form 311 language. Hosting the model as an unmonitored desktop binary shared via USB fails operationally and from a security standpoint: no patch cadence, no central auth, no observability, and no orchestrator integration.
Exam caveat: choosing NIM is the deployment shape for inference; it does not by itself define RAG corpora, guardrails, or tool permissions. Those remain separate civic architecture layers that the orchestrator and safety stack must still implement.
Operational check: confirm the 311 orchestrator calls the NIM service endpoint (not a desktop process), that authentication and network policies restrict access, that health/metrics are scraped, and that a canary or load test meets the city’s latency and error budget before full traffic.