A permit RAG agent needs both a chat LLM and an embedding model in production. How should NIM be used?
Select an answer to reveal the explanation.
Short Explanation
Chat brains and embedding brains aren’t the same appliance. Separate NIM services for the chat LLM and embedding models keep the permit RAG agent’s inference roles clean.
Full Explanation
A permit RAG agent needs both a generative chat LLM and an embedding model. Correct NIM usage deploys separate NIM services for the chat LLM and the embedding models so each role has the right model, scaling, and lifecycle. Embeddings power retrieval over ordinances and application guides; the chat NIM consumes retrieved context to answer. Separating services avoids modality confusion, lets ops scale embedding throughput independently of chat concurrency, and matches how production RAG systems are operated on NVIDIA platforms.
Forcing one NIM container to pretentiously serve every modality with no role separation fails because it blurs operational ownership and often mismatches model capabilities and batching characteristics. Skipping embeddings and hard-coding ten ordinance paragraphs into the system prompt fails as soon as the corpus grows or ordinances update—prompt stuffing is not RAG. Calling a single chat NIM for embeddings by asking it to “imagine vectors” fails conceptually: chat generation is not a substitute for a real embedding model’s vector space used by the vector database.
Exam caveat: separate NIM services still must share compatible operational governance—version pins, network policy, and evaluation. Embedding model changes can silently invalidate an index; chat model changes can alter answer style without fixing retrieval.
Operational check: verify two distinct NIM endpoints in the permit stack, rebuild or re-embed when the embedding NIM version changes, and score RAG faithfulness on a municipal ordinance golden set after either service is upgraded.