Before resident traffic reaches a new NIM deployment, platform ops must know the microservice is healthy. What should be wired?
Select an answer to reveal the explanation.
Short Explanation
Don’t open city hall’s doors until the lights are on. Wiring NIM health endpoints into Kubernetes probes keeps broken inference out of resident traffic.
Full Explanation
Before resident traffic reaches a new NIM deployment, platform ops must know the microservice is healthy. The correct practice is to wire NIM health endpoints into Kubernetes probes before accepting resident traffic. Readiness and liveness probes gate whether pods receive Service traffic and whether unhealthy instances are restarted, preventing broken inference from becoming the city’s public 311 brain.
Shipping traffic immediately and discovering failures only from resident complaints fails as an ops model—it makes the public the monitoring system. Disabling health checks to reduce Kubernetes API noise fails by removing the control that protects availability. Marking every pod Ready permanently regardless of NIM status fails because it lies to the scheduler and load balancer, sending civic requests to non-serving processes.
Exam caveat: health probes confirm process and endpoint liveness/readiness; they do not prove answer quality, guardrail correctness, or RAG faithfulness. Pair probes with smoke tests and eval gates in the release pipeline so “Ready” does not mean “safe and accurate for residents.”
Operational check: deliberately break or delay the NIM health endpoint in staging and confirm Kubernetes removes the pod from Service endpoints; measure startup probe timing so slow model loads do not flap; only then open the ingress path used by the resident agent orchestrator.