City needs a 311 text-classifier inference service on Kubernetes. How should the workload be deployed?
Select an answer to reveal the explanation.
Short Explanation
Always-on 311 inference wants a Deployment with GPU limits and HPA watching the queue—like staffing extra windows when the line grows. One-shot Jobs and SSH babysitting do not keep civic classifiers warm.
Full Explanation
A long-running 311 text-classifier belongs as a Kubernetes Deployment with explicit GPU resource limits and a Horizontal Pod Autoscaler (HPA) keyed to queue depth (or an equivalent custom metric). The Deployment keeps desired replicas continuously reconciled; GPU limits reserve accelerators; HPA adds replicas when the municipal ticket queue grows and sheds them when quiet, protecting both latency and budget.
A one-shot Job with no GPU limits and no scaling policy exits after completion—fine for batch, wrong for always-on inference. Interactive SSH sessions on random nodes bypass Schedulers, RBAC, health checks, and autoscaling; they are unsupportable for civic SLOs. A CronJob that deletes itself every minute creates constant cold starts and gaps in service instead of a stable endpoint.
Exam caveat: HPA on queue depth needs a metrics pipeline (custom/external metrics); CPU-only HPA may not track inference backlog well. Set GPU requests/limits correctly for the device plugin resource name, and use PDBs so scale-down or drains do not drop every replica. Readiness probes should fail until the model is loaded so Service endpoints do not black-hole traffic.
Operational check: deploy with GPU requests, confirm the Service only lists Ready pods, generate synthetic queue depth to watch HPA scale out, then drain depth and confirm scale-in. Verify 311 p95 latency stays within the civic target across the swing.