City MLOps stores prompt revisions for the permit agent. When should a new prompt version auto-promote to production?
Select an answer to reveal the explanation.
Short Explanation
Prompts are production artifacts, not midnight vibes. Auto-promote only after the city’s MLOps store says eval thresholds passed—then you can tune and version with a safety net.
Full Explanation
Prompt revisions for a municipal permit agent are production configuration, not informal notes. City MLOps should store each prompt version and auto-promote a new revision to production only after predefined evaluation thresholds pass in that store—covering permit accuracy, citation fidelity, refusal behavior, and regression suites tied to ordinance retrieval. Automatic promotion after green evals keeps iteration fast while preserving a safety net: every promoted prompt is tied to evidence, and failed thresholds keep risky wording out of resident-facing permitting flows.
Auto-promoting every midnight regardless of eval results fails by concept because schedule-driven release ignores quality gates and can ship regressions overnight into live permit guidance. Auto-promoting when a developer tweets that the prompt “feels better” fails because subjective social signals are not reproducible acceptance criteria and leave no auditable eval record for municipal change control. Never versioning prompts fails because without versions there is no rollback target, no diff between revisions, and no way to attach gates to a specific artifact—turning every change into an unrecoverable overwrite.
Exam caveat: NCP-AAI MLOps questions distinguish artifact lifecycle (version → evaluate → promote) from informal or calendar-based shipping. When the stem mentions stored prompt revisions and auto-promotion, the decisive control is eval-threshold gating in the MLOps store—not vibes, clocks, or unversioned edits.
Operational check: register the candidate prompt as a versioned artifact; run the city’s permit-agent eval suite against it; require threshold pass (and recorded results) before the promote pipeline updates the production pointer; keep the prior version pinned so ops can roll back within minutes if live permit metrics degrade.