Field inspectors review agent-drafted violation notices. How should their feedback improve the agent?
Select an answer to reveal the explanation.
Short Explanation
When inspectors override a draft, that’s gold—not noise. Agree/override marks that feed training signals turn field expertise into a structured improvement loop.
Full Explanation
Field inspectors reviewing agent-drafted violation notices should mark agree/override on drafts and use overrides as training (or evaluation) signals. Agree/override captures expert judgment at the point of work: when inspectors correct legal language, code sections, or severity, those overrides become labeled feedback for improving the drafting model, prompts, or retrieval pack. Structured inspector disagreement is a high-value municipal learning loop—not noise.
Never recording overrides so the agent cannot learn from experts fails by concept because it throws away the scarce labeled signal that field expertise provides. Auto-accepting every agent draft and banning inspector edits fails because it removes human quality control on enforcement documents that carry legal weight. Treating inspector disagreement as noise and discarding it fails because systematic disagreement patterns reveal model failure modes that averages miss.
Exam caveat: NCP-AAI improvement-loop items value expert override capture for domain drafts (inspections, notices). Prefer agree/override → training/eval signals over no logging, ban-edits auto-accept, or discarding disagreement.
Operational check: add agree/override controls on the notice review UI; store override diffs with inspector ID and code references; sample overrides into weekly eval/fine-tune sets; measure override rate trending down on repeated notice types after model updates.