A library-programs agent's production monitoring flags a recurring pattern of low-quality answers, and the team adds those flagged transcripts to the golden dataset used for evaluation before redeploying an updated version. What practice does this represent?
Select an answer to reveal the explanation.
Short Explanation
Real mistakes make the best homework. Instead of only fixing today's bad answers, the team feeds them into the golden dataset so tomorrow's version gets tested against the exact failure patterns residents actually hit. That's turning production pain into a stronger safety net before the next release.
Full Explanation
Closing the loop between production and evaluation means transcripts that reveal real failure patterns in live traffic get folded into the golden dataset, so future evaluation runs specifically test against the kinds of mistakes the agent has actually made, making the evaluation suite progressively more representative of real resident interactions rather than staying static from launch; this strengthens confidence in subsequent releases before they reach production. A blue/green deployment describes running two parallel environments and cutting traffic over, not the practice of enriching an evaluation dataset. A principal access boundary update changes what resources the agent can access, unrelated to evaluation content. A rollback reverts to a prior version after a bad release, a different response than proactively strengthening the test set for the next one. Scope caveat: adding flagged transcripts to the golden dataset only helps if the underlying issue is actually addressed before redeploying, since testing against a known failure without fixing it just documents the same gap again. Operational check: after redeploying, confirm the updated version now passes evaluation against the newly added flagged transcripts before it goes back into production.