After fine-tuning, a permitting bot begins emitting internal review checklists to public applicants. What customization side effect should security investigate?
Select an answer to reveal the explanation.
Short Explanation
When the public bot starts handing out the inspectors' private checklist, the fine-tune taught too much shop talk. That's a regression side effect—pull it, test it, and scrub the training crumbs that leaked process.
Full Explanation
After fine-tuning, a permitting bot that begins emitting internal review checklists to public applicants exhibits a customization side effect: capability or regression leakage of internal civic process content that should have remained staff-only. Security should investigate which training documents or labels taught those checklists, whether evaluation missed the disclosure, and how to roll back or remediate the checkpoint before more applicants receive internal procedures.
This is not a beneficial transparency feature to leave unchanged when the content is internal process control. It is not proof that fine-tuning cannot memorize internal documents—it is evidence of the opposite. It is not an unavoidable property of all temperature settings above zero; temperature affects sampling randomness, not whether internal SOPs were present in the fine-tune corpus.
Exam caveat: Post-customization leakage of internal municipal process is a memorization and data-minimization failure, not an inference-temperature inevitability. Distinguish intentional public transparency programs from accidental exposure of staff checklists.
Operational check: Pull the bot from public serving or add hard filters for checklist markers, inventory training sources for internal permitting SOPs, scrub or segregate those docs, retrain or revert to the last clean checkpoint, and add eval cases that specifically probe for internal checklist emission before re-release to applicants.