A tool-library late-notice writer wants to send a new foundation model to every branch because a demo sounded nice. No one has scored it against real overdue letters. What should the team insist on?
Select an answer to reveal the explanation.
Short Explanation
Think of tasting the late-notice letters before you mail them to every branch. Evaluation sits after customization and before production. A polished demo is not a score, and it does not fine-tune the weights.
Full Explanation
In the foundation-model lifecycle, evaluation sits after customization and before production. The tool-library late-notice writer must score the customized model on real overdue letters—measuring tone, correctness, and policy fit—before rolling it to every branch. A demo that merely 'sounded nice' is not that evaluation step.
Immediate deploy to every branch because a polished demo replaces evaluation skips the gated score on real work product and treats theater as evidence. Skipping evaluation and starting with live monitoring only pushes first discovery of failure onto real borrowers; monitoring is necessary after launch but does not replace pre-production evaluation. Treating the demo as fine-tuning is incorrect: sounding fluent in a walkthrough does not update weights, and fine-tuning is a training adaptation, not a substitute for evaluation.
Exam caveat: demos prove feasibility of a conversation; evaluation proves readiness on representative tasks with defined metrics before production exposure.
Operational check: define a scored sample of real overdue letters with acceptance thresholds (policy accuracy, required fields, tone), run the customized model blind against that set, and withhold branch-wide deploy until the thresholds pass.