A school-district tutoring assistant ships prompt and model updates without re-running its regression suite. What should the auditor recommend?
Select an answer to reveal the explanation.
Short Explanation
Changing the prompt or model is like swapping the engine and hoping the car still passes inspection. Regression suites catch tutoring quality slips before students see them. Relying on load and latency validation alone is too late to catch tutoring quality problems before students see them.
Full Explanation
Requiring regression testing after material prompt or model changes before production release emphasizes change discipline for civic GenAI that tutors students. The correct approach works because prompt and model updates can reintroduce known failure modes, alter safety boundaries, or degrade tutoring quality even when the surrounding application code is unchanged. A regression suite that checks tutoring content outcomes, refusal behavior, and critical curriculum pathways provides evidence that student-facing quality remains within acceptance thresholds before the school district ships the change.
Requiring load and latency validation after model changes before production release fails because performance checks do not verify tutoring content correctness, safety, or regression behavior, and they cannot substitute for a regression suite that catches quality slips before release.
Requiring A/B comparison of prompt variants with live student cohorts before full rollout fails because exposing students to unvalidated prompt variants creates preventable harm and is not a substitute for pre-release regression checks on tutoring content and safety.
Requiring benchmark scoring on public academic datasets after prompt changes before production release fails because public benchmarks may not reflect the district's specific curriculum, safety policies, or tutoring pathways, and they do not replace a regression suite tied to the assistant's actual student-facing behavior.
Exam caveat: regression suites must be maintained as prompts and tools evolve; stale cases can give false confidence after architecture changes such as new retrieval sources. Operational check: tie material prompt/model promotions to a gate that records suite version, pass/fail results, known flaky cases, and approver identity before production release of the district tutoring assistant.