After a prompt change to a filings Q&A bot, release process should:

Question

Accepted Answer

Run regression evals on golden Q&A pairs for faithfulness/numeric accuracy, canary, then rollout. — Prompt change → golden Q&A regression, canary, rollout. Latency-only is not enough.

Answer

Ship to all users immediately without tests.

Answer

Only check response latency.

Answer

Delete the golden set to save storage.

After a prompt change to a filings Q&A bot, release process should:

Short Explanation and Infographic

Full Explanation