After applying a LoRA adapter, a tax bot invents rebate amounts. What impact assessment action is appropriate?
Select an answer to reveal the explanation.
Short Explanation
When your tax bot starts inventing rebate amounts after a LoRA update, measure hallucination rates before and after, and roll back if they worsen. Pre/post checks help you see whether the adapter improved or harmed factual answers.
Full Explanation
After applying a LoRA adapter, a tax bot that invents rebate amounts calls for measuring hallucination rates before and after the change and rolling back the adapter if the rate worsens. PEFT can improve style or FAQ coverage while degrading factual money amounts, so an impact assessment should quantify that change on a fixed set of rebate and factual probes and treat a regression as a reason not to keep the adapter in use. Rolling back restores the prior baseline while the impact is addressed. Ignoring hallucination rate on the assumption that LoRA cannot affect factuality fails because LoRA changes the model’s effective behavior and can increase fabrications. Shipping the adapter faster to overwrite citizen complaints fails as risk management: speed without measurement can expose more taxpayers to incorrect rebate figures. Raising sampling temperature so fabricated amounts sound more confident also fails; higher temperature typically increases randomness and can make fiscal answers less trustworthy. Exam caveat: the pre/post comparison is meaningful only when prompts, decoding settings, and graders are identical, or evaluation drift may confound the measured impact. Practical check: score hallucination rate and amount errors on a fixed tax-rebate probe set for the base model and the model with the adapter; if fabrications rise, roll back the adapter.