A public-health GenAI summarizer may invent citations that staff could treat as authoritative. What ethical concern should the auditor raise?
Select an answer to reveal the explanation.
Short Explanation
Fluent does not mean true — made-up citations can send health staff down the wrong path. Put accuracy checks and human review in front of anything that steers civic health decisions. Don't let a fluency score or a vendor's assurance talk you into skipping citation checks.
Full Explanation
A public-health GenAI summarizer that may invent citations staff could treat as authoritative raises accuracy and harm concerns because fluent false references can misdirect clinical or population-health decisions. The auditor should require controls for accuracy, human review, and harm prevention when AI informs civic health decisions: constrain authoritative use, verify citations against trusted sources, escalate uncertain outputs, and keep qualified humans accountable for decisions that affect residents. Treating automated fluency metrics as adequate evidence of citation accuracy for health summaries fails by concept: fluency measures how smoothly text reads, not whether cited sources exist or are accurate, so fabricated references can pass a fluency check and still mislead health staff. Accepting vendor-provided accuracy assurances in place of independent review of cited health sources fails by concept: a vendor's claims are not independent evidence, and citation accuracy must be verified against trusted sources before AI outputs inform civic health decisions. Reducing human review to spot-checks once the summarizer clears a 95% validation-set accuracy benchmark fails by concept: a benchmark measured on known validation data does not bound hallucinated citations in live operational use, so the point of highest persuasive risk still needs full review, not sampling. Exam caveat: AAIA ethics items on generative civic health AI emphasize truthfulness safeguards and human review—not unconstrained reliance on fluent but unverified outputs. Operational check: define prohibited and allowed use cases for the summarizer, require citation verification before operational use, log human reviews, sample outputs for fabricated references, and update the public-health AI risk assessment with residual hallucination controls.