A city AI strategy office is designing how to inform residents interacting with a benefits chatbot about AI's role in the conversation. A staff member argues that telling residents they're chatting with an AI system satisfies the office's full responsible-AI obligation. What distinction should the office's leadership raise in response?
Select an answer to reveal the explanation.
Short Explanation
Think about the difference between a doctor telling you a resident is performing part of your exam versus actually asking whether you're okay with that. Telling someone an AI is involved is the disclosure step, but it doesn't automatically give them a real choice to talk to a human instead, and for something as consequential as benefits eligibility, residents may need both.
Full Explanation
Transparency and consent serve distinct responsible-AI functions: disclosure tells a resident an AI system is involved in the interaction, while consent gives the resident a meaningful choice, such as a genuine path to a human alternative, and depending on the stakes of the interaction, a benefits program may need to provide both rather than treating disclosure as a complete substitute for consent. Collapsing the two into one requirement means a resident could be informed an AI is present but have no real way to request human handling, which leaves them without agency over a high-stakes eligibility conversation. Claiming consent is unnecessary because residents can't opt out of government services generally conflates opting out of the benefit itself, which usually isn't optional, with opting out of the AI-mediated channel used to access that benefit, which is a narrower and more defensible ask. Arguing disclosure is unnecessary as long as an opt-out exists somewhere downplays that residents need to know AI is involved at the point of interaction in order to decide whether to exercise that opt-out at all; a buried option nobody knows exists doesn't function as real consent. A scope caveat: the appropriate mix of disclosure and consent can scale with how consequential the interaction is, a simple hours-of-operation chatbot may need lighter treatment than a benefits-eligibility conversation. A concrete check: confirm the chatbot interface both states AI involvement upfront and provides a clearly visible path to reach a human caseworker.