County counsel warns that a courts GenAI assistant could expose sealed case notes if hostile prompts manipulate it. Which concern is counsel correctly highlighting?
Select an answer to reveal the explanation.
Short Explanation
Think of a crafty question that tricks a clerk into opening a sealed drawer—prompt injection plays a similar game with GenAI. Courts assistants that can touch sensitive notes need security concern on the ethics agenda, not a shrug.
Full Explanation
County counsel warning that a courts GenAI assistant could expose sealed case notes if hostile prompts manipulate it is correctly highlighting a security concern that prompt injection or similar abuse could leak sensitive justice data. Generative assistants that can retrieve or surface documents inherit prompt-based attack paths that classical apps handle differently.
Why the other options fail by concept: Claiming sealed case notes are safe to expose because GenAI cannot retrieve any documents assumes away the retrieval/tooling that makes assistants useful—and dangerous. Saying courts assistants have no security relevance because they only draft text ignores that drafted text can contain leaked content. Limiting prompt-related risks to public websites and never to internal justice tools underestimates insider and compromised-account threats on internal systems.
Exam caveat: Defenses are layered—instruction hierarchy, tool permissioning, output filtering, retrieval scoping, and human review—not a single magic prompt. Sealed material may need to be excluded from the retrievable corpus entirely rather than trusted to the model to “not reveal” it.
Operational check: Threat-model the courts assistant for prompt injection and data exfiltration; exclude sealed notes from searchable indexes or gate them behind strong authorization independent of the LLM; red-team with hostile prompts; log tool calls; and keep counsel sign-off on any expansion of document access for GenAI.