An authorized test shows a 311 bot's ticket-priority wording can be steered by untrusted user text that the model treats as instructions. What risk class does this illustrate?
Select an answer to reveal the explanation.
Short Explanation
When resident chat starts rewriting how urgent a ticket sounds, that's prompt injection steering the workflow—not a model hallucination inventing unsupported case details. Spot the class: untrusted text acting like instructions over civic outputs.
Full Explanation
An authorized test showing a 311 bot’s ticket-priority wording can be steered by untrusted user text that the model treats as instructions illustrates a prompt-injection risk class that can distort civic workflow outputs such as ticket priority language. Untrusted content is interpreted as control rather than data, so priority labels, routing hints, or urgency phrasing may no longer reflect municipal policy. Correct classification drives defenses: input segregation, instruction hierarchy, output validation against allowlisted priority schemas, and monitoring for anomalous priority changes.
A training-data poisoning risk class that plants hidden triggers during large-scale model pretraining cycles fails conceptually: that is a pretraining integrity issue, not LLM instruction steering at inference time. A sensitive-data exposure risk class that reveals internal system prompts in verbose error output logs fails as an output-logging issue orthogonal to prompt interpretation. A model hallucination risk class that invents unsupported details when summarizing resident case records fails as generation inaccuracy unrelated to an attacker steering ticket priority via untrusted instructions.
Exam caveat: When untrusted text alters workflow artifacts (priority, routing, ticket fields), label it prompt injection (or direct injection), not training-data poisoning, sensitive-data exposure, or model hallucination classes.
Operational check: reproduce steering under RoE with benign markers; verify priority outputs against a schema validator; implement filters that strip or quarantine instruction-like user content before prioritization; and alert when ticket priority language diverges from policy templates.