A company implements a generative AI chatbot for customer service. Users discover they can override the chatbot's safety guidelines by embedding instructions in customer support tickets. What attack vector does this represent?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — you're seeing indirect prompt injection because attacker-controlled text in customer tickets gets processed by the LLM and overrides its original instructions. The attack comes through legitimate-seeming input data, not through the training set.
Full Explanation
Indirect prompt injection via untrusted content in the model's context window is correct because attacker-controlled content, such as text in customer tickets, is processed by the LLM and overrides its original instructions. This is distinct from direct prompt injection because the attack vector is through legitimate-seeming input data. Data poisoning of the chatbot's fine-tuning dataset affects training, not inference. Model extraction through repeated API queries aims to replicate the model. Membership inference attack to recover training data targets training data recovery.