A red team has demonstrated that an attacker can embed hidden instructions inside a document that a user uploads to a Copilot-powered assistant, causing the assistant to perform unauthorized actions. What type of attack is this, and what Azure AI Content Safety feature mitigates it?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — b is correct because embedding malicious instructions inside documents that an AI assistant processes is called an indirect prompt injection attack (as opposed to a direct jailbreak in the user message). Azure AI Content Safety's Prompt Shield has a document analysis mode that scans documents and other grounding content for injected instructions before the AI processes them, mitigating this attack vector.
Full explanation below image
Full Explanation
B is correct because embedding malicious instructions inside documents that an AI assistant processes is called an indirect prompt injection attack (as opposed to a direct jailbreak in the user message). Azure AI Content Safety's Prompt Shield has a document analysis mode that scans documents and other grounding content for injected instructions before the AI processes them, mitigating this attack vector. A is wrong because jailbreak attacks are direct user-message attacks, and Groundedness detection checks factual accuracy, not injection. C is wrong because data poisoning targets training data, and Custom categories detect specific content terms. D is wrong because model extraction attempts to steal model weights or outputs, and Protected material detection checks for copyright infringement.