A school-district homework helper reveals its hidden system-prompt instructions after a student asks for them in a cleverly worded request. Which LLM attack surface does this demonstrate?
Select an answer to reveal the explanation.
Short Explanation
The kid talked the bot into reading its own private instructions out loud — classic system-prompt disclosure. Those hidden rules are part of the attack surface for civic LLM apps. Treat prompt text like sensitive config, not something the model should cheerfully reprint.
Full Explanation
System prompts commonly encode safety rules, tone, tool permissions, and municipal policy the application does not want users to see or rewrite. Eliciting those hidden instructions through cleverly worded student queries is system-prompt disclosure: an LLM application attack surface where adversarial natural language extracts internal directives. A school-district homework helper that reveals its private instructions demonstrates that prompt text is sensitive configuration, not harmless UI copy, and must be protected like other secrets that shape behavior.
Pass-the-hash against the student SSO portal is a credential-reuse attack against authentication systems; it does not explain disclosure of system-prompt text via chat. Cold-boot attacks on inference GPU memory seek residual secrets from powered-down hardware; that is a physical/memory forensic class of threat, not what a student achieves with a crafted question. DNS rebinding against the district CDN tricks browsers about origin boundaries; it is unrelated to coaxing an LLM to print its system instructions.
Exam caveat: disclosure of the system prompt is not the same as jailbreaking every policy, though leaked prompts often help attackers craft better jailbreaks. Some vendors treat system prompts as non-secret; for civic education apps, assume disclosure still aids social-engineering of the model and may leak internal escalation rules or grading criteria.
Operational check: minimize secrets in system prompts; put hard policy in application code and filters; test for prompt-extraction probes; monitor outputs for instruction echo; and treat leaked prompts as configuration exposure requiring rotation of sensitive wording. District appsec should assume students will try extraction and design so revealed prompts do not unlock grade changes or PII access.