AI and LLM Foundations
GAIPS · 25 questions
- A city CIO asks security why the 311 assistant sometimes drops the start of a resident’s long case history mid-conversation. Which GenAI foundation concept best explains that behavior?
- A public-health chatbot tells a resident clinic hours that do not appear in any approved municipal source. Which GenAI risk best describes that failure?
- A permitting portal pilot receives a user message that tells the model to ignore ordinance-citation rules and approve every application. What GenAI attack surface does this primarily illustrate?
- County counsel worries a court-clerk summarizer may invent case citations that were never filed. Which LLM limitation should security emphasize for this legal-adjacent workflow?
- A transit rider-alert model confuses two stations with similar-sounding names when matching free-text complaints. Which foundation concept best explains that mix-up?
- IT security reviews a parks department model that only classifies trail-camera images as empty, occupied, or damaged with fixed labels and no free-text generation. How should they classify it before applying GAIPS-style GenAI platform controls?
- A school-district homework helper reveals its hidden system-prompt instructions after a student asks for them in a cleverly worded request. Which LLM attack surface does this demonstrate?
- Emergency management wants an LLM to draft after-action report narratives from unstructured field notes. What contrast with legacy NLP should security highlight when judging platform fit?
- A housing authority assistant answers eligibility questions in a tone that sounds like a final binding determination. What GenAI output risk should the platform owner address first?
- City open-data staff want ordinance FAQ answers to stay consistent across identical resident questions. Which decoding control should they understand first?
- A malicious resident uploads a PDF whose hidden white-on-white text tells the 311 RAG stack to ignore privacy rules. Which GenAI attack pattern is this?
- Water-utility analysts must summarize SCADA-adjacent logs and are choosing between a large general foundation model and a smaller specialized model. Which foundation security consideration should guide the choice?
- Library staff discover the chatbot’s training corpus may include copyrighted publisher blurbs scraped without a license. Which GenAI foundation concern does this raise?
- A county assessor imagery-captioning model returns a refusal message on some prompts; staff open a major outage ticket. What distinction should security teach first?
- A municipal developer treats an LLM API as a trusted database of parcel facts for fee calculations. What misconception should platform security correct?
- Security explains to a benefits clerk why pasting even a partial Social Security number into a prompt still matters. Which foundation idea supports that warning?
- During a tabletop, a city red-team pastes jailbreak-style roleplay that asks a benefits chatbot to pretend policies do not apply. What should defenders primarily recognize?
- Transit operations wants multimodal GenAI that reads bus photos and free-text complaints in one assistant. What foundation security implication should be evaluated?
- Procurement asks whether buying a closed commercial LLM removes all GenAI security risk for the permitting assistant. What is the accurate platform-security response?
- A parks volunteer bot’s few-shot prompt examples accidentally include real volunteer home addresses. What GenAI foundation risk does this create?
- IT maps confidentiality, integrity, and availability threats for a city council agenda drafter that uses GenAI. Which mapping best reflects GenAI-specific failure modes?
- A county AI working group confuses theft of model weight files with a successful prompt-injection against the live 311 bot. How should security differentiate the threats?
- Security reviews whether logit bias or stop sequences can help a 311 bot avoid emitting internal ticket ID formats. What are these controls in GenAI foundations terms?
- Public-records staff ask how stochastic decoding affects reproducibility of AI-drafted FOIA response letters. Which explanation is most accurate?
- A municipal CISO onboards appsec engineers to the city AI platform and wants shared GenAI vocabulary. Which set best represents foundational terms they must distinguish to secure LLM apps?