Model Customization and Alignment
GAIPS · 25 questions
- A city plans to fine-tune a foundation model on historical 311 transcripts that still contain resident phone numbers. What is the primary security concern for the customization corpus?
- County public health proposes LoRA adapters for a clinic FAQ model but plans to store adapter weights on the same shared host used by unrelated departmental apps. What should security require first?
- Permitting staff fine-tune a chatbot on an outdated building-code edition, and the bot later cites repealed sections to applicants. What customization risk does this illustrate?
- A housing authority adds RLHF-style preference data that rewards faster case answers more than eligibility accuracy. What alignment risk should the security lead highlight?
- Library IT enables a vendor moderation API for a youth-services chatbot but never maps blocked categories to the library's content policy. What gap remains?
- Transit operations fine-tunes a rider-alert model for production. What evaluation practice should security require around the customization?
- Court clerks request a custom system prompt that forbids legal advice, but developers deploy the change without regression tests. What control is missing?
- A school district wants to fine-tune a homework helper on student essays, but data-handling rules for the training set are unclear. What should security require before customization proceeds?
- A water utility considers continued pretraining on internal SOPs stored in a wide-access file share. What is the most important data-security step for that customization path?
- City appsec reviews a 311 bot and must choose where to place moderation and alignment controls. Which approach best matches layered defense for civic GenAI?
- Emergency management wants to fine-tune on after-action PDFs that still include unredacted personal narratives. What must happen before customization?
- Parks staff train a vision-language adapter on geotagged trail photos that reveal private property layouts beside park boundaries. What training-data risk should security flag?
- Procurement accepts a vendor 'aligned municipal model' that provides no documentation of safety evaluation harnesses. What should the city demand?
- County IT merges several departments' fine-tunes into one shared checkpoint without change control. What governance failure does this create?
- A benefits chatbot's safety filter blocks legitimate appeals language as 'self-harm adjacent,' preventing residents from completing appeals. What alignment action is needed?
- Security discovers a developer laptop running unsanctioned fine-tunes on production parcel data for a municipal assessor assistant. What platform risk does this represent?
- City open-data staff debate domain-adaptive fine-tuning versus a RAG-only design for ordinance Q&A. From a security perspective, what contrast matters most?
- After user complaints, operators weaken a clerk summarizer's refusal policy to 'be more helpful.' What alignment failure should the CISO call out?
- Transit security wants customized rider-alert models reproducible for incident review. Which practice best supports that goal?
- Public-health alignment evaluations for a clinic chatbot omit non-English languages used by many residents. What evaluation gap does this create?
- A housing authority stores RLHF preference-ranking labels in a spreadsheet with editor-wide write access. What integrity risk should security address?
- A municipal ML team uses 'synthetic' fine-tuning data but never checks whether case IDs from real records remain in the set. What validation is required?
- The city CISO requires human review when moderation confidence is low on 311 replies. What alignment pattern does this implement?
- County counsel asks whether fine-tuning a commercial foundation model on local ordinances may create licensing or derivative-work constraints. What should the platform team verify?
- After fine-tuning, a permitting bot begins emitting internal review checklists to public applicants. What customization side effect should security investigate?