AIP-C01 practice questions
AWS · AIP-C01 · 200 questions
Original practice questions for the AWS Certified Generative AI Developer – Professional (AIP-C01) exam, covering foundation model integration, data management and compliance, implementation and integration (including agentic AI), AI safety, security and governance, operational efficiency and optimization for GenAI applications, and testing, validation and troubleshooting — framed in civic and municipal scenarios such as city AI pilots, public-health records, permitting chatbots, 311 assistants, and transit / utilities operations.
This course contains the use of artificial intelligence.
About the AIP-C01 exam
- Time allowed
- 3 hours
- Questions
- 75
- Passing score
- 750 (scaled score of 100-1,000)
- Languages
- English, Japanese, Korean, Simplified Chinese
- Format
- Multiple choice and multiple response. 75 questions total: 65 scored plus 10 unscored. Pass/fail designation, compensatory scoring.
Exam details published by the vendor, checked 6 November 2026. Vendors change fees and formats without notice — confirm on the vendor's own page before you book.
Practice Quizzes
Test your knowledge with standard 20-question practice sets.
Quiz 1
Quiz 2
Quiz 3
Quiz 4
Quiz 5
Quiz 6
Quiz 7
Quiz 8
Quiz 9
Quiz 10
Browse by Domain
Study specific topics at your own pace.
Foundation Model Integration, Data Management, and Compliance · 62 questions
- A city CIO wants a permitting assistant that drafts staff summaries from ordinance PDFs without training a new model from scratch. Which architecture best fits that need?
- A county public-health team wants to prove value before funding a full resident FAQ chatbot. Which next step best validates feasibility on AWS?
- A municipal open-data office wants one reusable GenAI reference architecture for parks, library, and clerk apps. Which approach best supports that goal?
- A transit authority must support rider alerts that mix text policy answers with speech and image context. Which design choice best aligns the foundation-model stack to that requirement?
- A housing authority wants GenAI case-note assist that must meet existing casework SLAs and audit rules. What should drive the solution design?
- A water utility evaluates where GenAI should sit relative to field apps that already call internal APIs. Which integration pattern best fits municipal enterprise boundaries?
- A court clerk office needs GenAI answers that cite local procedural manuals rather than general legal web knowledge. Which architecture best meets that need?
- A 311 program manager wants one design that can later swap foundation models without rewriting every department UI. What architectural requirement should the design emphasize?
- An emergency-management agency needs the GenAI assistant to degrade gracefully when regional foundation-model capacity is constrained. Which design element is most important?
- A school-district IT board wants GenAI help drafting IEP paperwork, but approvals must stay with humans. How should that control be embedded?
- A city must choose among Amazon Bedrock foundation models for long council-packet summarization versus short 311 ticket classification. What should guide the selection?
- A county wants runtime switching between a cheaper foundation model for FAQ answers and a stronger model for complex permitting edge cases. Which approach best supports that goal?
- A transit operations team needs continued GenAI answers when a preferred foundation model is unavailable in their primary Region. Which pattern best addresses continuity?
- A public-health department wants LoRA-adapted clinical-policy phrasing without owning a full model-training operations stack. What customization approach fits best?
- A municipal AI platform team needs versioned deployment and rollback for a customized permitting foundation-model adapter. Which practice best supports that need?
- A parks department app must fail soft to static FAQ content if the foundation-model endpoint times out. Which pattern should the team implement?
- A clerk's office is choosing a foundation model and has both vendor benchmark scores and sample summaries on local ordinances. Which evaluation approach is most appropriate?
- A city wants foundation-model routing changes that legal can approve without redeploying Lambdas every time. Which approach best supports controlled change management?
- A housing authority retires an older fine-tuned adapter after policy manuals are rewritten. What lifecycle action is most important?
- A county IT team sees predictable evening spikes in 311 GenAI traffic and must balance cost with responsiveness. Which foundation-model access pattern should they evaluate?
- A city open-data portal must ensure CSV quality before embeddings are generated for a resident assistant. Which step should run first?
- A public-health team needs to send PDF inspection reports plus photos into Amazon Bedrock for multimodal analysis. What pipeline capability is required?
- A permitting office must send ordinance excerpts and conversation turns to Amazon Bedrock in the shape the chosen model expects. What should the data pipeline enforce?
- A transit authority receives messy free-text rider complaints before classification and response drafting. Which preprocessing step most improves foundation-model consistency?
- A water utility transcribes call-center audio and then needs foundation-model help rewriting work orders. Which pipeline awareness is most important?
- A city clerk’s office must keep packet uploads out of Amazon Bedrock until schema and PII checks pass. Which pipeline design best enforces that gate?
- A housing authority wants unstructured case notes turned into structured fields before prompting an FM for draft case summaries. What preprocessing step best supports that goal?
- A municipal library chatbot must ingest mixed HTML pages, PDF guides, and catalog records for FM consumption. Which approach best handles that heterogeneous corpus?
- A county metrics team notices RAG answers degrading and suspects the ingestion path. Which observability focus best isolates whether the data pipeline is harming GenAI quality?
- A municipal AI team must prepare cleaned, chunked city council meeting minutes for next-day RAG refresh. Which pattern best fits that workload?
- A city needs hierarchical organization of zoning-code documents in a managed retrieval corpus for GenAI. Which design choice best matches that requirement?
- A county must choose a vector store for a mid-scale permitting FAQ assistant and has strong relational-database skills but limited search-platform operations experience. Which selection rationale is most appropriate?
- A transit authority wants one retrieval corpus spanning an internal wiki and PDF SOPs, with filters for transit mode and route. What vector-store design element is most critical?
- A public-health department indexes guidance documents that differ by outbreak program, language, and effective date. How should the vector store be enriched to improve retrieval precision?
- A housing authority rewrites a single policy chapter and needs the RAG index updated without rebuilding the entire corpus. Which vector-store practice best meets that need?
- A clerk’s office manages a large OpenSearch-backed corpus and wants specialized retrieval per court division. Which indexing strategy best supports that goal?
- A parks department designs civic RAG storage with source documents, searchable vectors, and queryable document attributes. Which combination best fits that architecture?
- A 311 team’s knowledge articles change frequently in a CMS and the vector store must stay aligned. Which operational pattern best keeps retrieval current?
- A water utility revises field SOPs mid-week and fears the GenAI assistant still retrieves outdated procedure chunks. What maintenance capability is most important?
- City legal requires every retrieved ordinance chunk to show authorship and adoption date for accountability in GenAI answers. What vector-store design satisfies that requirement?
- A school district must keep FERPA-sensitive handbook sections out of the general staff assistant index. Which vector architecture best enforces that separation?
- A permitting RAG system retrieves poor answers from long zoning ordinances when chunks ignore section boundaries. Which segmentation approach should the team adopt?
- A city evaluates Amazon Bedrock embedding models for multilingual 311 knowledge articles. Which selection criteria matter most for retrieval quality?
- A transit chatbot must resolve exact stop IDs while also answering fuzzy policy questions. Which retrieval design best covers both query types?
- A public-health assistant’s top-k retrieved chunks are topically related but unevenly useful for the resident’s question. What retrieval refinement should the team add?
- Residents often ask a clerk bot vague questions such as “that fee thing,” which retrieve poor ordinance chunks. Which retrieval technique should the team apply before searching?
- A housing RAG assistant receives multi-part resident questions that mix eligibility, waitlist status, and document requirements. What retrieval pattern best handles that complexity?
- A water operations assistant must call retrieval through a consistent tool interface usable by multiple FM clients. Which approach best standardizes that access?
- A library IT team wants RAG with minimal vector-infrastructure operations burden. Which retrieval-store approach best matches that constraint?
- A county must generate embeddings for a large ordinance corpus ahead of RAG go-live. Which processing pattern is most appropriate?
- A city 311 knowledge team keeps changing RAG chunk size for policy articles and wants fewer inventively worded answers that still cite the retrieved snippets. What should they measure to decide whether a new chunking setting is actually helping?
- An emergency-management assistant must pull only SOP chunks that match the active Region and incident type before drafting staff guidance. Which retrieval design best meets that requirement?
- A municipal AI platform wants every department chatbot to share the same role framing, citation rules, and safety constraints without each team hard-coding prompts in Lambda. What approach best establishes that control?
- A permitting chatbot must remember the resident’s parcel ID across follow-up turns without asking again each message. Where should the application keep that conversation state for reliable FM prompting?
- A county policy requires legal and communications sign-off before any new resident-facing bot prompt template goes live. Which governance practice best satisfies that requirement?
- A transit FAQ bot’s underlying Bedrock model version is scheduled to change next week. What should the team do to protect rider-facing answer reliability?
- A public-health staff-facing assistant must draft multi-step triage guidance that separates symptoms, red flags, and escalation paths without inventing clinical policy. Which prompting strategy best supports that complex drafting task?
- A court clerk assistant must route residents differently for fee questions, hearing-date questions, and forms help, each with its own retrieval and prompt path. Which Bedrock-oriented pattern best expresses that branching?
- A housing authority auditor asks which prompt template version produced a contested case-note assist last Tuesday. What logging practice best enables that reconstruction?
- A parks chatbot hears “Is the big park open Sunday?” when the city has several similarly named parks. What conversational design should run before retrieval?
- A water utility wants approved prompt templates stored centrally with separate stages for draft, staging, and production. Which pattern best matches that governance need?
- A 311 product owner wants every bot answer to include a short summary, a confidence cue, and explicit next steps for the resident. How should the team iterate prompts to enforce that shape?
Implementation and Integration · 52 questions
- A city permitting helper splits work across three agents: one retrieves code sections, one drafts conditions, and one flags conflicts before staff review. What implementation focus best supports that design?
- A county agent must query parcel APIs and fee calculators through a consistent tool interface rather than bespoke one-off HTTP glue in every prompt. Which approach best fits that requirement?
- A transit ops agent must observe delayed-bus complaint clusters, decide which analytics tool to call, then refine the diagnosis in discrete steps. How should that reasoning loop be orchestrated on AWS?
- A public-health agent can call external lookup tools while drafting staff guidance. Leadership worries about runaway tool loops. Which safeguard should be implemented first-class in the agent runtime?
- A clerk agent ensemble must OCR handwritten filings with a vision-capable FM while routing ordinary text Q&A to a text-only FM. What coordination approach is most appropriate?
- A housing casework agent can draft resident-facing letters, but policy forbids sending them without a caseworker’s review. How should the agentic workflow be designed?
- A water utility agent uses Lambda-backed tools to read operational status APIs. What practice most improves reliability and safety of those tool integrations?
- A 311 platform exposes a lightweight fee-lookup MCP tool and a heavier GIS analysis tool. How should compute be chosen for the MCP servers?
- An emergency-management agent must retain short-term incident context—affected zones, open tasks, and prior tool results—across multiple tool calls in one session. What capability is required?
- A parks agent may read park calendars through tools but must never modify billing systems. Which control best enforces that boundary?
- A school-district operations desk uses specialist agents for attendance anomalies and transportation delays, then needs one staff-facing summary. What logic should the orchestrator apply?
- A municipal AI platform wants every department agent to call the same parcel lookup tool without redefining parameters ad hoc. What should the platform standardize?
- A city library assistant sees only occasional evening questions and must minimize idle cost while still calling Amazon Bedrock. Which deployment approach best fits that sparse workload?
- A county expects a sharp midday surge in its permitting-season chatbot and needs predictable Bedrock capacity without relying on on-demand throttling. Which deployment approach best fits that requirement?
- A transit authority needs a custom delay-ranking model on SageMaker while still using Amazon Bedrock to draft rider-facing disruption notices. Which deployment pattern best supports both needs?
- A public-health department wants routine staff triage-draft requests handled by a smaller specialized foundation model, escalating only rare complex cases to a larger FM. Which deployment strategy implements that approach?
- A clerk’s office must containerize a memory-heavy local helper model for an air-gapped evidence-review room where policy allows on-prem inference. Which deployment consideration is most distinctive for that LLM workload versus a typical lightweight web API?
- A housing authority wants simple resident FAQ traffic to avoid an expensive foundation model while still using that stronger model for nuanced case drafts. Which deployment approach best meets the cost and performance goal?
- A water utility is deploying a GenAI work-order rewrite assistant and mistakenly sizes capacity as if it were their existing classical fraud-detection endpoint. Which statement best captures the deployment difference they must plan for?
- A 311 platform must keep answering residents while the primary foundation-model configuration undergoes a maintenance window. Which deployment strategy best preserves continuity?
- A city wants its GenAI permitting agent to look up parcels on a legacy mainframe without giving the agent a direct mainframe session. Which enterprise integration approach best fits?
- A county wants closed 311 tickets to kick off knowledge-base refresh workflows without hard-coding callers inside the ticketing UI. Which integration pattern best achieves that loose coupling?
- A transit authority must ensure only authorized operations staff can invoke internal foundation-model tools that summarize live service incidents. Which enterprise integration control best addresses that requirement?
- A court system must keep sensitive case text on jurisdiction-controlled infrastructure while still using cloud foundation-model APIs under strict data-handling rules. Which integration pattern best matches that constraint?
- A housing CRM team wants GenAI letter drafting for caseworkers without embedding foundation-model credentials in the CRM user interface. Which architecture best supports that goal?
- A public-health team promotes prompt templates and agent stacks through CI/CD and must catch security issues before production. Which practice best fits that GenAI delivery pipeline?
- A water utility wants every department to consume foundation models through shared observability and policy controls instead of each team calling Bedrock differently. Which enterprise pattern best provides that?
- A clerk’s office needs document metadata from its enterprise content management system reflected in the RAG index without tightly coupling the two applications. Which integration approach best fits?
- A school district is connecting a student-facing app to foundation-model services and must minimize what that app can do. Which API access pattern best applies least privilege?
- A parks department wants AI-written confirmation emails when a reservation system creates a new booking. Which integration approach best triggers that GenAI step?
- A 311 platform plans stadium-event kiosk assistants that need very low latency near the venue. Which enterprise integration consideration is most relevant?
- A municipal enterprise architecture board requires GenAI components to meet the same continuous delivery rigor as other city services. Which pipeline expectation best reflects that mandate?
- A city mobile 311 app needs foundation-model answers but must not hold long-lived Bedrock credentials on residents’ phones. Which FM API integration pattern is most appropriate?
- A county must summarize a large backlog of ordinance PDFs without blocking interactive clerk applications on long foundation-model calls. Which FM API integration pattern best fits?
- A transit rider app must show disruption explanations as tokens arrive instead of waiting for the full foundation-model completion. Which FM API integration capability should the team use?
- A public-health FM client sees Amazon Bedrock throttling during an outbreak-driven traffic spike. Which resilience technique should the client implement?
- A clerk portal shares foundation-model endpoints across departments and one team’s burst traffic is starving others. Which FM API integration control best restores fair use?
- A housing bot should send simple FAQ questions to one foundation model and eviction-prevention counseling drafts to a stronger model based on request content. Which FM API integration approach implements that behavior?
- A water utility’s SCADA-adjacent assistant must keep answering from approved SOP text when Bedrock calls fail. Which FM API client pattern provides that resilience?
- A parks department kiosk should display exhibit explanations as tokens arrive from Bedrock rather than waiting for the full completion. Which delivery mechanism fits that real-time civic UX?
- A 311 backend must reject malformed resident-assistant requests before any foundation model is invoked. Where should that validation be enforced?
- A city observability team needs to follow a single resident 311 GenAI request across API Gateway, Lambda, and Bedrock. Which integration practice supports that cross-service visibility?
- Department apps call a shared city GenAI API that must stream tokens and surface token-limit errors clearly. Which application-integration design addresses those GenAI-specific failure modes?
- County analysts who are not professional developers need to compose internal research assistants from approved prompt steps. Which development approach fits that requirement?
- A transit authority wants its CRM to draft rider delay notices through GenAI when a disruption event arrives. Which application integration pattern best adds that capability?
- A public-health developer needs help refactoring Bedrock client code while still owning architecture decisions for the outbreak FAQ service. Which development-tool pattern is appropriate?
- A clerk’s office needs a multi-step document-intake agent that OCR-checks filings, classifies them, and routes exceptions. Which orchestration approach fits that municipal workflow?
- A housing authority wants other city systems to integrate cleanly with its GenAI microservices. Which API development approach should the platform adopt?
- A water utility receives large compliance PDF uploads that must be structured before GenAI review. Which application pattern best prepares that content?
- A 311 engineering team sees a cluster of poor answer classes and needs to inspect stored prompt and response fields. Which troubleshooting tooling approach helps?
- A parks web team wants a modern UI that calls GenAI without exposing Bedrock credentials in the browser. Which integration pattern accelerates that civic app?
- A municipal platform must draft resident notices in stages: summarize the incident, check tone policy, then produce the final message. Which application pattern supports that flow?
AI Safety, Security, and Governance · 40 questions
- A city youth-services chatbot must block toxic and self-harm content in resident chats. Which safety control should be enabled first?
- A county permitting assistant must not invent criminal penalties that do not appear in local code. Which output safety approach best addresses that risk?
- A transit chatbot should explain delays using Knowledge Base facts and escalate low-confidence drafts for human review. Which safety pattern implements that behavior?
- A public-health GenAI stack wants layered misuse protection beyond a single filter. Which architecture best reflects defense-in-depth safety?
- A clerk system must catch adversarial instructions hidden inside uploaded PDF text before those chunks enter retrieval. Which defensive control should be applied?
- A housing chatbot must refuse attempts that try to make it ignore fair-housing policy constraints. Which safety behavior is required?
- A water utility allows a GenAI assistant to query operational metrics but must prevent unsafe free-form SQL from the model. Which control reduces that risk?
- A 311 product team wants safety assurance before each resident-assistant release. Which practice should be part of the release gate?
- A parks kiosk GenAI experience must block user prompts that request disallowed adult content in real time. Which control applies?
- A school-district assistant drafts parent emails that must be screened for toxic language before send. Which output safety technique fits?
- An emergency-management app must ensure every FM-generated alert draft includes severity and affected-area fields. Which accuracy and safety control enforces that structure?
- A municipal security operations center wants early warning when resident-facing bots keep receiving the same prompt-injection patterns. Which operational control best surfaces that threat signal?
- A city wants its permitting RAG corpora to reach Amazon Bedrock without placing those corpora on a public internet path. Which data-security pattern best meets that requirement?
- A county health department needs GenAI apps used by authorized analysts to read case tables while other city bots cannot. Which control best enforces that granular access?
- A transit authority must keep rider phone numbers and addresses out of foundation-model interaction logs. Which privacy control should the team apply in the messaging pipeline?
- A public-health chatbot must avoid echoing Social Security numbers that residents sometimes paste into chat. Which privacy-preserving control best addresses that risk?
- A clerk’s office must expire prompt and response logs according to municipal records-retention schedules. Which data-security practice best aligns GenAI logs to that policy?
- A housing authority wants to use real case notes to evaluate prompt changes without exposing tenant identities in the evaluation dataset. Which privacy approach is most appropriate?
- A water utility GenAI assistant should answer service FAQs but must not read unrestricted customer billing tables. Which IAM design best enforces that boundary?
- A 311 platform needs to notice if GenAI service roles suddenly pull unusually large volumes of resident case data. Which monitoring approach best supports that data-security goal?
- A parks membership bot summarizes support tickets that sometimes include payment card numbers. What should the pipeline do before any foundation-model summarization?
- A school district wants GenAI tutoring aids to help with coursework without default access to full student record systems. Which privacy-by-design choice best matches that goal?
- A city wants each resident-facing foundation-model application to carry clear documentation of purpose, data, and known limits for governance reviews. Which artifact best satisfies that need?
- A county counsel’s office must show how an ordinance PDF became the text behind a RAG answer during a records review. Which governance mechanism best supports that traceability?
- A transit authority wants every generated rider notice to show which standard operating procedure supplied the guidance. Which governance practice enables that systematic attribution?
- A public-health board requires new GenAI features to pass an organizational Responsible AI checklist before go-live. Which governance action best implements that requirement?
- A clerk’s office needs audit-ready evidence of who changed prompt templates for the filing assistant. Which control best supports that compliance review?
- A housing GenAI program drafts denial-letter language and must watch for unfair wording shifts across ZIP codes over time. Which governance control should the team implement?
- A water utility needs ongoing detection when FM outputs violate published communication policy, plus a path to fix them. Which governance pattern fits?
- A 311 team will wire retrieval only to knowledge sources that governance has formally approved. What should happen before those sources connect to the GenAI app?
- A parks commission must retain GenAI interaction logs for public-records requests while preventing sensitive tokens from being disclosed. Which advanced governance control fits?
- An emergency-management agency must present evidence that its GenAI alerting stack was monitored and governed during an audit. Which preparation best supports that review?
- A city wants 311 bot replies to help residents understand why an answer was given without exposing internal tooling details. Which Responsible AI transparency practice fits?
- A county permitting assistant should quantify how uncertain recommendations are so staff can intervene on low-confidence cases. Which Responsible AI practice supports that goal?
- A transit team worries that delay messages might treat some language communities less clearly than others. Which fairness-oriented evaluation approach should they use?
- A public-health unit needs scalable checks that outreach drafts meet fairness rubrics before campaigns launch. Which automated evaluation approach best supports that Responsible AI goal?
- A municipal clerk bot sometimes returns contested answers that staff must defend. Leadership wants agent reasoning traces available to clerks for audit review, but not automatically shown on the public resident portal. Which Responsible AI approach best fits that requirement?
- A housing authority ships an eviction-prevention assistant and must document where the foundation model can fail so caseworkers do not treat drafts as binding legal advice. Which Responsible AI practice best addresses that need?
- A water utility generates boil-water advisory drafts with a foundation model, then a Lambda step must block publication if required policy checks fail. Which approach best implements Responsible AI compliance for those outputs?
- A school district wants Amazon Bedrock Guardrails topics to reflect board-approved student-safety policies rather than generic defaults alone. Which Responsible AI alignment step should the GenAI team take?
Operational Efficiency and Optimization for GenAI Applications · 24 questions
- A city finance office wants lower 311-bot spend and clearer chargeback by department. Which operational efficiency approach best combines usage visibility with direct cost control?
- A county parks FAQ bot mostly answers simple hours and closure questions, but still calls the most expensive foundation model for every request. Which cost-efficiency change best fits that workload?
- A transit authority must summarize overnight incident reports where near-real-time replies are not required. Which resource-efficiency strategy best improves throughput and cost for that GenAI workload?
- A public-health chatbot sees many near-identical outbreak FAQ questions that still trigger fresh foundation-model calls. Which optimization best reduces unnecessary invocations?
- A clerk RAG assistant retrieves large ordinance packets, and token spend rises because generation includes lengthy low-value context. Which efficiency practice best reduces tokens while preserving useful citations?
- A housing chatbot team must choose among Amazon Bedrock foundation models under a fixed monthly budget. Which evaluation approach best supports that municipal selection decision?
- A water utility’s GenAI ops console sees predictable morning peaks when field supervisors open shift briefings. Which capacity approach best balances cost and readiness for that pattern?
- A parks visitor kiosk repeatedly receives identical exhibit questions from different guests. Which caching pattern best avoids redundant foundation-model calls for those exact repeats?
- A city permitting assistant must feel responsive at the counter while fee tables change only nightly. Which performance design best balances latency and cost?
- A county runs both live 311 chat and overnight research summarization on Amazon Bedrock. Which model-selection practice best optimizes performance for the live path?
- A transit RAG path is slow when riders ask about stop disruptions because retrieval dominates end-to-end time. Which optimization focus best improves foundation-model augmentation performance?
- A public-health workflow must both translate and summarize the same outbreak bulletin before staff review. The steps do not depend on each other’s outputs. Which performance pattern best reduces wall-clock time?
- A clerk team uses one Bedrock application for creative outreach drafts and for deterministic fee answers. Which performance and quality configuration practice best matches those different needs?
- A housing platform considers adding a reranker stage before generation and must know whether quality gains justify extra latency. Which optimization practice should the team use?
- A water operations API feels slow, and engineers disagree whether prompt assembly, retrieval, or foundation-model completion dominates. Which practice best drives latency reduction?
- A 311 GenAI service scales only on CPU utilization, yet outages still occur during token-burst traffic when CPUs look moderate. Which autoscaling improvement best fits GenAI traffic patterns?
- A parks festival weekend drives sudden spikes in exhibit-assistant traffic and foundation-model throttling appears. Which performance control best manages throughput during those spikes?
- An emergency-management dashboard must prove GenAI answer time meets published board SLAs after each release. Which practice best supports that performance goal?
- A city SRE team already watches HTTP 5xx rates but lacks insight into GenAI-specific health for the resident assistant. Which monitoring approach best closes that gap?
- A county must forensically review a bad permitting answer to see the exact prompt and model response. Which monitoring capability best supports that analysis?
- After a transit authority ships a new multi-tool agent, overnight token spend jumps in irregular bursts. Which monitoring practice best helps SREs catch that GenAI cost anomaly early?
- A public-health multi-agent outbreak Q&A stack feels slow, and leaders need to know which tool call in the chain is the bottleneck. Which monitoring approach gives the most actionable visibility?
- A clerk platform’s ordinance RAG answers grow sluggish as the corpus expands. What operational practice best keeps the vector store healthy for foundation-model augmentation?
- After a housing-authority prompt edit, staff worry answers drifted even though classic uptime metrics look fine. Which GenAI-specific observability practice best detects that consistency problem?
Testing, Validation, and Troubleshooting · 22 questions
- A city wants a scorecard for its permitting assistant that reflects GenAI answer quality rather than classical ML classification accuracy alone. Which metric set best matches that goal?
- A county must choose between two foundation models for ordinance Q&A using a structured, repeatable comparison. Which approach best supports that decision?
- A transit disruption assistant needs continuous signal on whether riders find explanations helpful. Which evaluation mechanism best captures that user-centered feedback?
- A public-health team refuses to promote GenAI builds that regress hallucination rates. How should that policy be enforced in the delivery pipeline?
- A clerk office drafts hearing notices with a foundation model and needs scalable review that still catches high-stakes mistakes. Which assessment pattern best fits?
- A housing RAG team sees fluent answers that still miss the right policy sections. What evaluation design best isolates the failure?
- A water utility work-order agent must prove it finishes tasks and uses tools correctly, not merely that it returns text. Which evaluation focus is most appropriate?
- A 311 product manager must keep department stakeholders informed about GenAI answer quality without dumping raw CloudWatch logs. What practice best meets that need?
- Before a parks department upgrades the chatbot’s foundation model, release engineers want confidence that behavior did not silently drift. Which validation approach is most appropriate?
- A school district wants to try a new foundation model for a staff-facing assistant with limited blast radius. Which rollout pattern best fits?
- An emergency-management team must balance storm-season answer speed against answer quality for a resident briefing assistant. Which analysis should guide the configuration choice?
- A city regression suite must fail builds when ordinance answers stop including required citation formatting. What testing practice enforces that standard?
- A county needs subject-matter judgment on whether clerk-bot answers are factually correct. Which evaluation mechanism best supplies that signal?
- A transit RAG team already tracks relevance scores but riders still abandon lookups that feel slow. What should they add to retrieval quality evaluation?
- A city 311 bot truncates mid-answer after operators stuffed large case histories into each prompt. What troubleshooting focus best addresses the failure?
- A county sees spikes of Bedrock 4xx/5xx errors right after shipping a new client library for a permitting assistant. What should engineers do first to diagnose the integration issue?
- Transit FAQ quality dropped immediately after a prompt edit. How should the team troubleshoot the regression most effectively?
- After a public-health corpus migration, RAG answers pull off-topic SOPs even though the generator settings are unchanged. Which troubleshooting path best targets the likely cause?
- A clerk bot confuses fee schedules, and investigators suspect different environments are loading mismatched prompt templates. Which diagnostic approach best confirms that?
- A housing assistant cites outdated policy chapters even after authors publish updates in the source repository. What remediation best targets the retrieval pipeline?
- A water agent’s tool calls succeed, yet the final recommendation is wrong. What troubleshooting technique best reveals where the multi-step reasoning failed?
- A parks kiosk stream hangs until users give up, and distributed traces show gateway timeouts on long foundation-model completions. What should the team prioritize?
These questions are original practice material and are NOT actual exam questions or brain-dump content. All vendor marks are trademarks of their respective owners. This site is not affiliated with, endorsed by, or sponsored by Amazon Web Services or Amazon.com, Inc.