MLA-C02 practice questions
AWS · MLA-C02 · 200 questions
Validates associate-level AWS machine learning engineering skills for both traditional ML and generative AI, covering data preparation for ML and AI (including vector databases, embeddings, and RAG document prep), ML model and foundation model development (Amazon Bedrock, fine-tuning, evaluation), deployment and orchestration of ML and AI workflows (SageMaker, agentic workflows, CI/CD), and operating, monitoring, and securing ML and AI solutions, set in municipal and civic-government scenarios.
This course contains the use of artificial intelligence.
About the MLA-C02 exam
- Exam fee
- $75 USD
- Time allowed
- 2 hours 10 minutes
- Questions
- 65 (50 scored, 15 unscored)
- Passing score
- 720 (scaled, 100-1,000)
- Languages
- English only (Beta); Japanese, Korean, and Simplified Chinese planned at general availability
- Format
- 65 questions; multiple choice (one correct response, three distractors) and multiple response (two or more correct out of five or more options, all must be selected for credit); 50 scored questions and 15 unscored questions not identified on the exam; compensatory scoring model; unanswered questions scored as incorrect, no penalty for guessing
Exam details published by the vendor, checked 13 September 2026. Vendors change fees and formats without notice — confirm on the vendor's own page before you book.
Practice Quizzes
Test your knowledge with standard 20-question practice sets.
Quiz 1
Quiz 2
Quiz 3
Quiz 4
Quiz 5
Quiz 6
Quiz 7
Quiz 8
Quiz 9
Quiz 10
Browse by Domain
Study specific topics at your own pace.
Data Preparation for ML and AI · 56 questions
- A city 311 call center streams both audio recordings and their text transcripts through Amazon Kinesis Data Streams and lands them in Amazon S3. An ML engineer needs a storage layout that supports both model training on the transcripts and later replay of the paired audio for quality review. Which approach best meets this need?
- A municipal permitting chatbot team is choosing between Amazon RDS for PostgreSQL with the pgvector extension and Amazon OpenSearch Service as the vector store behind the chatbot's retrieval step. The team expects tens of millions of document embeddings and heavy concurrent similarity-search traffic during business hours. Which choice best fits this workload?
- A public-works department collects water-meter inspection records that include both photos of meter dials and structured sensor readings (flow rate, pressure, timestamp). Before feature engineering begins, where should the photos and the structured readings be stored?
- A transit authority is choosing between Amazon Kinesis Data Firehose and Amazon Kinesis Data Streams to feed ridership counts into a demand-forecasting feature pipeline. The forecasting job only needs to read fresh batches of ridership data every few minutes from S3, with no need for custom per-record processing before landing. Which service fits this requirement?
- A court clerk's office converts case-record exports from CSV to Parquet to reduce Amazon Athena and AWS Glue scan costs for a downstream ML feature pipeline. Which property of Parquet is the main reason this conversion lowers those costs?
- A housing authority needs to merge inconsistent casework records — some CSV, some JSON, arriving from several county source systems with slightly different schemas — into a single feature table for a benefits-eligibility model. Which approach best handles this heterogeneous, multi-source merge?
- A benefits-eligibility scoring service needs low-latency access to the newest risk features at inference time, and it also needs historical feature snapshots for periodically retraining the model. Which SageMaker Feature Store design fits both needs?
- A permitting office ingests scanned permit-application PDFs into Amazon S3 and needs a metadata catalog that keeps each scanned document usable together with its searchable fields (applicant name, permit type, submission date) for a downstream multimodal document-understanding pipeline. Which approach best supports this?
- An emergency-management team streams 911 dispatch events through Amazon MSK into a pipeline that trains an anomaly-detection model on dispatch patterns. A colleague suggests replacing MSK with a nightly batch export instead. Which reason best justifies keeping the Kafka-based streaming approach?
- A city's 311 resident-facing chatbot needs to persist conversation session state, keyed by session ID, that will later be reused as fine-tuning data. The team is deciding between Amazon DynamoDB and Amazon RDS for this store, given that access is almost entirely single-item reads and writes by session key with no complex joins. Which choice fits this access pattern?
- A school district uses Apache Flink on Amazon Kinesis Data Analytics to compute windowed aggregations (for example, hourly attendance counts per building) from attendance-sensor streams before the results are stored for modeling. Why does performing this aggregation in the streaming layer, rather than as a later batch job against raw stored events, benefit the downstream modeling pipeline?
- An engineer partitions municipal water-quality sensor data in S3 by date, agency, and sensor ID to speed up downstream AWS Glue jobs that typically filter on a date range for a specific agency. Why does this partitioning scheme improve Glue job performance?
- A municipal open-data portal embeds its public-records document corpus and stores the vectors in Amazon OpenSearch Service to power a RAG-based public-records assistant. Which configuration choice is most important for this vector store to serve accurate retrieval for the RAG pipeline, beyond simply enabling k-NN search?
- A transit authority's ridership data lake is queried heavily through Amazon Athena, where most queries filter on a handful of columns out of dozens available and occasionally use complex predicate pushdown on nested fields. The team is choosing between ORC and Parquet as the storage format. Which consideration should drive the choice for this Athena-centric workload?
- A 911 call center ingests audio recordings and their transcripts into Amazon S3 to feed a multimodal content-safety screening pipeline that flags calls needing supervisor review. Which ingestion design best supports this downstream screening step?
- A county assessor's office has parcel records spread across several legacy source databases with inconsistent table structures. An ML engineer wants to build a unified view of the schemas before designing a feature-engineering pipeline. Which use of AWS Glue best fits this discovery step?
- A housing-eligibility model's SageMaker training job runs on an on-demand training instance and needs to persist checkpoints during the run so training can resume if the job is interrupted. Which storage choice is appropriate for these checkpoints during and after the run?
- A team configures an Amazon OpenSearch Service vector index to hold embeddings for a large municipal code corpus supporting a legal-research assistant. The embedding model produces 1,536-dimension vectors. Which configuration decision most directly affects whether retrieval works correctly?
- A transit authority streams real-time GPS pings via Amazon Kinesis into a data lake and is deciding whether to write raw JSON directly to S3 or add a conversion step to Parquet before landing. The downstream feature pipeline runs frequent, selective Athena queries filtering on route and time window. What is the most relevant tradeoff in this decision?
- A permitting office uses SageMaker Data Wrangler to bin continuous permit-processing-time values into categories such as fast, typical, and slow ahead of training a wait-time prediction model. Which situation makes binning this feature the right transformation choice?
- A court clerk's office is preparing scanned case-filing PDFs for a RAG knowledge base and must choose between fixed-size chunking and semantic chunking to split documents before embedding. Case filings contain long, legally structured sections (facts, arguments, rulings) where splitting mid-argument would hurt retrieval quality. Which chunking approach best fits this document type?
- A public-health department needs to redact PII such as Social Security numbers and home addresses from patient intake records before those records are used for feature engineering in a health-outcomes model. Which AWS Glue capability is designed for this redaction step?
- A water utility's billing and usage data is heavily right-skewed, with most accounts showing low usage and a long tail of very high-usage accounts. The team applies a log transform to this feature in a SageMaker Data Wrangler flow before training a leak-detection model. Why is a log transform an appropriate fix here?
- A 311 team is building a semantic-search assistant over historical service-request text and needs to select an embedding model to convert that text into vectors for retrieval. Which consideration should most directly guide this embedding-model selection?
- A permitting chatbot team applies tokenization and domain-specific text augmentation to municipal-code legal text before using it to fine-tune a foundation model. Which outcome is this advanced text pre-processing step primarily intended to achieve?
- A transit agency streams live bus-sensor data into a Kinesis data stream and needs to apply lightweight unit conversion and deduplication to each record before it lands in Amazon S3, with sub-second processing per record. Which approach best fits this requirement?
- A benefits-eligibility model uses income (ranging into six figures) and household size (single digits) as input features. Before training, an ML engineer notices the model trained on raw values weights income far more heavily than household size. Which SageMaker Data Wrangler transform addresses this most directly?
- A city clerk's public-records-request assistant retrieves answers from a RAG system built over a mixed archive of ordinances, meeting minutes, and permits. Residents often ask for documents from a specific department and date range. Which document-preparation step most directly enables the assistant to filter retrieval results by department and date?
- A county's feature-engineering pipeline must join a property-tax table with tens of millions of rows against a permitting table of similar size, and a single-node transform tool times out on the join. Which approach is best suited to complete this join reliably?
- A team wants to adapt a foundation model to municipal terminology by continuing its pre-training on a large volume of raw 311-service text, rather than teaching it to follow a specific instruction format. Which data-preparation choice correctly reflects continuous pre-training rather than supervised fine-tuning?
- A housing authority wants to fine-tune a foundation model on historical casework notes to help staff draft case summaries, but the notes contain resident names and home addresses. What should happen to this text before it is used as fine-tuning data?
- An emergency-management team is chunking lengthy response-plan documents before embedding them for a RAG-based dispatcher assistant. Using very large chunks with no overlap causes the assistant to retrieve passages that bury the specific procedure a dispatcher needs inside unrelated surrounding text. What is the most direct fix?
- A school district merges enrollment records from several source systems and finds that enrollment dates appear in multiple inconsistent formats across the combined dataset. Before feature engineering begins, which tool is best suited to profile the data and standardize these date formats?
- A public-works department is preparing a dataset of street-sign photographs in SageMaker Data Wrangler before training a computer-vision model to detect damaged signs. Which pre-processing steps are appropriate for this image data, as opposed to the techniques used for tabular features?
- A team wants to train a smaller, cheaper model to mimic the behavior of a larger foundation model on municipal chatbot queries. They plan to run a set of representative queries through the larger model and use its outputs as training targets for the smaller model. What does this describe, in terms of data preparation?
- A crime-pattern-analysis model uses a neighborhood/district code field with several hundred distinct values as an input feature. One-hot encoding this field would create hundreds of new sparse columns and bloat the feature set. Which SageMaker Data Wrangler encoding approach better fits this high-cardinality field?
- A 911 dispatch system streams call records through Amazon Kinesis before they land in a data lake for downstream ML use. Caller phone numbers and home addresses must be masked before any raw record is persisted. Which approach applies the masking at the right point in the pipeline?
- A document-understanding assistant for permit applications needs to answer questions using both the scanned permit-form images and their OCR-extracted text, along with metadata like permit type. During data preparation, what is essential to preserve so retrieval and generation work correctly at query time?
- A municipal open-data portal ingests a monthly CSV upload that feeds a downstream retraining pipeline. The team wants an automated check that flags when a column's data type or expected value range shifts from prior months, before the new data is used for retraining. Which capability addresses this directly?
- A housing-eligibility approval model is trained on historical decisions where the vast majority of records are approvals and only a small fraction are denials. The engineer wants the model to learn to recognize denial cases accurately rather than ignoring them. Which strategy most directly addresses this class imbalance?
- A water utility's sensor network occasionally reports pressure readings that are orders of magnitude outside the physically plausible range due to sensor malfunctions. Before these readings feed a leak-detection model, the team wants to automatically flag statistically implausible values. Which approach addresses this during data preparation?
- A water utility's consumption-forecasting dataset has gaps where sensor readings failed to report for a few hours at a time. The gaps are sparse and the underlying consumption pattern is skewed rather than symmetric. Which imputation choice best fits this situation?
- Before deploying a benefits-eligibility model, a city wants to confirm the model doesn't produce systematically different approval rates across demographic subgroups such as zip code or age bracket. Which SageMaker capability is purpose-built for evaluating this kind of subgroup disparity during data preparation and pre-deployment analysis?
- An auditor asks a municipal ML team to reproduce exactly which dataset version and feature transformations produced a permitting-delay model deployed six months ago. The team needs to trace this after the fact. Which practice during data preparation makes this kind of reproducibility possible?
- A permitting department's application dataset contains near-identical records for the same permit, created when residents accidentally submitted a form multiple times. Left uncorrected, these near-duplicates could bias a permit-approval-time model toward whatever pattern those repeated records happen to show. What should happen during data preparation?
- A county's property-valuation model joins a parcel table with a separate tax-assessment table using a parcel ID. Some parcel IDs in the assessment table don't correspond to any parcel in the parcel table, likely due to an upstream data-entry error. Which check would catch this before the join corrupts the training set?
- A public-safety team uses SageMaker Ground Truth to have human reviewers label dashcam footage for a hazard-detection model. Different reviewers frequently label the same clip differently, suggesting the labeling guidance may be ambiguous. Which practice helps identify and address this before the labels are used for training?
- A team is preparing a time-ordered dataset of benefits applications to train a model that predicts approval outcomes. They split the data into training and test sets by randomly shuffling all records regardless of application date. What data-preparation problem does this random split risk introducing?
- A parks-department maintenance-request dataset has a required 'reportedbydepartment' field, but a growing share of monthly uploads arrive with that field blank. The team wants automated rules that specifically track completeness thresholds for required fields over time and alert when a field's null rate climbs too high. Which capability fits this need?
- A transit-ridership forecasting model has been deployed for several months. The team wants to detect if the statistical properties of live production input data, like average trip distance or time-of-day distribution, have shifted away from what the model was originally trained on. Which capability addresses this ongoing concern after deployment?
- A municipal permitting office trains a resident-facing chatbot on historical request transcripts, and a bias metric shows most training examples originated from a single council district. What should the team do to correct this imbalance before continuing model development?
- A housing authority has three caseworkers manually label housing-casework case notes with sentiment tags to train a triage model. Before trusting the labeled set, what should the team do to validate label quality across the three annotators?
- An engineering team merges a county resident-services dataset with a city resident-services dataset and profiles the combined table using Glue DataBrew before modeling. The profile surfaces columns with high null rates and several duplicate resident records. What should the team do next?
- A public-works department is preparing a dataset of infrastructure photographs submitted by residents to train a model that flags road-damage severity. Before training, the team wants to check the image dataset for demographic and geographic bias the same way they would check a tabular dataset. What should they do?
- A public-records assistant retrieves from a RAG knowledge base built from municipal-code documents. Before ingesting new documents, the data-preparation team wants to validate training-data integrity for the knowledge base. What should they check?
- A school district's dataset for predicting student outcomes underrepresents one demographic group relative to district enrollment figures. The engineer applies a re-sampling technique to the training data as a bias-mitigation step. What is the engineer most likely doing?
ML Model and Foundation Model (FM) Development · 48 questions
- A city's 311 department is evaluating several Amazon Bedrock foundation models for a resident-service chatbot that must handle long conversation histories and respond fluently in the languages spoken by the city's resident population. What should guide the department's model selection?
- A county permitting office wants an assistant that answers fee-schedule questions, but permit fees change every quarter. The team is deciding whether to fine-tune a foundation model on the current fee schedule or use retrieval-augmented generation against a fee-schedule document. Which approach better fits the requirement that answers stay current as fees change?
- A transit authority needs to forecast daily bus ridership from years of structured, numeric ridership history. A data scientist is deciding between a traditional gradient-boosted model and a foundation-model-based approach. Which consideration should drive the decision?
- A school district is deciding whether to use Amazon Comprehend's built-in text classification or train a custom text classifier to route parent complaint emails to the correct department. The categories map closely to generic complaint types Comprehend already models, and the district wants a solution quickly. What should guide the decision?
- A water utility engineering team is choosing between a SageMaker built-in anomaly-detection algorithm and a fully custom training script for identifying anomalies in pipe-pressure sensor readings. The readings follow a standard time-series pattern that closely matches the built-in algorithm's intended use case. What should guide the decision?
- A public housing authority designing a tenant-handbook Q&A assistant must pick between a single consolidated knowledge index and multiple topic-segmented indexes for its RAG architecture, given a handbook that spans maintenance requests, lease terms, and grievance procedures. Which pattern better fits documents that span several distinct topics with different retrieval precision needs?
- A county clerk's office digitizing decades of scanned filings must decide whether Amazon Textract or a custom-built OCR pipeline better fits its structured-form extraction needs. The forms are common layouts that Textract's form and table extraction is designed to handle. What should guide the decision?
- An emergency management agency assessing storm-damage photos submitted by residents through a mobile app must choose between Amazon Rekognition and a custom-trained computer vision model. The damage categories are highly specific to the agency's local infrastructure and don't map to Rekognition's general-purpose labels. What should guide the decision?
- A city parking enforcement team is selecting a modeling approach for classifying violation photos, and officers need a classification result in well under a second at the point of issuing a citation in the field. What should the team weigh most heavily when choosing between approaches?
- A 311 assistant team finds their foundation model doesn't reliably recognize municipal-specific vocabulary and department names used across the city's service catalog, even though it already follows instructions well. The team is deciding between instruction fine-tuning and continued pre-training as the customization strategy. Which should they choose?
- A county courts department needs to transcribe recorded hearings for the public record and is deciding between Amazon Transcribe and a custom-built automatic speech recognition model. The audio is standard courtroom speech in a supported language, and the department wants to avoid the cost of training and maintaining a speech model. What should guide the decision?
- A public health department is building a symptom-report search tool and is deciding whether a pretrained embedding model is sufficient or whether training custom embeddings on its own clinical notes is justified. Early testing shows the pretrained model already captures semantic similarity between symptom descriptions well. What should guide the decision?
- A city IT team is weighing a single large foundation model against a cascade of smaller, specialized models for a resident-facing assistant, trying to balance response quality against inference cost. What is the key tradeoff the team should reason through?
- A permitting office is deciding between fine-tuning and RAG for a building-code compliance assistant. The relevant regulations rarely change, and the office's top priority is maximizing answer accuracy against the source material rather than minimizing how often the system needs updating. Which approach better fits this priority?
- A county benefits agency must pick a SageMaker modeling option for detecting fraudulent application submissions, weighing training time against detection accuracy under a tight rollout deadline. What should guide the choice?
- A public library system wants to gauge patron feedback sentiment from survey comments and is deciding between Amazon Comprehend's built-in sentiment analysis and training a custom NLP sentiment model. The sentiment categories are the standard positive, negative, neutral, and mixed classes. What should guide the decision?
- A water utility trains an XGBoost model to predict water-main failures and wants to search hyperparameters efficiently rather than manually trying combinations one at a time. What should the team use?
- A county housing agency needs to score housing-assistance eligibility using its own custom feature engineering logic that doesn't match any SageMaker built-in algorithm's expected input format. The team writes a custom SageMaker script-mode training job. What does this decision reflect?
- A transit authority customizes a Bedrock foundation model using prompt-based customization so it consistently answers schedule questions in the agency's preferred tone and format, without retraining model weights. What does this choice reflect?
- A school district is training an attendance-prediction model and wants to cut training time without hurting accuracy. The team configures the training job to halt automatically once validation loss stops improving for several consecutive epochs. Which outcome does this early-stopping configuration most directly produce?
- A public health department is training a large chest X-ray screening model, and the image dataset keeps growing each month as new scans are collected. Single-instance training runs are taking longer every cycle and are starting to delay releases. Which change best keeps training time manageable as the dataset scales?
- A 311 assistant team is training a small service-request classifier on a limited labeled dataset. Training accuracy keeps climbing toward near-perfect while validation accuracy has plateaued far lower and is starting to drop. Which change best addresses this pattern?
- An emergency management agency combines predictions from several independently trained flood-risk models by averaging their outputs, aiming to make forecasts more reliable heading into storm season. Which statement best explains why this ensembling approach tends to improve reliability?
- A city permitting office is tuning a document classifier's training job across epoch count and batch size to reach acceptable accuracy while keeping training cost under control. The team notices that a larger batch size lets each epoch finish faster but the model needs more epochs to reach the same accuracy. Which conclusion best fits this tradeoff?
- A county courts department wants to improve retrieval quality for a case-law search assistant. Generic embeddings return semantically related but legally imprecise matches, often conflating distinct legal terms with similar everyday meanings. The team decides to fine-tune the embedding model on legal terminology. Why is this the appropriate lever for this problem?
- A public housing authority wants a foundation model to broadly understand its internal housing-policy terminology, procedures, and case history before it is used for a casework assistant. The team runs continuous pre-training on its internal policy corpus rather than fine-tuning on a small labeled task dataset. What distinguishes this choice from fine-tuning?
- A city is deploying a lobby kiosk chatbot to answer simple resident questions and wants a cheaper, faster model than the large foundation model currently used for more complex casework elsewhere. The team trains a smaller student model to reproduce the larger model's behavior on this narrow set of simple questions. What technique does this describe?
- A water utility fine-tunes a foundation model on new sensor terminology and maintenance procedures. After fine-tuning, staff notice the model has become excellent at sensor questions but has lost much of its earlier ability to hold a general conversation about unrelated topics. Which approach best mitigates this pattern?
- A county IT team wants to flag irregular patterns in utility billing records without having labeled examples of what billing fraud or errors actually look like. They plan to use a SageMaker built-in algorithm suited to this kind of unsupervised outlier detection. Which consideration should most directly guide their algorithm choice?
- A 311 chatbot team wants to improve handling of multilingual resident queries and is weighing adding few-shot examples through prompt engineering against fine-tuning the model. Early testing shows the base model already understands the languages well but occasionally misapplies the city's specific response format. Which approach fits this situation best?
- A transit authority's RAG-based schedule assistant sometimes returns answers built from irrelevant retrieved passages, and sometimes misses relevant schedule details that were split across two separate retrieved chunks. The team tunes chunk size and top-k retrieval count to address this. Why are these the right parameters to adjust here?
- A permitting office is training a zoning-classification model and chooses to write a custom PyTorch training script run through SageMaker script mode instead of using a SageMaker built-in algorithm. The model needs a nonstandard architecture with a custom loss function tailored to how zoning violations are weighted. Which factor best justifies this choice?
- A city water-quality modeling team has had several staff transitions over the past year, and each new hire has struggled to reproduce prior experiment results because parameters and outcomes were tracked informally in spreadsheets. The team adopts MLflow on SageMaker AI to log experiment parameters and results going forward. What problem does this most directly solve?
- A county benefits office wants to choose between several candidate foundation models for summarizing eligibility determinations before wider rollout. The team uses Bedrock Model Evaluation to run each candidate against a defined summarization task and compare results. What does this evaluation step most directly provide to the team?
- A 311 chatbot team measures response quality by comparing generated answers against reference answers using BLEU and ROUGE scores. A reviewer notices these scores penalize a response that used different wording but conveyed the exact same information as the reference answer. What does this reveal about BLEU and ROUGE as evaluation metrics?
- A public health department evaluates generated symptom-report summaries using BERTScore-based semantic similarity instead of exact word-overlap metrics. A reviewer asks why the team moved away from lexical-overlap scoring for this task. What is the most accurate justification?
- A city establishes a performance baseline for its permit-approval-time prediction model by recording its accuracy on a representative validation set at launch. Several months later, monitoring shows live predictions have drifted noticeably from that recorded baseline. What does this drift most directly indicate the team should do next?
- A transit authority runs a new ridership-prediction model as a shadow variant, receiving live traffic alongside the current production model but with its predictions withheld from riders, before deciding whether to promote it. Why is this evaluation approach useful before promotion?
- A housing authority must justify to a review board why a specific application for rental assistance was denied by its scoring model. The team applies model explainability techniques to produce feature-level reasoning for that individual prediction. What does this evaluation approach provide that overall accuracy metrics do not?
- An emergency management team debugging a flood-prediction model notices training loss oscillating wildly and failing to steadily decrease across epochs, rather than converging smoothly. Which cause is most plausible and directly addressable during training configuration?
- A school district adds a human-in-the-loop review step where staff must approve FM-generated individualized education plan summaries before they reach families. Given how high-stakes and legally sensitive these summaries are, what is the most important design consideration for this review step?
- A county courts system uses an LLM-as-a-judge approach, where a separate foundation model scores whether chatbot-generated legal-information responses are accurate and complete, to evaluate output at a scale human reviewers could not sustain. What is the most important limitation the team should account for when relying on this approach?
- A city evaluates its benefits-scoring foundation model for differences in outcomes across demographic groups before wider rollout, beyond just checking its overall accuracy score. Why is this bias-detection step necessary in addition to overall accuracy?
- A water utility uses Bedrock Prompt Management to version and A/B test prompt variants for its automated leak-report classifier, rather than editing prompts directly in application code each time a change is tested. What advantage does this approach provide?
- A 311 resident-services team's retrieval-augmented assistant answers questions from a knowledge base that grows and changes every week. Six months after launch, more answers cite outdated or missing policy text even though the generation model itself hasn't changed. What should the team do to catch this kind of drift going forward?
- A permitting office runs a hybrid pipeline: a traditional classifier first sorts an application into a permit type, then a foundation model drafts a plain-language explanation for the applicant. How should the team evaluate this pipeline's performance?
- Before committing to a foundation model for production case-management summaries, a county IT team wants to compare several candidate Bedrock FMs on that specific summarization task. How should they use Bedrock evaluations here?
- A transit authority's RAG-based schedule assistant scores well on BLEU and ROUGE against reference answers, yet riders keep reporting confidently wrong schedule times in its replies. What should the team add to its evaluation process to catch this?
Deployment and Orchestration of ML and AI Workflows · 48 questions
- A city permitting-chatbot team must serve sub-second interactive responses to residents typing permit questions throughout business hours, with continuous, predictable traffic. Which inference option best fits this requirement?
- A county 311 call-center intent-classification model receives sporadic, unpredictable bursts of traffic with long idle gaps between citizen contacts. The team wants to avoid paying for capacity that sits idle. Which inference option fits best?
- A public-works department must run inference on scanned permit application PDFs that are individually large and can take several minutes to process, exceeding the payload size and timeout limits of a real-time endpoint. Which inference option should the team use?
- A water utility wants to score its entire customer base overnight for usage-anomaly detection, with no requirement for a persistent endpoint or an immediate response. Which inference approach fits this need?
- A county's data-science team fine-tuned an open-weights foundation model outside AWS and needs a way to bring those model weights into Bedrock for managed inference, without retraining inside SageMaker. What should they use?
- A school district trains a small eligibility-scoring model per individual school — dozens of near-identical models — and wants to avoid provisioning a separate costly endpoint for each one. What deployment pattern fits best?
- A transit authority's ridership-prediction workload requires a preprocessing container to run ahead of the model-inference container on every request, with both steps served from one endpoint invocation. Which deployment pattern fits this requirement?
- A court clerk's office trained a case-category classification model on a data scientist's laptop entirely outside any AWS service, and now needs it hosted in production for clerks to use. What is the right way to get it into service?
- A 311 resident-services assistant must call several municipal backend APIs — pothole reporting and utility billing lookup among them — as part of answering a citizen's question. What should the team configure to support this?
- A city is deploying a permitting agent and a separate records agent that must hand off work to each other mid-conversation as a citizen's request moves between the two domains. What does the team need to configure to support this?
- A public-health department needs high-volume, steady translation of health-education materials via a Bedrock foundation model, running continuously at a predictable rate. Which throughput option best fits this workload?
- A municipal-code Q&A assistant's answers are missing relevant regulation text because its retrieval step only performs keyword search. What retrieval strategy should the team adopt to also capture semantically related matches?
- A permitting-regulations assistant retrieves too many marginally relevant chunks from its knowledge base, diluting answer quality. The team wants to improve relevance ordering without changing the underlying embedding model. What should they add?
- An emergency-management office needs to classify incoming alert reports. Traffic is near-zero on most days but spikes sharply during storm events. Which inference option handles this pattern most cost-effectively?
- An emergency-dispatch office is deciding whether to deploy a domain-fine-tuned foundation model for triage classification or invoke a Bedrock base foundation model directly, weighing accuracy against deployment complexity. How should the team approach this decision?
- A housing-authority eligibility system requires a feature-transformation step and a scoring step to run in strict sequence for every request. What deployment architecture should the team use to chain these two steps reliably behind one prediction call?
- A water-quality inference workload runs at a steady, predictable rate around the clock, and the finance office wants the lowest-cost compute-purchasing approach for that steady baseline versus occasional demand spikes. What should the team do?
- A city IT department needs every department's SageMaker endpoint stack — permitting, 311, transit — deployed identically and reproducibly across dev, test, and prod accounts, without manual console configuration. What should the team adopt?
- A transit authority built a custom inference container for predictive-maintenance scoring and needs to run it at scale outside SageMaker, as a long-lived, always-on ML workload. What kind of service should the team use to run it?
- A public-health department's risk-scoring endpoint processes sensitive patient-adjacent records and must be reachable only from the department's private network, with no path to or from the public internet. How should the team configure the endpoint?
- A county's MLOps engineer wants a model's endpoint automatically redeployed the moment a new version is approved in the model registry, without any manual console step, driven by code invoked from a pipeline. How should the team implement this?
- A permitting office's chatbot runs on a real-time SageMaker endpoint, and the platform team must pick a CloudWatch metric to drive an Auto Scaling target-tracking policy for the endpoint's instance count. Which metric best reflects the actual request load each instance is handling?
- A county already runs its citizen-services data in an Amazon RDS for PostgreSQL database and is adding a Bedrock knowledge base for a resident-facing RAG assistant, with a document corpus in the low tens of thousands of pages. The team is weighing a dedicated managed search service against extending their existing relational database. Which choice best fits this situation?
- A courts department's Bedrock knowledge base keeps returning outdated guidance because newly filed procedural documents aren't reflected in retrieval results. Before launch, the team is designing the chunking and embedding configuration for the ingestion pipeline. Which approach best supports timely, accurate coverage of new documents?
- Residents chatting with a multi-turn 311 assistant find that the agent forgets what they said just a few messages earlier, forcing them to repeat themselves within the same conversation. The team traces this to how the agent handles context between invocations. Which fix directly addresses the problem?
- After a hurricane, an emergency-management office needs to run image-based structural-damage assessment on thousands of drone photographs and must provision compute that handles this workload efficiently. Which choice best matches the job?
- A housing authority is preparing to launch a caseworker-assistant agent that looks up case records and needs somewhere to hold conversation state across a caseworker's multi-step session. Before the agent can go live, what supporting infrastructure must the team provision alongside the agent definition itself?
- A transit authority trains its ridership-demand model on GPU instances but only needs to serve predictions afterward. The platform team is picking compute for each phase separately. Which approach best matches cost to workload?
- An emergency chatbot deployed on a serverless SageMaker endpoint shows noticeable delay on the first request after periods of low traffic, and residents report the assistant feels unresponsive at the start of a conversation. Which endpoint configuration change best addresses this cold-start delay?
- A housing authority's case-management model endpoint must only be reachable from the internal subnets caseworkers use, not from the broader network. Which combination of controls best restricts access at the network level?
- A permitting office's asynchronous document-inference endpoint accumulates a growing backlog of queued requests during peak filing season, and the current Auto Scaling policy reacts to a per-instance invocation-rate metric borrowed from a real-time endpoint. Which metric should the scaling policy target instead for this asynchronous workload?
- A school district's multi-model endpoint hosts dozens of per-school models, and the team notices frequent model swapping and latency spikes because the endpoint is undersized. How should the team right-size the endpoint's instance type?
- A benefits-eligibility fraud-detection model needs to move automatically through dev, test, and production stages as code changes are merged, with each stage gated by an approval or automated test step. Which pipeline design best supports this?
- A permitting-classification endpoint is being updated to a new model version, and the rollout must shift traffic gradually while automatically reverting if the new version's error rate rises. Which deployment approach fits this requirement?
- A 311 assistant's new endpoint version, deployed as a canary, starts showing elevated latency shortly after rollout. The team wants the rollback to happen automatically the moment the problem is detected, rather than depending on someone noticing and intervening manually. Which mechanism achieves this?
- A public-health risk model must not reach production until a designated reviewer explicitly approves the newest trained version. The team wants this approval to be the actual gate that controls deployment, not just a policy people are expected to follow. Which mechanism enforces this?
- A transit authority retrains its ridership-demand model periodically and needs to track and compare metrics across the retrained candidates so the team can decide which one to promote. Which practice best supports this comparison?
- A water utility's anomaly-detection model needs to retrain automatically once monitoring detects that incoming usage data has drifted meaningfully from the training distribution, without anyone manually starting the job. Which design achieves this?
- A county's ML pipeline occasionally fails in production because malformed input data reaches the training job partway through a run, wasting compute time before the failure surfaces. The team wants automated checks that catch schema problems before a training job ever launches. Which pipeline design addresses this?
- A permitting chatbot's prompt template needs to be versioned and promoted from a test environment to production separately from the underlying foundation model, so prompt edits can be reviewed and rolled back on their own. Which approach fits this need?
- A 311 Bedrock agent needs a new version promoted to production behind an alias, while the previous version stays available so the team can roll back if needed, as part of an automated agent-release pipeline. Which mechanism supports this?
- Before promoting a revised prompt template for the permitting chatbot, the team wants an automated test suite that compares the new prompt's outputs against a set of known-good reference answers, catching regressions before the change reaches residents. Which practice best supports this?
- A county fine-tunes a foundation model periodically on updated case-notes data and wants each new fine-tuned version automatically brought into Bedrock with proper version tracking as part of the deployment pipeline, rather than handled as a one-off manual task each time. Which approach fits?
- A municipal-code knowledge base must be reindexed on a recurring schedule whenever new ordinances are published, coordinating document ingestion, chunking, embedding, and vector-store update as one automated workflow. Which approach best orchestrates this multi-step process?
- A school district wants its model-training pipeline to trigger automatically whenever a data scientist pushes updated training code to the district's source repository, with no manual pipeline start required. Which pipeline design achieves this?
- A county's data-science team wants the same training job definition reused across multiple model variants by parameterizing inputs like hyperparameters and dataset location, instead of hand-editing a separate script for each variant. Which approach fits this goal?
- Before fully cutting over an emergency-dispatch triage agent to a new version, an MLOps team wants the new version to process live dispatch traffic and have its outputs compared against the current version's outputs, without the new version's responses ever reaching dispatchers, until confidence is established. Which deployment pattern in the CI/CD pipeline best achieves this?
- A unified citizen-services chatbot release bundles both an updated traditional eligibility-scoring model and a revised foundation-model prompt template. The release must ship both artifacts together in one coherent rollout so they stay in version sync. How should the CI/CD pipeline handle this?
Operating, Monitoring, and Securing ML and AI Solutions · 48 questions
- A housing authority's SageMaker endpoint scores casework eligibility, and applicant income data shifts after a policy change even though the endpoint keeps returning responses without errors. Staff want to catch this shift before it silently degrades decision quality. Which monitoring approach addresses this?
- A city's 311 resident-service chatbot, built on a Bedrock agent, starts cutting off long answers mid-sentence during peak call volume. The ops team wants an automated way to detect this truncated-streaming pattern rather than relying on resident complaints. Which capability should they use?
- A transit authority's ridership-forecasting SageMaker endpoint has predictions quietly diverging from the baseline established at launch. The team wants an approach that flags this gradual divergence automatically, without manually reviewing every batch of predictions. Which approach fits?
- A permitting-office Bedrock agent calls a Lambda tool to check permit status, and the tool intermittently times out, causing the agent to return incomplete answers to applicants. Which monitoring approach directly surfaces this coordination problem?
- A school district's content-review foundation model is periodically re-evaluated to confirm output quality hasn't degraded as prompts and source documents change over the semester. Which capability fits this recurring re-evaluation need?
- A court clerk's office wants to compare a new document-classification model variant against the current production model on live traffic before fully switching over. Which approach lets them make that comparison safely?
- A water utility's leak-detection anomaly model begins producing an unusual spike in invocation errors overnight, and on-call staff need to be paged automatically rather than discovering the problem the next morning. Which setup achieves this?
- An emergency-management dispatch system uses multiple coordinating Bedrock agents, and one sub-agent silently stops responding to hand-offs during a storm event, delaying triage. Which monitoring approach is needed to catch this kind of failure?
- A county open-data portal's RAG search assistant starts returning less relevant document excerpts to residents after a content refresh, even though the underlying foundation model hasn't changed. Which diagnostic path should the team prioritize?
- A public-health department's case-report text classifier begins seeing incoming reports that look statistically different from its training corpus after a new intake form rolls out. Which monitoring capability is needed to catch this?
- A permitting chatbot team wants a dashboard that surfaces a rising hallucination rate across resident conversations, flagged by automated evaluation rather than manual transcript review. Which combination of capabilities supports this?
- A transit maintenance-prediction model's accuracy is checked periodically against the baseline recorded at deployment time, with alerts triggered when accuracy falls below an agreed threshold. What monitoring practice does this describe?
- A 311 assistant's agent begins truncating streamed responses more often specifically during evening peak load, and staff need to confirm the pattern correlates with load rather than a recent code regression. How should they investigate?
- A housing authority's document-extraction pipeline built on Textract, feeding a downstream model, shows a sudden spike in extraction errors after a new scanner is introduced. Which monitoring setup helps the team catch and localize this kind of issue going forward?
- A school-district assistant team wants to validate a newer foundation model against the currently deployed one before swapping the production model ID. Which approach gives them a reliable basis for that decision?
- A courts docket-summarization agent occasionally fails to retrieve case details from a case-management API tool, and staff need automated alerting when this tool-call failure rate rises. What should the team configure?
- A county deploying a custom document-classification model to a SageMaker endpoint needs to pick an instance family that balances the permitting chatbot's latency needs against monthly compute cost. What should guide this decision?
- A transit authority's steady, always-on forecasting endpoint is currently billed on-demand, and the team wants to know when a longer-term purchasing commitment would actually reduce cost. What should guide that decision?
- A 311 assistant's Bedrock inference bill grows faster than call volume, and the team traces the mismatch to unusually long prompts being sent per resident query. Which monitoring approach would have surfaced this cost driver earlier?
- A housing authority's RAG chatbot recomputes document embeddings on every query, and the team wants a cost-optimization change that doesn't hurt answer freshness. Which change addresses this?
- A county's OpenSearch Service vector index backing an open-data RAG assistant grows steadily each month, and finance asks engineering to explain and control the storage cost trend. What should engineering identify as the underlying cost pattern to monitor and manage?
- A water utility team wants one place to see both latency and spend for its leak-detection inference endpoint, so they can spot a performance regression and a cost spike together rather than checking two separate tools. What should they build?
- A school district's multi-agent tutoring assistant handles many concurrent student sessions, and administrators want visibility into which sessions are consuming disproportionate compute and tokens. Which capability gives them that per-session view?
- An emergency-management department's multi-step triage agent feels sluggish end to end, and engineers need to see exactly which tool call inside the chain is adding the most latency before they change anything. Which approach pinpoints the slow step?
- A permitting office wants to be notified automatically the moment its monthly Bedrock inference spend crosses a set dollar threshold, well before the finance team's monthly report is due. Which approach meets this need?
- A public-health analytics team runs large nightly batch-inference jobs and wants to cut compute spend without changing the model or missing the overnight turnaround window. What should the team do?
- A transit authority discovers several SageMaker endpoints still running at low utilization long after the pilot programs that used them wrapped up, quietly consuming budget every day. What is the appropriate fix?
- A 311 resident-service agent's observability data shows one particular tool integration consistently driving excess token consumption compared to the agent's other tools. How should the team use this signal?
- A county's RDS-with-pgvector knowledge base backing its open-data assistant costs more to query than the team expects at its current configuration, and they want to bring per-query cost down. Which lever should they examine first?
- A housing authority runs predictable nightly batch-transform casework-scoring jobs alongside a spikier interactive agent that resident-facing staff use during business hours. How should the authority choose purchasing options for the two workloads?
- A school district runs a model-evaluation job against every single prompt-template change and notices its evaluation costs climbing faster than the pace of actual model updates. What should the district do?
- A courts clerk office's document-classification endpoint is provisioned for business-hours peak load but sits mostly idle overnight, and the team wants to see this pattern clearly before deciding what to change. What should they do first?
- A permitting chatbot's backend service, running as an AWS Lambda function inside the office's own account, needs to call a Bedrock model. The team is deciding between issuing a Bedrock API key and attaching an IAM role to the function for this access. Which choice best fits least-privilege service-to-service access?
- A housing authority's casework assistant must never surface an applicant's Social Security number in a generated response, even when that number appears in the retrieved case notes the model is grounded on. Which control addresses this directly?
- A court clerk's document-inference SageMaker endpoint handles sensitive docket text and must not be reachable from the public internet. Which deployment choice satisfies this requirement?
- A school district's CI/CD pipeline for model-training code needs an automated check that catches insecure code patterns before the code is deployed. What should the pipeline add?
- A transit authority's container image for a SageMaker inference job must be scanned for known vulnerabilities before it is pushed to the registry used in production. Which capability fits this step?
- A county IT team notices a data scientist's IAM role has access to an entire S3 bucket when the role's job only requires reading a single training-data prefix within that bucket. What is the appropriate remediation?
- A public-health department must be able to show an auditor exactly which staff account invoked a particular Bedrock model call and when, as part of a records-handling review. Which service provides that auditable record?
- An emergency-management department wants continuous detection of any SageMaker endpoint deployed without encryption at rest, across all of its AI workloads, rather than a one-time manual audit. Which approach fits?
- A water utility's SageMaker notebook instance, used to analyze SCADA sensor data, should only be able to reach the specific internal services it needs rather than the broader network. Which combination of controls achieves this?
- A permitting chatbot occasionally drifts into answering unrelated or inappropriate resident questions, and the office wants an automated boundary on the topics and content the agent will respond to, rather than relying on manual review of transcripts. What should the office configure?
- A housing authority discovers a cross-account trust relationship that lets a partner agency's role read its model artifacts more broadly than the partnership actually requires. What is the correct remediation?
- A courts docket-summarization agent needs to let a third-party vendor's external system call it, and the office must decide which credential mechanism fits that integration. Which choice is the better fit?
- A school district's training-data S3 bucket holding student records must be encrypted with a customer-managed key, and only the training role should be able to decrypt objects in that bucket. What should the team configure?
- A county's open-data chatbot has a public-facing endpoint and an internal knowledge-base backend, and the architecture must keep the two properly separated on the network. How should the county lay this out?
- A 311 resident-service assistant program must maintain a defensible record of every configuration change made to its deployed agent, ahead of an upcoming compliance review. Which combination of services builds that auditable trail?
- A transit authority's CI/CD pipeline deploys SageMaker models automatically, and the deployment role currently has access to every model package group in the account instead of just the one it manages. What is the appropriate fix?
These questions are original practice material and are NOT actual exam questions or brain-dump content. All vendor marks are trademarks of their respective owners. This site is not affiliated with, endorsed by, or sponsored by the exam vendor.