A housing chatbot team must choose among Amazon Bedrock foundation models under a fixed monthly budget. Which evaluation approach best supports that municipal selection decision?
Select an answer to reveal the explanation.
Short Explanation
Buying a foundation model is like picking a city fleet vehicle: check cost per mile and whether it actually hauls the load. Price-to-performance tests on real housing prompts beat “cheapest” or “fanciest” guesses.
Full Explanation
Under a fixed monthly budget, a housing chatbot team must choose Amazon Bedrock foundation models with evidence, not brand or price extremes. Measuring price-to-performance on representative housing prompts—quality metrics, latency, refusal behavior, and token cost—lets the municipality pick the model that delivers acceptable caseworker outcomes per dollar. That evaluation framework is the Domain 4 selection practice: compare candidates under the same civic workload and budget constraint rather than guessing.
Picking the cheapest model without checking answer quality on housing tasks risks unusable or unsafe guidance that fails residents and creates rework. Picking the most expensive model and assuming quality always justifies the cost can exhaust the monthly budget without proving incremental value on the actual prompts. Ignoring pricing tables and selecting by marketing names skips both cost and measured performance.
Exam caveat: FM selection questions reward structured price-to-performance evaluation on representative tasks; cheapest-only, dearest-only, and marketing-name choices are distractors. Operational check: build a scored eval set from real housing intents, run each candidate Bedrock FM under identical prompts and decoding settings, plot quality versus $/1K tokens, and document the selection rationale for procurement and audit.