A county runs both live 311 chat and overnight research summarization on Amazon Bedrock. Which model-selection practice best optimizes performance for the live path?
Select an answer to reveal the explanation.
Short Explanation
Live 311 is a phone call; overnight research is a homework packet. Put the snappy, latency-optimized Bedrock model on the phone line and leave the heavy lifter for batch jobs.
Full Explanation
A county that runs both live 311 chat and overnight research summarization on Amazon Bedrock should select models by interaction mode. Latency-optimized Bedrock models belong on the live 311 path so residents and agents get timely turns; heavier, higher-capacity models can stay on batch research jobs where wall-clock delay is acceptable. Matching model latency profiles to workload type is a core performance and cost optimization practice for mixed municipal GenAI estates.
Forcing the slowest research model into every live 311 turn for “consistency” sacrifices responsiveness without helping overnight jobs and routinely misses chat SLAs. Using only batch-oriented models for interactive chat to save engineering effort similarly violates chat latency needs while underusing the right tool for each path. Ignoring latency SLAs and selecting solely by parameter-count marketing claims skips measured performance for branding.
Exam caveat: Live versus batch model selection is a recurring AIP-C01 performance theme—optimize the interactive path for latency. Operational check: define separate model IDs and routes for chat and batch, load-test p95 chat latency against the SLA, keep batch jobs on the heavier model overnight, and prevent configuration drift that points 311 at the batch model.