A 311 GenAI service scales only on CPU utilization, yet outages still occur during token-burst traffic when CPUs look moderate. Which autoscaling improvement best fits GenAI traffic patterns?
Select an answer to reveal the explanation.
Short Explanation
GenAI rush hour shows up as token bursts, not always a hot CPU gauge. Point autoscaling at concurrency and token-shaped signals so 311 capacity rises when the model traffic does.
Full Explanation
A 311 GenAI service that scales only on CPU can still fail during token-burst traffic when CPUs look moderate, because foundation-model load often stresses invocation concurrency, queue depth, and token throughput more than classic CPU saturation. Autoscaling policies should therefore follow GenAI traffic patterns—token bursts, concurrent invocations, and related signals—not CPU alone. That alignment keeps municipal capacity rising when model demand rises and prevents the outage mode described in the stem.
Keeping CPU-only scaling and ignoring token-burst and concurrency signals repeats the failure pattern: CPUs stay calm while Bedrock-facing workers saturate. Scaling exclusively on disk free space tracks an unrelated resource that does not reflect FM demand. Disabling autoscaling and pinning a single instance for citywide 311 load creates a hard capacity cliff during bursts and removes elastic recovery.
Exam caveat: GenAI autoscaling metrics differ from web CPU heuristics—prefer concurrency and token-oriented signals on AIP-C01 performance items. Operational check: publish custom metrics for in-flight invocations and tokens/sec, configure scale-out on those thresholds, load-test a token-burst scenario, and verify scale-in cool-downs so capacity does not thrash after short spikes.