An emergency-management team streams 911 dispatch events through Amazon MSK into a pipeline that trains an anomaly-detection model on dispatch patterns. A colleague suggests replacing MSK with a nightly batch export instead. Which reason best justifies keeping the Kafka-based streaming approach?
Select an answer to reveal the explanation.
Short Explanation
Think of a batch export like checking the mailbox once a night — fine for routine mail, useless if the house is on fire and you needed to know an hour ago. Anomaly detection on 911 dispatch patterns is exactly the know-now case: a surge in calls needs to surface quickly, not at tomorrow's batch run. Kafka's streaming model keeps that detection loop close to real time, which is the actual reason to keep it, not a blanket rule about public-safety data.
Full Explanation
The deciding factor for a streaming architecture is whether the downstream use case needs low-latency signal, and an anomaly-detection model watching for incident surges is a textbook case: every hour of delay is an hour where an emerging pattern (a mass-casualty event, a widespread utility failure) goes undetected. MSK's Kafka-compatible streaming lets consumers process dispatch events continuously, keeping the anomaly model's view close to current. Justifying Kafka as the standard for public-safety data as a category, independent of the specific latency need, is the wrong kind of reasoning — the right justification is tied to what the downstream consumer actually requires, and a batch-friendly workload (say, monthly reporting) wouldn't need streaming at all. Leading with a blanket cost claim skips the actual requirement entirely; relative cost varies by data volume and retention needs, and it isn't what should drive the architecture decision here. Claiming a nightly batch job can't read from S3 is simply false — batch jobs read from S3 constantly — and it isn't relevant to the latency argument in the first place. Scope note: if the anomaly model later moves to a daily retraining cadence rather than live scoring, batch ingestion becomes a legitimate simplification. Operational check: measure the anomaly model's actual detection lag under the streaming pipeline and confirm it beats the incident-response window the emergency-management team needs.