A housing authority's document-extraction pipeline built on Textract, feeding a downstream model, shows a sudden spike in extraction errors after a new scanner is introduced. Which monitoring setup helps the team catch and localize this kind of issue going forward?
Select an answer to reveal the explanation.
Short Explanation
When a pipeline has multiple steps and something breaks, watching only the very last step is like checking a relay team's final time without knowing which leg slowed everyone down. Setting up CloudWatch error-rate monitoring at each stage, extraction, then downstream scoring, lets you see exactly where a spike originates. Here, that pinpoints the Textract extraction step right where the new scanner came in.
Full Explanation
Stage-by-stage CloudWatch error-rate monitoring instruments each step of the workflow independently, so an extraction-error spike introduced by the new scanner shows up clearly at the Textract stage rather than being blended into a single end-to-end signal, letting the team localize the root cause quickly. Monitoring only the downstream model's overall accuracy captures the final effect of the problem but obscures where it started, since a degraded extraction feeding the model looks the same, from that single metric's perspective, as the model itself performing worse. A monthly manual audit of already-processed documents is both slow and backward-looking; by the time staff catch the pattern, weeks of documents have already been affected. A fixed quarterly retraining schedule addresses model staleness on a calendar, not an acute extraction-quality regression tied to new scanner hardware, and retraining doesn't fix garbled input data anyway. Scope caveat: per-stage monitoring adds instrumentation overhead, so it's worth prioritizing the stages most likely to change, like a newly introduced hardware dependency, rather than instrumenting every internal step equally. A concrete operational check: compare the Textract extraction error rate before and after the new scanner's rollout date to confirm it's the originating stage.