A water utility wants Spark to enrich live meter readings with last year's HDFS partition history while new deltas keep arriving. How should that architecture be described?

A water utility can use Spark to read last year’s HDFS partition history for context while also processing streaming live meter deltas—batch historical enrichment plus streaming ingest in one architecture. Spark does not require deleting HDFS history before streaming starts, nor is it limited to browser clickstreams while only MapReduce may touch HDFS. Claiming live streams and HDFS history cannot share a municipal pipeline is a false exclusivity; enrichment joins against historical partitions are a common pattern. Replacing HDFS entirely so historical partitions must be deleted before streaming starts throws away governed context meters need for anomaly baselines. Exam caveat: align watermarks and partition filters so enrichment reads do not scan the entire multi-year lake on every micro-batch. Operational check: run a job that joins a live meter stream to a filtered prior-year HDFS partition, verify enriched fields appear on new events, and measure that historical reads stay bounded by partition predicates. In production terms for Spark enrichment of streams with HDFS history, assign an owning team, wire a dashboard that shows the pattern is healthy under peak civic load, and require a staging drill that rehearses failure and recovery before resident impact. Revisit the choice when traffic shape, retention, or integrity needs change, and keep a named rollback owner on the change record. Re-test both the happy path and the failure path for Spark enrichment of streams with HDFS history after material dependency upgrades so regressions are caught before the next public release.