A solid-waste route analytics team must speed Dask processing of EMS CAD extracts. What partitioning strategy helps GPU/Dask workers?
Select an answer to reveal the explanation.
Short Explanation
When you partition CAD logs by day, you give each Dask worker a manageable shard instead of the whole year’s route sheets. Parallelism lets workers process those shards on GPUs together, while one uncompressed blob makes efficient scheduling harder.
Full Explanation
Partitioning EMS CAD extracts by day gives Dask workers natural shards to read and transform in parallel on GPUs. Time-based partitions fit how solid-waste and emergency analytics typically filter and reprocess civic logs, improve cache locality, and avoid making every worker scan a monolithic file. This is a core scalable GPU/Dask ETL practice for multi-year municipal archives. Storing the entire multi-year extract as one uncompressed text blob maximizes scan cost and obstructs parallel shard scheduling. Having each worker read the full unpartitioned extract anew for every task multiplies I/O without adding useful parallelism. Merging all days into a single GPU that cannot spill or redistribute creates a single-node bottleneck and risks out-of-memory failures as the archive grows. Exam caveat: partition granularity should match query and reprocessing patterns; hourly partitions may over-fragment small days, while monthly partitions may be oversized for peak incident days. Practical check: write day partitions, run a Dask/RAPIDS job targeting a week of partitions, verify that only those shards are touched, and compare runtime and peak memory with the unpartitioned extract using the same cleaning logic.