A county public-health analytics team stores multi-year case extracts and usually queries only the current fiscal year on GPU. How should Parquet be laid out to help scans skip unused years?
Select an answer to reveal the explanation.
Short Explanation
If analysts almost always ask about this fiscal year, do not make the GPU dig through a decade in one box. Partition Parquet by fiscal year so unused year folders get skipped like closed archive drawers.
Full Explanation
A county public-health analytics team storing multi-year case extracts that usually query only the current fiscal year should partition Parquet by fiscal year so RAPIDS scans can prune directories outside the requested year. Partitioning aligns physical layout with the dominant civic filter, letting GPU jobs skip unused year folders instead of opening a decade of cases for every current-year question. That is the layout choice that turns “we almost always want this fiscal year” into cheaper, narrower reads. Concatenating all fiscal years into one unpartitioned Parquet file fails by concept: readers must open the combined object even when only one year is needed. Partitioning randomly by ingestion UUID fails because year filters cannot prune directories meaningfully. Storing each column as a separate CSV named by year without a shared schema fails by abandoning a coherent tabular contract and forcing fragile joins and reparsing. Exam caveat: partition columns should match actual filter predicates; partitioning only by fiscal year does not help if most queries filter by program or facility instead—extend or compose partitions carefully. Operational check: write case extracts partitioned by fiscal year, query the current year through RAPIDS, confirm other year directories are pruned from the read set, and measure bytes scanned versus an unpartitioned multi-year control.