In a MapReduce job counting 311 reason codes, one mega-popular code sends almost all intermediate values to a single reducer and the job crawls. What problem does this illustrate?
Select an answer to reveal the explanation.
Short Explanation
If one mail bin gets ninety percent of the city's letters, that clerk becomes the bottleneck. Hot keys create data skew in MapReduce. Even partitions and healthy YARN scheduling are the opposite story.
Full Explanation
When one mega-popular 311 reason code sends almost all intermediate values to a single reducer and the job crawls, the problem illustrated is data skew: a hot key overloads one reducer and slows the municipal job. Perfectly even partitions that always finish in lockstep describe the opposite of this symptom. HDFS block size expanding automatically to zero bytes is not what hot keys cause. YARN forbidding more than one map task citywide is not the illustrated failure mode. Exam caveat: skew is a reducer-side hotspot from uneven key distribution—recognize symptoms and mitigations. Operational check: inspect reducer input records per task for the 311 job, identify the hot reason code, and apply salting, custom partitioning, or pre-aggregation techniques before the next production run. In municipal platform operations, record the rationale in the architecture decision record so future on-call staff know which pattern was chosen and which failure modes the rejected options would create for city services. Re-validate after the next maintenance window that monitoring, access paths, and recovery steps still match the documented design for agencies that depend on this platform. Prefer evidence from job counters, catalogs, and rehearsed runbooks over tribal knowledge when explaining the design to auditors or council technology committees.