Parallel processing with MapReduce

BDPC · 30 questions

• The city wants a parallel count of citation reason codes across a huge distributed ticket archive. How does MapReduce organize that work?
• During a citywide MapReduce job, something must assign map and reduce tasks to workers on the cluster. Which role description fits?
• Staff training asks for the correct order of MapReduce stages for a batch 311 analytics job. Which sequence is correct?
• An instructor compares MapReduce to filtering lines with pipeline commands and then aggregating counts. What teaching point is that analogy meant to convey?
• Analysts must count how often each service category appears across millions of 311 text blurbs. Which MapReduce thinking applies?
• A city 311 desk runs a MapReduce job over complaint tickets so analysts can tally each reason code. What should each mapper emit during the map phase?
• After mappers emit (reason-code, 1) pairs for municipal 311 tickets, what must the shuffle and group stages guarantee before reduce runs?
• In a MapReduce tally of transit delay reason codes, what is the reducer's primary responsibility?
• A metropolitan planning office needs last year's mobility indices recomputed overnight from archived fare-tap and sensor files. Why is classic MapReduce a strong fit?
• A municipal payment portal must raise a fraud flag within a fraction of a second on each card authorization. Why is overnight MapReduce a poor primary design?
• While counting commutative 311 reason codes, engineers want fewer bytes crossing the network during shuffle. Which optional MapReduce idea helps?
• A county open-data lake stores permit archives as many HDFS blocks. What generally increases how many map tasks can run in parallel for a MapReduce job?
• During a citywide log-parse MapReduce job, one mapper node crashes mid-task. What resilience behavior should operators expect in a classic MapReduce framework?
• Housing analysts who already write SQL want ad-hoc summaries over Hadoop tables, while an ETL team prefers scripted dataflow transforms. How should the city choose between Hive and Pig?
• A housing open-data team wants ad-hoc, SQL-like queries over tables stored on HDFS without hand-writing low-level MapReduce Java for every ask. Which layer best matches that need?
• Public-works engineers need scalable ETL-style transforms to clean messy permit text fields before loading a warehouse. Which Hadoop-era tool is aimed at that scripting/dataflow style?
• A parks department proposes putting live festival ticket checkout—inventory locks, payments, and immediate confirmations—on Hive tables in HDFS. What is the sound judgment?
• A city needs to re-encode millions of archived sensor files into a new columnar layout without aggregating across keys. Which MapReduce pattern awareness is most relevant?
• In a MapReduce job counting 311 reason codes, one mega-popular code sends almost all intermediate values to a single reducer and the job crawls. What problem does this illustrate?
• A few map tasks on a slow municipal worker drag out an otherwise healthy MapReduce job. Which classic framework idea races duplicates to mitigate stragglers?
• Data stewards want to confirm how many permit records entered and left each stage of a nightly MapReduce cleanse. What MapReduce observability feature supports that check?
• Building a mobility data mart requires cleanse, then join, then aggregate stages that do not fit cleanly in one MapReduce job. What pipeline approach is appropriate?
• Transit analysts need to join massive fare-tap logs to daily weather observations by date using Hadoop-era batch processing. Which statement best frames the MapReduce role?
• When designing a MapReduce job that totals citations by violation type, what primarily determines which values a reducer sees together?
• Shuffle traffic between mappers and reducers is saturating municipal cluster links during a large MapReduce job. Which performance lever directly targets intermediate map output size on the wire?
• Each night a city web team must parse raw access logs into structured session features for analysts. Why is batch MapReduce a natural fit?
• In a modern Hadoop deployment used by a regional transit agency, how does MapReduce relate to YARN?
• Analysts only need a straightforward group-by on already-cataloged Hive tables of recycling-tonnage facts. What is the better abstraction choice?
• After a MapReduce mobility job finishes successfully, where do durable final results typically land for downstream civic services?
• A municipal data platform already runs nightly MapReduce batch jobs on HDFS and plans NoSQL serving plus Spark streaming next. How should leaders view MapReduce's place?