Distributed computing with Hadoop

BDPC · 30 questions

• Citywide smart-meter history no longer fits or processes in time on one powerful server. What is the core rationale for moving to cluster computing?
• Council staff want a plain-language briefing line that defines Hadoop without vendor hype. Which statement is most accurate at concept level?
• IT staff use “Hadoop” and “MapReduce” as if the words were identical. How should a BDPC-oriented explanation separate them?
• Meter archive files must be processed in parallel across the cluster. What foundational distributed-computing practice enables that parallelism?
• Architects brief operations on Hadoop-style cluster roles. Which description matches the classic coordinator versus worker pattern?
• Why would the city store multi-terabyte camera index archives on HDFS rather than relying on a conventional NAS share alone?
• An analyst job reads a large HDFS dataset. Conceptually, how does the read path involve cluster roles?
• Ingest pipelines write large files into HDFS for durable analytics storage. What conceptual write behavior should operators expect?
• Which HDFS characteristic best matches typical Big Data analytics workloads for municipal archives?
• A DataNode fails overnight, yet morning mobility jobs still find their input blocks. Which HDFS characteristic primarily explains that resilience?
• A city GIS archive of multi-terabyte aerial imagery must land on the municipal Hadoop cluster. How does HDFS typically store each huge file?
• Public-works ops asks which Hadoop role holds the namespace metadata versus the actual block bytes for storm-drain sensor archives. Which distinction is correct?
• Planning, transit, and public-health teams all want CPU and memory on the same city Hadoop cluster. What is YARN’s primary role?
• A county architect insists storage and resource management are separate Hadoop layers. Which statement correctly separates HDFS from YARN?
• Each month the city mashes multi-agency open-data dumps for a retrospective equity report. Where should that heavy batch mashup run?
• Next year’s IoT expansion will roughly double traffic-sensor volume on the municipal lake. How does a Hadoop cluster typically grow capacity?
• Transit analysts want MapReduce tasks to run close to the AVL block replicas they read. What benefit does data locality primarily provide?
• A parks department has a 2 GB CSV of playground inspection notes used by three analysts. What is the sound capacity judgment?
• Council marketing claims HDFS replication means the city never needs operational practices for disk or node loss. What is the accurate view?
• A lab standup checklist says the city will configure HDFS on the new cluster. At BDPC depth, what should staff understand?
• Planning’s equity model and public-works’ pavement job both submit heavy work to the shared city cluster. What multi-tenant concern should capacity planning address?
• Years of bus AVL archives are rarely queried but must stay available for deep historical studies. What role fits HDFS well?
• Daily 311 partitions land once as immutable files and many overnight jobs read them. Which HDFS-aligned pattern is that?
• Environmental sensors drop millions of tiny one-reading files onto HDFS each day. What classic problem should the platform team anticipate?
• When placing HDFS replicas for water-quality archives, why do replication policies care about racks or failure domains?
• Semi-structured JSON 311 payloads and free-text inspector logs need a landing zone before schema-on-read jobs. How does HDFS support that Variety need?
• A vendor claims a Hadoop appliance removes any need for architecture thinking about storage, resources, and workloads. How should the city respond?
• Jobs that ignore locality haul raw blocks across the data-center fabric during large shuffles. What constraint becomes visible?
• Analysts across departments can read shared open-data paths, but not everyone should delete sensitive HDFS directories. What awareness is required?
• After HDFS and YARN are in place, the analytics team wants parallel batch processing over those stored blocks. How do the layers relate?