Environmental sensors drop millions of tiny one-reading files onto HDFS each day. What classic problem should the platform team anticipate?
Select an answer to reveal the explanation.
Short Explanation
A library card catalog with a card per sticky note collapses under its own weight. Millions of tiny sensor files do that to the NameNode’s metadata. Compact or batch them—don’t celebrate a flood of postage-stamp files as healthy HDFS design.
Full Explanation
Millions of tiny one-reading environmental sensor files dropped onto HDFS each day create the classic tiny-files problem: NameNode metadata stress and inefficient classic HDFS usage for municipal IoT ingest at scale. HDFS does not refuse to store any file smaller than a full rack of disks as a hard physical rule of that form. YARN will not automatically merge all tiny files into one OLTP database table as a hidden service. MapReduce does not become unnecessary because tiny files never need processing—they still need processing, preferably after compaction into larger files. Exam caveat: tiny files hurt NameNode scalability; design ingest to roll up or containerize small objects. Operational check: monitor NameNode file and block counts, introduce rolling compaction or sequence/container files for sensor readings, and alert when daily tiny-file ingest exceeds a governance threshold. In municipal platform operations, record the rationale in the architecture decision record so future on-call staff know which pattern was chosen and which failure modes the rejected options would create for city services. Re-validate after the next maintenance window that monitoring, access paths, and recovery steps still match the documented design for agencies that depend on this platform.