Stream processing with Spark

BDPC · 30 questions

• A traffic bureau wants to act on sensor events as they arrive instead of waiting for tonight’s batch files. What does stream computing emphasize?
• The same automatic vehicle location feed can produce a nightly MapReduce ridership summary or a continuous delay detector. What contrast should planners remember?
• Bridge strain gauges must page on-call engineers within seconds when readings cross a safety threshold. Which streaming application class is this?
• A municipal operations center wallboard must refresh curb-occupancy and incident counts throughout the day from live feeds. What streaming use does this illustrate?
• Street-sensor analytics must run with limited memory while seeing only one pass or a sliding window of events. What trait should streaming algorithms accept?
• A license-plate camera desk wants a fast test of whether a plate was already seen today without storing every plate string in a huge exact set. Which structure fits?
• Operators ask what a standard Bloom filter can get wrong when testing whether a plate is in today’s set. Which statement is accurate?
• A city selects Apache Spark as the engine for micro-batch and stream analytics on curb-sensor events. What does that choice primarily recognize?
• An iterative model that revisits the same civic graph many times runs far slower on classic disk-heavy MapReduce than on Spark. Why?
• A municipal architect places Spark beside HDFS, YARN, Kafka, and related tools rather than treating it as a lone product. What idea does that convey?
• On a Spark cluster analyzing bus GPS partitions, which mental model of roles is correct?
• Training material describes Resilient Distributed Datasets for a flood-sensor analytics job. What core idea should staff take away?
• Developers chain map and filter calls on sensor RDDs, then call a count that finally returns a number. What distinction matters?
• A Spark job on neighborhood sensor partitions runs as pipelined work across the cluster. How is that execution commonly described at a high level?
• A syllabus example uses Spark for PageRank-style scoring of linked civic web pages. What capability does that highlight without requiring code memorization?
• Analysts must count 311-style social alerts in sliding five-minute windows as messages arrive. Which streaming analytics pattern is this?
• Duplicate curb-sensor events could inflate occupancy counts if the streaming job treats every delivery as unique. What concern does that raise?
• A streaming app tracks each bus’s last-seen timestamp so stale vehicles can be flagged. What concept does maintaining that per-bus value illustrate?
• A team describes Spark Streaming–style processing as short batches fired back-to-back to approximate continuous work. What mental model is that?
• A public-works ETL that once ran as classic MapReduce is being rewritten as Spark batch jobs for speed and richer APIs. What shift does this represent?
• A transit analytics team caches several huge AVL history tables in Spark memory just in case, and executors begin failing with out-of-memory errors. What lesson should the program lead take?
• A water utility wants Spark to enrich live meter readings with last year's HDFS partition history while new deltas keep arriving. How should that architecture be described?
• A finance desk wants unusual permit-fee refunds flagged as payment events stream in, not only after a nightly batch. Which approach fits that need?
• Public works needs continuous plow GPS points to refresh street coverage maps during a storm. What processing pattern matches that need?
• Flood sensors publish faster than a city's stream processors can keep up. What operational concept should architects plan for?
• After a node failure, a parking-citation streaming job must resume without losing its rolling window state. What mechanism supports that recovery?
• A clerk needs a nightly 10-row lookup of holiday parking exceptions. What is the most appropriate tooling judgment?
• A web crawler for open-data portals fetches many duplicate URLs before expensive parsing. What lightweight structure can help dedupe early in the ingest path?
• Analysts want familiar SQL to join large municipal DataFrames in Spark without writing low-level RDD code. What access style does that describe?
• A city operations center streams 311 and AVL events through messaging topics that Spark jobs continuously consume. What typical pairing does this illustrate?