Permit-office staff update a transactional database all day, and analysts need those changes as near-real-time events on Kafka. Which ingestion pattern fits?
Select an answer to reveal the explanation.
Short Explanation
When clerks edit the permit DB, those row changes can ride out as events—CDC is the tap on the pipe. Kafka then feeds analytics without waiting for a yearly dump or retyping. Live changes, streamed once.
Full Explanation
Change-data-capture style pipelines stream inserts, updates, and deletes from the permit office’s transactional database into Kafka as near-real-time events. CDC preserves a chronological change feed while leaving clerks on their system of record, decoupling OLTP from downstream civic analytics consumers. It is a core modern ingestion pattern alongside messaging buses for BDPC awareness and keeps analysts from waiting on annual dumps.
Exporting the entire database to HDFS only once a year reintroduces batch latency and misses day-of permit changes analysts need for operational dashboards. Asking clerks to retype every update into a separate analytics spreadsheet creates dual entry, inconsistency, and audit risk across the permit office. Disabling database writes during business hours to keep analytics consistent freezes the operational mission that the transactional database exists to serve.
Exam caveat: CDC still requires handling of schema changes, deletes, snapshot bootstrap, and exactly-once or idempotent sink design; enabling a connector is not a complete architecture by itself. Operational check: perform a controlled update and delete in a non-production permit database and confirm corresponding events appear on the Kafka topic with keys and operation types the analytics consumers expect before promoting CDC to production.