A permits vendor CSV sometimes adds unexpected columns. What should bronze ingest do?
Select an answer to reveal the explanation.
Short Explanation
Schema enforcement is the permits front desk saying "we don't take mystery columns"—unexpected CSV fields don't widen bronze unchecked.
Full Explanation
When a permits vendor CSV suddenly adds unexpected columns, bronze ingest should use schema enforcement so those fields do not silently widen the table. Enforcement fails or quarantines drift until stewards approve a schema version—protecting municipal pipelines from surprise columns that break downstream contracts. Controlled evolution beats uncontrolled widening on every vendor whim.
Always appending every new vendor column into bronze with no review fails by concept: unapproved drift flows straight into the lakehouse. Disabling schemas entirely so CSV structure never matters abandons contract validation. Renaming all columns to col1, col2 on every load makes enforcement and semantic understanding impossible.
Exam caveat: Schema enforcement vs. schema evolution/merge modes is a frequent exam contrast. Enforcement is correct when unexpected columns must not widen unchecked; versioned evolution is used when optional fields are intentionally adopted. Anonymous colN renaming is not a strategy.
Operational check: Submit a vendor CSV with an extra column under enforcement and confirm the ingest fails or lands in quarantine without altering the production bronze schema. After steward approval, version the schema and replay. Verify dashboards still bind to approved column names only.
Prefer to study offline? Printable PDF study book + test engine ($7.50, unofficial). These questions stay free here.