A dozen regional branches each export their collection-management catalogue on a different schedule and periodically add a new field — one branch recently started including a 'condition_grade' column that no other branch has yet. A nightly load process ingesting these exports into a Lakehouse table needs to keep working without manual schema edits every time a branch changes its export. What should the ingestion approach do to handle this?
Select an answer to reveal the explanation.
Short Explanation
Think of schema drift handling like a form that automatically grows a new field when someone fills one in, instead of the whole stack of forms getting thrown out because they don't all match. Letting the Delta table's schema evolve keeps the pipeline running as branches change what they send.
Full Explanation
Handling schema drift means designing the ingestion path — whether a Dataflow Gen2 or a Spark write with schema evolution enabled (for example, a mergeSchema-style option) — to accept new or missing columns from a source without failing, and to extend the target Delta table's schema automatically when a genuinely new column like condition_grade shows up. That keeps the nightly load resilient as branches independently change their exports over time, which is exactly the stated goal of no manual intervention per change. Rejecting any export that doesn't exactly match every other branch's columns guarantees the pipeline breaks the moment a single branch adds a field, which is the opposite of resilience and would require constant manual fixes. Manually editing the target DDL every time a branch changes its export is the exact manual burden the scenario says must be eliminated — it doesn't scale across a dozen independently-changing branches. Flattening every export into a single text column avoids schema conflicts only by discarding structure entirely, making the data useless for typed queries, aggregation, or joins downstream. Before trusting automatic schema evolution in production, review new columns periodically to confirm they're genuine additions and not a sign of a broken export on the source side.