A team plans to run transformation logic directly against production tables, overwriting them in place with each run. What is the main objection?
Select an answer to reveal the explanation.
Short Explanation
If you overwrite in place, a bad run doesn't fail safely—it fails onto everyone. Build into a staging table, validate it, then swap. That way your worst outcome is a stale table rather than a wrong one.
Full Explanation
The main objection is that a failed or incorrect run leaves consumers with corrupted production data and no clean prior state to fall back to. Destructive in-place transformation offers no isolation between the run and its consumers. If the logic is wrong or the run fails partway, production carries the damage immediately. The standard alternative is to write to a staging location, validate the result, and only then publish, so a failure leaves the previous good data in place.
The claim that open table formats do not permit overwriting an existing table is incorrect because open table formats support overwrites. Their snapshot history helps with recovery, but it does not make the pattern advisable.
The claim that in-place transformation always requires exclusive locking, blocking all readers for the duration, is incorrect because these formats use optimistic concurrency rather than reader-blocking locks. Readers continue against the current snapshot.
The claim that the objection is only about performance, since in-place writes are slower than creating new tables, is incorrect because it reduces a data integrity concern to throughput. The problem is corrupted production data, not slower writes.
For the exam, prefer any answer describing validate-then-publish over one that writes directly to a consumed table.