A transformation job completes with no SQL errors, but downstream users report that a revenue column is now null for most rows. What does this indicate about the pipeline's design?
Select an answer to reveal the explanation.
Short Explanation
SQL succeeding only means the syntax parsed and the engine ran it. It says nothing about whether a join matched, a cast worked, or an upstream column got renamed. You need checks on the output—row counts, null rates, totals—or the first person to notice is a user.
Full Explanation
The correct answer is the data quality validation option: successful execution is not evidence of correct output, so the pipeline needs assertions on the result. A successful execution proves the statement was valid and completed. Mass nulls typically come from a join that no longer matches, a failed cast, or an upstream schema change—none of which raise an error. A controlled pipeline asserts on its output: row counts within expected bounds, null rates per column, and reconciliation against source totals, failing the run when they are violated.
The under-sized engine option is incorrect because engines do not silently discard rows under load. They queue, slow down, or fail explicitly.
The stale catalog metadata option is incorrect because catalog metadata describes structure. It does not repair values, and there is no refresh that would restore them.
The expected-nulls option is incorrect because normalizing the defect and pushing it to consumers means every downstream team invents its own workaround for a broken upstream table.
For the exam, the query succeeded framing is a signal that the intended answer concerns validation.