A pipeline's reformatting notebook is set to run with a dependency condition of Completed on the upstream Copy activity, rather than Succeeded. The team notices the notebook still runs and produces garbage output even on nights when the Copy activity clearly failed. What should be changed?
Select an answer to reveal the explanation.
Short Explanation
Completed just means the upstream step is finished, one way or another — success and failure both count. If the goal is 'only run this after a real success,' that's what the Succeeded condition is for, not Completed, and swapping the two fixes exactly the garbage-output symptom described.
Full Explanation
A dependency condition of Completed is satisfied whenever the upstream activity finishes, regardless of whether it finished with success or failure, which explains precisely why the reformatting notebook is running even on nights the Copy activity clearly failed — from the pipeline's perspective, the Copy activity did complete, just unsuccessfully. Switching that condition to Succeeded restricts the notebook to running only when the upstream Copy activity genuinely finished without error, which is what the team actually wants and eliminates the garbage-output symptom at its source. Leaving the condition as Completed and calling it standard practice ignores the actual behavior difference between the two conditions and would let the same failure mode continue indefinitely. Removing the dependency condition entirely would make the notebook run unconditionally as soon as the pipeline starts, an even looser and worse version of the same problem, with the notebook potentially running before the Copy activity has even started. Swapping the Copy activity for a Dataflow Gen2 changes nothing about dependency condition semantics — the same Succeeded/Failed/Completed options apply to a Dataflow Gen2 activity just as they do to a Copy activity, so this doesn't address the actual misconfiguration at all. The caveat: Succeeded is the right default for most sequential chains, but a deliberate Completed dependency is occasionally useful when a downstream step is designed to handle either outcome itself. A concrete check: force the Copy activity to fail in a test run and confirm the notebook no longer executes once the condition is set to Succeeded.