A nightly digitisation-ingestion pipeline and a separate conservation-alert notebook are each scheduled independently, the alert notebook set to start thirty minutes after the ingestion pipeline's usual finish time. On nights the ingestion pipeline runs unusually long, the alert notebook still starts on its own fixed schedule and processes stale, incomplete data. What change fixes this properly?
Select an answer to reveal the explanation.
Short Explanation
Two separately scheduled items guessing at each other's timing is exactly how a fast finish or a slow night breaks things. Fold the alert notebook into the ingestion pipeline as a real activity with a success dependency, and it starts precisely when the ingestion is actually done — not thirty minutes after some average guess.
Full Explanation
Two independently scheduled items with a fixed time buffer between them are only ever as reliable as that buffer being long enough, and any night the upstream step runs longer than usual breaks the assumption entirely, which is exactly the symptom described. Bringing the alert notebook into the ingestion pipeline as a Notebook activity with a dependency on the ingestion steps' success removes the guesswork completely: the alert only ever starts once the actual data is actually ready, on that specific night, regardless of how long ingestion took. Adding more buffer time to the fixed schedule only shrinks the odds of the same failure, it doesn't eliminate them, and it also needlessly delays the alert on every ordinary night when ingestion finishes on time. Scheduling the alert notebook to run twice as a hedge is a workaround that risks the notebook running against genuinely incomplete data on the first pass and duplicating alerts or wasted compute on the second, without actually solving the underlying timing problem. Reducing how often the ingestion pipeline runs doesn't address run duration at all and would only reduce how often the museum's digitisation backlog actually clears, unrelated to the scheduling conflict described. The caveat: chaining the two together does mean the alert notebook is now coupled to the ingestion pipeline's run, so a hard ingestion failure needs its own explicit handling for the alert step rather than leaving it stranded. A concrete check: force an artificially slow ingestion run and confirm the chained alert notebook still starts only after that slower run actually finishes, not at the old fixed clock time.