Transit must ingest AVL, then enrich, then publish—and later tasks must wait on earlier ones. How should the Lakeflow Job be designed?

Lakeflow Jobs support multi-task logic with explicit dependencies so municipal pipelines run in a required order. Transit AVL processing that must ingest, then enrich, then publish should be modeled as job tasks with precedence: enrich waits on ingest, publish waits on enrich. That dependency graph—not Run on failure dependencies between tasks—guarantees later tasks see outputs of earlier ones and proceed only when upstream work succeeds, with appropriate fail or skip behavior when upstream work fails. Designing ingest AVL, then enrich, then publish tasks with separate job dependencies for those tasks fails by concept: splitting work across separate jobs removes shared-run Depends on ordering rather than enforcing that enrich waits on ingest and publish waits on enrich. Designing ingest AVL, then enrich, then publish tasks with Run on failure dependencies between tasks inverts the control: Run on failure runs downstream only when upstream fails, it does not establish success-gated Depends on ordering within a run and can publish empty or stale enrichment. Designing ingest AVL, then enrich, then publish tasks with cluster dependencies for those tasks misplaces the control: sharing compute provides execution resources, not a linear ingest→enrich→publish precedence with gating and failure propagation.

Exam caveat: dependencies express order and gating; they do not by themselves make writes idempotent—publish tasks still need merge keys or overwrite semantics appropriate to transit facts. Operational check: fail the ingest task in a staging job run and confirm enrich and publish do not proceed; succeed ingest/enrich and confirm publish runs once; inspect the job DAG UI to verify the ingest→enrich→publish edges.