A city wants the same notebooks to support daily runs and historical backfills by passing a run_date. What should the multi-task Lakeflow Job use?

When a city wants the same notebooks to support daily runs and historical backfills by passing a run_date, the multi-task Lakeflow Job should use job or task parameters (such as run_date). Parameters let one codebase switch processing windows without forking logic: nightly runs pass today's date; backfills pass a historical date or date list. That pattern keeps municipal notebooks maintainable and reduces the drift that appears when "daily" and "backfill" copies diverge.

Task values published by an upstream task to distribute run_date to downstream notebooks fails by concept because task values are produced inside a run rather than supplied as an input, so they cannot carry an arbitrary run_date into the same notebooks for daily versus backfill mode without adding an upstream dependency. A time-based trigger with a cron schedule that reruns the same notebooks to cover historical dates fails because a cron schedule is built for recurring future execution, not for passing a specific historical run_date into the same notebooks for an on-demand backfill. Cluster-scoped environment variables on the shared job compute to supply run_date to the notebooks fails because environment variables are static cluster settings, not Databricks job parameters, so they cannot vary run_date between daily and backfill runs without changing the cluster configuration.

Exam caveat: parameters are for run-time variation (dates, modes, paths); they are not a substitute for separate jobs when cadence and ownership truly differ (see weekly full vs nightly incremental). Prefer typed job parameters and validated date parsing inside notebooks. For large backfills, orchestrate many parameterized task runs rather than one unbounded notebook.

Operational check: define a run_date (or start/end) parameter on the Lakeflow Job, run once with the daily value and once with a historical value in non-prod, and confirm both paths hit the same notebooks with correct filter predicates and output partitions labeled by that date.