A housing authority needs a nightly eligibility SCD on Azure Databricks. What is the appropriate first job setup step?

A housing authority nightly eligibility SCD is a recurring, production-grade batch workload. The appropriate first setup step is to create and configure a Lakeflow Job with suitable job compute and notebook (or pipeline) task settings so the SCD runs on a schedule with defined compute, libraries, parameters, and failure handling. That job definition becomes the durable contract for who runs the eligibility history load, when it runs, and on what capacity—critical when housing staff depend on current eligibility status for waitlists, inspections, and compliance extracts.

Creating a Lakeflow Job with all-purpose compute and notebook task settings fails on compute because a nightly eligibility SCD needs isolated job compute for scheduled production runs; all-purpose compute is built for interactive exploration rather than scheduled jobs, costing more and lacking job-run isolation and auto-termination. Configuring a Lakeflow Job with suitable job compute and streaming task settings fails on task type because a nightly eligibility SCD is a discrete batch workload, not a continuous stream; streaming does not produce the discrete nightly run, retry, and history that audits require for eligibility history. Creating a Lakeflow Job with suitable job compute and SQL warehouse task settings fails on task type because a SQL warehouse is meant for serving queries and dashboards, not for executing batch SCD logic; it lacks the notebook task entrypoint with libraries, parameters, and failure handling the eligibility history load requires.

Exam caveat: "create the job" is the setup foundation, not the entire SCD design. You still need correct SCD Type 1/2 logic, keys, and testing. The distractors test the wrong compute-task pairing, not the merge syntax. Prefer job compute over all-purpose compute for cost and isolation once the job exists.

Operational check: confirm a Lakeflow Job exists with a nightly schedule, a notebook or pipeline task pointing at the eligibility SCD entrypoint, job compute selected under city policy, and at least one successful scheduled run in job history before declaring the SCD production-ready.