Transit operations wants ad-hoc notebook exploration of ridership without leaving idle clusters burning budget overnight. Which compute choice best fits?

Transit operations analysts exploring ridership in notebooks need compute that does not leave classic clusters idle overnight. Serverless compute for exploratory notebooks addresses that by provisioning capacity on demand for interactive notebook work and releasing it when sessions are idle, so the agency avoids the familiar municipal cost leak of forgotten always-on clusters. The pattern fits ad-hoc ridership analysis: start exploring, stop paying when work stops, without operating long-lived classic fleets.

An all-purpose classic cluster with fixed sizing for notebooks so teams avoid idle spend overnight does not avoid that spend—it still leaves a classic cluster you have to operate, with fixed capacity billing while idle between sessions and overnight. Jobs compute with job clusters for notebook runs so idle notebooks stop incurring cost overnight is wrong for exploratory notebooks because job compute runs automated, non-interactive jobs rather than interactive notebook sessions; analysts cannot attach cell-by-cell and iterate, so it does not solve daytime ridership questions as they arise. A SQL warehouse with elastic scale for notebooks so idle warehouses stop accruing cost overnight is the wrong product for this work because warehouses are optimized for SQL analytics, not for interactive Python/Spark notebook exploration; even with elastic scale to limit idle cost, it does not provide the serverless notebook experience for ad-hoc ridership digging.

Exam caveat: “serverless” appears in multiple Databricks products (serverless notebooks/jobs, serverless SQL warehouses). Match the product to the workload—here the stem is exploratory notebooks, not dashboards-only. Also remember an all-purpose classic cluster with fixed sizing for notebooks does not avoid idle spend overnight—fixed capacity still bills when idle and still leaves a classic cluster to manage, and when the exam contrasts serverless vs classic compute for exploration, serverless is the intended control for that friction.

Operational check: enable serverless notebook compute per workspace policy for transit analysts, confirm idle billing behavior in cost reports, keep production ridership ETL on job compute, and still apply auto-termination policies on any remaining classic all-purpose clusters used for exceptions.