The permits office schedules weekend PDF-extract backfills that vary sharply in volume. How should they configure compute scaling?

Weekend PDF-extract backfills for a permits office that vary sharply in volume should use autoscaling so the cluster grows during the extract burst and shrinks afterward. Autoscaling lets Azure Databricks add workers when OCR/extract stages fan out across large permit document sets and remove workers when the backlog clears, aligning municipal spend with actual weekend demand instead of static oversizing. That elasticity is the designed answer for highly variable civic batch jobs.

Enabling autoscaling so the cluster grows after the extract burst and shrinks before it inverts the timing: it removes workers right when OCR/extract stages fan out across large permit document sets and adds workers after the backlog clears, so the backfill queues during the burst and wastes spend when idle.

Enabling autoscaling so the cluster scales to peak before the weekend and shrinks on Monday cannot track sharply varying volume: holding peak capacity all weekend cannot expand if the PDF pile spikes beyond the early guess and cannot release workers if the pile is smaller, so jobs queue when under-sized and money is wasted when over-sized until Monday.

Enabling autoscaling so the cluster tracks memory during the extract burst and trims afterward misses the scaling signal: memory stays relatively flat while OCR/extract task queues spike, so memory-based scaling cannot add workers fast enough for sharp volume spikes and the backfill sits under-provisioned while waiting for tasks to clear.

Exam caveat: autoscaling needs sensible min and max bounds; “autoscaling” without a max can still create cost incidents. The exam often pairs autoscaling with variable load stems—do not pick fixed max-all-day just because the weekend peak is large. Also distinguish cluster autoscaling from SQL warehouse scaling; both exist, but this stem is about job/batch extract compute.

Operational check: enable autoscaling on the permits backfill job cluster with a documented min/max, review worker-count charts from a busy weekend run, confirm shrink after completion, and alert finance if max is hit continuously (signal to raise max or optimize the extract, not to disable scaling).