Permits ETL must call source APIs on a schedule without depending on an interactive engineer session. Which authentication approach should they use?

Permits ETL that must call source APIs on a schedule needs a service principal (or equivalent app registration) rather than an interactive engineer login. Service principals authenticate non-interactively, work with secret/certificate or federated credentials, and align with Entra and Unity Catalog identity models for municipal automation. The robot badge runs after hours without waiting for someone to complete MFA at 2 a.m.

Embedding a senior engineer's password in job cluster environment variables couples automation to a human credential, resists rotation, and often violates policy. Requiring a human MFA click each night before batch start destroys reliability for scheduled permits pulls. Disabling authentication on permitting APIs for simpler automation exposes municipal APIs to abuse and fails security baselines.

Exam caveat: the service principal still needs least-privilege API roles and, if writing lakehouse tables, scoped Unity Catalog grants—identity choice and privilege scope are both exam themes. Store client secrets in Key Vault, not in job definitions as plaintext. Operational check: create the SP, grant it API access for permits extracts, configure the Databricks job to use that identity, run off-hours without human login, and confirm engineer password env vars are removed.