When a feature engineering pipeline needs to connect to an external database, what is the Databricks-recommended way to handle credentials?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Databricks Secrets is the vault for sensitive data — store credentials once in a secret scope, retrieve them with dbutils.secrets.get() at runtime, and they never appear in notebook output or logs.
Full explanation below image
![Infographic 1024x768 showing Databricks Secrets flow: CLI command storing secret in scope (step 1). Notebook code: dbutils.secrets.get(scope, key) → [REDACTED] output shown (step 2). Connection to external DB using retrieved password (step 3). Secret scope ACL permission matrix showing which users/groups can access. Red X through hard-coded credentials in code. Green shield icon on Secrets pattern.](/media/databricks/dbml/DBML-q070-infographic.webp)
Full Explanation
Databricks Secrets provides secure credential management: Store secrets: via Databricks CLI (databricks secrets put-secret --scope ml-scope --key db-password) or the REST API. Retrieve at runtime: password = dbutils.secrets.get(scope='ml-scope', key='db-password'). Security properties: secrets are never displayed in notebook output (they show as [REDACTED]). Secrets are not stored in notebook code (no credential rotation needed when notebook is shared). Access controlled by ACLs on the secret scope. Secret scopes: backed by Databricks or Azure Key Vault. Hard-coding credentials is a security anti-pattern and fails code review. Storing in Delta tables requires separate access control and doesn't prevent display in notebooks. Environment variables via SPARK_ENV_VARS are visible in cluster configuration UI and to anyone with cluster admin rights.