A city wants production hydrant inspection loads to carry durable quality rules rather than one-off notebook asserts. What should they prefer?
Select an answer to reveal the explanation.
Short Explanation
Put the quality gauges on the production pipe (declarative expectations), not on Unity Catalog row filters you apply in production. They'll hide bad rows from some readers, but they won't validate hydrant records and pipeline monitoring won't see the quality results.
Full Explanation
Production civic pipelines should embed durable quality via declarative pipeline expectations rather than relying on ad-hoc notebook asserts. City hydrant inspection loads that matter operationally need expectations versioned with the pipeline graph, visible in quality UI, and enforced on every production run. Applying Unity Catalog row filters in production only restricts what different consumers see at query time, and is invisible to pipeline quality monitoring. Enforcing hydrant inspection quality with Unity Catalog row filters in production fails by concept: it is serving-time access control, not durable pipeline policy, and it hides bad rows from some readers rather than validating, quarantining, or failing bad hydrant records on every production run. Managing hydrant inspection defects with Auto Loader rescued columns in bronze ignores that rescued columns capture bad data rather than enforce durable quality rules, leaving device, transcription, and integration defects unchecked. Auditing hydrant inspection freshness with Unity Catalog lineage graphs in gold is post-hoc theater—bad rows may already have driven work orders or public maps, and lineage does not enforce quality rules during ingestion. Exam caveat: exploratory notebooks can still use asserts while prototyping rules; those rules should be promoted into declarative expectations before calling the path production. Operational check: add expectations for required hydrant IDs, inspection dates, and valid result enums in the production pipeline; run a bad batch and confirm quarantine/fail behavior; confirm quality metrics appear without depending on Unity Catalog row filters in production or Unity Catalog lineage graphs in gold.
Prefer to study offline? Printable PDF study book + test engine ($7.50, unofficial). These questions stay free here.