A parks-department maintenance-request dataset has a required 'reported_by_department' field, but a growing share of monthly uploads arrive with that field blank. The team wants automated rules that specifically track completeness thresholds for required fields over time and alert when a field's null rate climbs too high. Which capability fits this need?
Select an answer to reveal the explanation.
Short Explanation
It's a fuel gauge for your required fields, tracking what fraction is missing month over month so a slow leak doesn't go unnoticed. Glue Data Quality completeness rules do exactly that, watching a field's null rate against a threshold you set.
Full Explanation
Mechanism: Glue Data Quality completeness rules let a team define a required field and a threshold for its acceptable null rate, then evaluate that rule against each new upload, alerting when the proportion of blank values climbs beyond what's expected, which is precisely the kind of ongoing, field-specific monitoring the scenario calls for. Why the wrong options fail by concept: Ground Truth is built for human labeling of training examples, using it to manually backfill missing operational fields in an ongoing ingestion pipeline is not what it's designed for and doesn't scale as an automated monitoring solution. Feature Store's online store serves already-computed feature values at low latency during inference, it has no role in detecting or tracking completeness problems in newly ingested source data before those features are even computed. A join transform combines two tables based on a key, it doesn't evaluate or monitor field completeness at all, and joining with an unrelated department directory wouldn't fill in or flag the missing values. Scope caveat: a completeness rule flags that a field is blank, it doesn't diagnose why, a rising null rate could stem from a form change, a broken integration, or a genuine shift in how departments report, so the alert is a starting point for investigation, not the full answer. Operational check: review the completeness rule's trend over the past several monthly uploads to confirm whether the null rate for the required field is stable or trending upward.