During an audit of a medical AI model, regulators require the company to trace a specific prediction back to the exact version of the raw dataset, showing all extraction, cleaning, and normalization steps it went through. What MLOps concept describes this tracking of data history and transformations?

Data lineage refers to the systematic tracking and documentation of data's life cycle in an MLOps and data engineering pipeline. It records where data originates, how it is transformed, where it moves across systems, and how it is ultimately consumed by machine learning models.

In production MLOps, maintaining clear data lineage is essential for several reasons: 1. Auditability and Compliance: Regulations (such as GDPR or HIPAA) often require organizations to explain decisions made by automated systems. Data lineage allows teams to trace a model's prediction back to the exact raw data points and training features that influenced it. 2. Debugging and Troubleshooting: If a model begins producing anomalous outputs, engineers can trace the input features backward through the pipeline to identify corrupt sources, broken API calls, or faulty transformation scripts. 3. Reproducibility: To retrain a model and get identical results, you must be able to recreate the exact state of the data at the time of the original training run. 4. Impact Analysis: If a source schema changes (e.g., a database column is renamed), data lineage maps show which downstream pipelines, feature stores, and models will be affected.

Data lineage goes beyond simple collection, labeling, or security; it is a map of the data's entire history and transformations over time.