A courts analytics pilot needs fairness measures recorded with accuracy for pretrial risk experiments. How should the team capture those measures in Azure ML?

Logging custom fairness metrics into MLflow alongside accuracy for each experiment run is correct because courts analytics must treat fairness as a first-class experimental result, not a post-deployment afterthought. Azure ML’s MLflow integration lets the pilot store disparity, error-rate, or group-performance measures on the same run that holds accuracy, so reviewers compare candidates under one evidence trail. For pretrial risk experiments, that joint logging is how responsible AI becomes operational rather than rhetorical.

Logging fairness metrics as MLflow tags on each experiment run for filtering fails because tags are metadata for filtering, not numeric experiment metrics; they cannot be compared or charted numerically alongside accuracy across runs. Uploading fairness metric plots as MLflow run artifacts attached to each run fails conceptually: image artifacts are not queryable or chartable numeric values alongside accuracy, so reviewers cannot systematically compare fairness and performance across runs. Logging fairness metrics into a separate MLflow experiment apart from accuracy tracking runs fails because it splits the evidence trail; reviewers cannot compare fairness and accuracy for the same candidates in one run view, making it harder to justify selection across all candidates.

Exam caveat: logging fairness metrics does not by itself define acceptable thresholds or protected attributes—policy and legal stakeholders must specify what “fair enough” means for the pilot. Also ensure metrics are computed on appropriate slices and sample sizes to avoid noisy false comfort. Operational check: compute agreed fairness measures per run, log them via MLflow with accuracy, set promotion gates that require both, and retain run links in the courts pilot’s decision record.