A county permitting assistant should quantify how uncertain recommendations are so staff can intervene on low-confidence cases. Which Responsible AI practice supports that goal?
Select an answer to reveal the explanation.
Short Explanation
Confidence scores are the “how sure is this?” dial on the permitting desk. Track them in CloudWatch and send the shaky ones to a human—so uncertainty becomes a Responsible AI feature, not a hidden risk.
Full Explanation
Tracking confidence or uncertainty metrics in CloudWatch for permitting recommendations and routing low scores to review is the Responsible AI practice that lets a county permitting assistant quantify uncertainty so staff can intervene on weak cases. The application should estimate confidence from retrieval quality, model scores, or calibrated heuristics; emit those metrics to CloudWatch; and automatically enqueue low-confidence recommendations for human review instead of auto-finalizing them. Uncertainty becomes an operational control that protects applicants when the FM is unsure.
Suppressing all confidence signals so staff never second-guess the model hides the information needed for oversight. Treating every FM recommendation as certain regardless of retrieval quality ignores a primary failure mode of RAG—thin or conflicting corpora. Publishing raw model logits to the public permit portal without interpretation dumps unusable internal scores onto applicants and does not create a staff intervention path.
Exam caveat: Responsible AI uncertainty handling combines quantified confidence, monitoring (e.g., CloudWatch), and human routing for low scores—not public logit dumps or forced certainty.
Operational check: emit confidence metrics per permitting recommendation, alarm or ticket when below threshold, verify low-score items reach staff queues, calibrate thresholds with permitting SMEs, and keep raw logits off the public portal.