A permitting chatbot team wants a dashboard that surfaces a rising hallucination rate across resident conversations, flagged by automated evaluation rather than manual transcript review. Which combination of capabilities supports this?
Select an answer to reveal the explanation.
Short Explanation
Catching hallucinations by hand is like proofreading a whole newspaper one issue at a time; it works, but it never scales and it's always a step behind. Pairing CloudWatch generative AI observability with Bedrock evaluations lets the system score responses against factuality checks automatically and surface a hallucination-rate trend on a dashboard. That turns a manual chore into a metric ops can actually watch.
Full Explanation
CloudWatch generative AI observability provides the operational metrics pipeline for a Bedrock agent, while Bedrock evaluations run automated scoring against conversation samples, including factuality-oriented checks, and together they let a hallucination-rate metric flow onto a dashboard without a human reading every transcript, exactly the automated flagging the team wants. A CloudTrail dashboard of IAM roles is an access-audit view; it shows who called the agent, not whether what the agent said was accurate. A manual weekly sampling-and-tally process is the very thing the team is trying to move away from; it's slow, inconsistent between reviewers, and only sees a fraction of conversations. A cost-explorer breakdown by daily spend tracks money, not the correctness of what residents were told, so a rising hallucination rate could hide behind a flat or even declining cost trend. Scope caveat: automated hallucination scoring is itself a model-based judgment, so its flagged rate should be periodically spot-checked against human review to confirm the evaluation criteria are calibrated correctly. A concrete operational check: confirm the evaluation job's sampling rate is high enough to catch a rising trend before it's diluted across a full week of conversations.