A QA team is evaluating a Foundry-hosted assistant that answers customer questions by retrieving passages from a product manual and summarizing them. They want automated evaluation metrics that specifically catch two failure modes: the assistant inventing facts not present in the retrieved manual passages, and the assistant answering a question that has nothing to do with what was asked. (Select TWO.) Which two built-in evaluation metrics should the team use?
Select all correct answers, then click Submit.
Short Explanation
Picture two very different ways a summary can go wrong. In one, the assistant sounds confident and well-written but slips in a detail that simply isn't in the manual it was supposed to be quoting, it made something up. In the other, the writing is fine and every sentence connects smoothly, but the answer just doesn't touch what the customer actually asked, it wandered off topic. Checking whether a response reads well or flows logically won't catch either of those problems, because both a fabricated answer and an off-topic answer can still be grammatically perfect and beautifully organized. What you need instead is one check that compares the response against the source material to see if it's actually supported by what was retrieved, and a separate check that compares the response against the original question to see if it actually addresses it. Run those two checks together and you cover both failure modes the QA team is worried about: made-up facts and answers that miss the point.
Full Explanation
The correct answers are A and D. Groundedness measures whether a response is actually supported by the retrieved manual passages, which directly catches the case where the assistant invents facts that are not present in the source content. Relevance measures whether the response actually addresses what the customer asked, which directly catches the case where the assistant answers D question that has nothing to do with the original query. Together these two metrics target exactly the two failure modes the QA team described. Option B is incorrect because fluency only checks whether the response is grammatically well-formed and reads smoothly; a response can be perfectly fluent while still fabricating facts or completely missing the point of the question, so it does not catch either failure mode. Option C is incorrect because coherence checks whether a response is logically organized and easy to follow from one sentence to the next, which says nothing about whether the content is grounded in the retrieved passages or whether it actually answers the question asked. Since the team specifically needs to catch fabrication against source material and off-topic answers, groundedness and relevance are the metrics built for those two purposes, while fluency and coherence evaluate writing quality rather than factual accuracy or topical fit.