A QA team is evaluating a Foundry-hosted assistant that answers customer questions by retrieving passages from a product manual and summarizing them. They want automated evaluation metrics that specifically catch two failure modes: the assistant inventing facts not present in the retrieved manual passages, and the assistant answering a question that has nothing to do with what was asked. (Select TWO.) Which two built-in evaluation metrics should the team use?
Select all correct answers, then click Submit.
Short Explanation
You need Groundedness to catch made-up facts that aren't in the retrieved manual passages, and Relevance to catch answers that miss the question the customer actually asked. Fluency and Coherence only judge writing quality, so they won't flag fabrication or off-topic responses.
Full Explanation
The correct metrics are Relevance and Groundedness. Groundedness measures whether a response is actually supported by the retrieved manual passages, which directly catches the case where the assistant invents facts that are not present in the source content. Relevance measures whether the response actually addresses what the customer asked, which directly catches the case where the assistant answers a question that has nothing to do with the original query. Together, these two metrics target exactly the two failure modes the QA team described. The fluency metric is incorrect because it only checks whether the response is grammatically well-formed and reads smoothly; a response can be perfectly fluent while still fabricating facts or completely missing the point of the question, so it does not catch either failure mode. The coherence metric is incorrect because it checks whether a response is logically organized and easy to follow from one sentence to the next, which says nothing about whether the content is grounded in the retrieved passages or whether it actually answers the question asked. Since the team specifically needs to catch fabrication against source material and off-topic answers, Groundedness and Relevance are the metrics built for those two purposes, while fluency and coherence evaluate writing quality rather than factual accuracy or topical fit.