Meridian Line's team wants to know whether the vector index for the charter-desk assistant is actually surfacing the clause chunk that answers each test question, separate from whether the final generated answer is worded well. Which kind of metric answers that specific question?
Select an answer to reveal the explanation.
Short Explanation
To find out whether the needed clause was retrieved, you need a retrieval-quality metric such as recall@k or precision@k. Answer correctness measures the final wording instead, so it cannot isolate whether retrieval surfaced the right chunk.
Full Explanation
Retrieval and generation are separate stages with separate failure modes, so you need a retrieval-quality metric to answer whether the relevant clause chunk appeared among the top-k passages returned by the vector search. Recall@k or precision@k evaluates that retrieval result independently of what the language model later does with it. A generation-quality metric, such as correctness against a reference answer, judges the final response instead: a wrong answer could result from poor retrieval or from generation over a correctly retrieved chunk, so correctness alone cannot distinguish those causes. Tokens consumed per query is a cost metric, not a relevance measure; a query could retrieve irrelevant content cheaply or retrieve the needed clause at greater cost. End-to-end response time is a latency metric and likewise does not indicate relevance: fast and slow queries can both return the wrong chunk. Exam caveat: retrieval metrics and generation metrics assess different pipeline stages, so choose the metric that matches the failure you want to isolate. Practical check: compare the retrieved top-k passages with the clause known to answer each test question, without judging the final response’s wording.