A courts/clerk agent's retrieval step returns the top twenty case-law snippets by vector similarity, then a reranking model reorders those twenty before the top few are passed to the generation step. When does adding this reranking stage most clearly improve grounding compared to using the initial vector similarity ranking alone?
Select an answer to reveal the explanation.
Short Explanation
Think of initial retrieval like a fast first pass pulling twenty candidates roughly in the right neighborhood, and reranking like a careful second read that sorts them by who's really closest to the question. Vector similarity doesn't always put the single most relevant snippet at the top - reranking earns its keep exactly when that fine-grained ordering matters. It's a precision step layered on retrieval, not a fix for a small corpus or a mismatched embedding model.
Full Explanation
Reranking adds value when the coarse similarity scores from the initial vector search leave genuine ambiguity about which of the retrieved candidates is actually most relevant - a reranker, often a cross-encoder that jointly considers the query and each candidate, can distinguish fine-grained relevance that a single embedding-space distance can't capture, and promoting the best snippet to the top before generation directly improves what the model grounds its answer on. A corpus small enough to fit entirely in the prompt removes the need for retrieval altogether, which is a different scaling question and not a case for reranking specifically. An embedding-model mismatch between ingestion and query time is a retrieval-quality bug in its own right - reranking the resulting bad candidate set doesn't repair the underlying mismatch, it just reorders already-poor results. And whether the agent generates new summaries versus retrieving existing text describes the generation task, not a property of when reranking helps; reranking improves the input to generation, whatever the generation step then does with it. The scope caveat: reranking adds latency, since it scores each candidate individually rather than relying on precomputed vector distances, so it's worth reserving for cases where relevance ambiguity is a real problem. A concrete check: compare the top-ranked snippet before and after reranking on a sample of queries to confirm the reordering actually changes which case is surfaced first.