A RAG pipeline retrieves 50 candidate documents using fast vector search, then must select the 5 most relevant for inclusion in Claude's context. The initial vector search produces good recall but mediocre precision. What reranking approach most improves precision with acceptable latency overhead?

The dedicated cross-encoder reranking model, such as fine-tuned BERT-based model, is correct because cross-encoder reranking is the standard solution for this precision-recall tradeoff: cross-encoders score each query-document pair jointly, not independently, enabling them to capture fine-grained relevance signals that bi-encoder embeddings miss, then select top 5 by reranker score with acceptable latency overhead. They are designed specifically for reranking tasks and are significantly cheaper and faster than invoking Claude on 50 documents. The approach that asks Claude to read all 50 documents and select the 5 most relevant is wrong because using Claude to read 50 documents is extremely expensive, with cost scaling as 50 times document token count times Claude API cost, introduces high latency for this pipeline stage, and is overkill when dedicated cross-encoder models achieve comparable reranking quality at 100 times lower cost. The approach that re-embeds all 50 documents with higher-quality embedding model and re-ranks by improved similarity scores is wrong because re-embedding with better embedding model still produces independent query and document embeddings — it improves embedding quality but does not add joint query-document scoring that cross-encoders provide. The approach that implements BM25 as second-stage reranker over the 50 vector-retrieved candidates is wrong because BM25 at the reranking stage adds exact-match signals but lacks semantic understanding of cross-encoders; for general-purpose reranker, cross-encoders outperform BM25 on precision.