A RAG system for a pharmaceutical knowledge base uses cosine similarity with a threshold of 0.75 to filter retrieved chunks before generation. During QA, pharmacists report that the system returns 'I don't have information on that' for questions where relevant information clearly exists in the corpus. What is the most likely cause and appropriate fix?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — you’re getting false negatives because the 0.75 cosine-similarity cutoff is too strict. Pharmaceutical wording differences, like brand versus generic names, can push relevant chunks below that cutoff. Lower the cutoff, or tune it on held-out question-answer data.
Full Explanation
The high-threshold answer is correct because a hard threshold of 0.75 on cosine similarity is a common cause of false negatives in retrieval-augmented generation systems. Pharmaceutical vocabulary mismatch, such as a query using a brand name where the corpus uses a generic name, or different clinical nomenclature, causes semantically equivalent content to have cosine similarity below the threshold. Lowering the threshold or, better, performing threshold selection empirically on a question-answer benchmark with known relevant pairs, prevents over-filtering. The floating-point-above-one answer is wrong because vectors normalized to unit length produce cosine similarity scores in the range [-1, 1]; floating-point errors do not produce values above 1.0 in standard implementations, and this is not the described symptom. The full-context-window answer is wrong because context window truncation would produce incomplete or cut-off answers, not 'I don't have information on that'; the 'no information' response pattern specifically indicates no chunks passed the retrieval threshold. The Euclidean-distance answer is wrong because cosine similarity is the standard and well-validated metric for embedding-based retrieval in biomedical NLP; Euclidean distance in high-dimensional spaces is actually less discriminative due to the curse of dimensionality, not more.