A RAG system for a pharmaceutical knowledge base uses cosine similarity with a threshold of 0.75 to filter retrieved chunks before generation. During QA, pharmacists report that the system returns 'I don't have information on that' for questions where relevant information clearly exists in the corpus. What is the most likely cause and appropriate fix?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — a is correct because a hard threshold of 0.75 on cosine similarity is a common cause of false negatives in RAG systems. Pharmaceutical vocabulary mismatch (e.g., a query using a brand name where the corpus uses a generic name, or different clinical nomenclature) causes semantically equivalent content to have cosine similarity below the threshold.
Full explanation below image
Full Explanation
A is correct because a hard threshold of 0.75 on cosine similarity is a common cause of false negatives in RAG systems. Pharmaceutical vocabulary mismatch (e.g., a query using a brand name where the corpus uses a generic name, or different clinical nomenclature) causes semantically equivalent content to have cosine similarity below the threshold. Lowering the threshold or, better, performing threshold selection empirically on a QA benchmark with known relevant pairs, prevents over-filtering. B is wrong because vectors normalized to unit length produce cosine similarity scores in the range [-1, 1]; floating-point errors do not produce values above 1.0 in standard implementations, and this is not the described symptom. C is wrong because context window truncation would produce incomplete or cut-off answers, not 'I don't have information on that'; the 'no information' response pattern specifically indicates no chunks passed the retrieval threshold. D is wrong because cosine similarity is the standard and well-validated metric for embedding-based retrieval in biomedical NLP; Euclidean distance in high-dimensional spaces is actually less discriminative due to the curse of dimensionality, not more.