Domain 5: RAG & Knowledge Retrieval

Claude Certified Architect Professional · 48 questions

An architect is designing a RAG system for a 10,000-document legal corpus where users ask highly specific questions requiring precise clause-level retrieval. Which chunking strategy is most appropriate?
A production RAG system uses cosine similarity vector search but users report that factual queries return semantically similar but topically irrelevant chunks (e.g., a query about 'contract termination fees' retrieves chunks about 'employee termination policies'). What architectural addition most directly addresses this precision problem?
A RAG system retrieves 10 chunks and passes them to Claude. Quality testing reveals Claude sometimes synthesizes answers that contradict the retrieved context by blending in training knowledge. What prompt engineering change most directly addresses this grounding failure?
An architect is designing a RAG system for a financial news application where information freshness is critical (data must be from the last 24 hours). The vector database is updated in real-time. What architectural element is most critical for ensuring Claude never cites stale information?
A RAG system serves 10,000 users with highly personalized queries. Each user has a unique profile including preferences, history, and data access permissions. How should user-specific context be architecturally integrated into the retrieval system?
A team is building a RAG system for technical documentation and observes that 'API rate limiting' and 'request throttling' retrieve the same chunks for either query, but users asking about rate limits want policy numbers while users asking about throttling want retry strategies. How should the retrieval architecture handle this semantic overlap?
A legal RAG system requires that every claim Claude makes be attributable to a specific source document with page number and section. How should citation grounding be architecturally implemented?
A RAG system shows that with 15 retrieved chunks Claude's answer quality peaks, but with 25 chunks quality degrades — Claude gives less specific, more generic answers. What phenomenon explains this, and what is the architectural mitigation?
An e-learning platform builds a RAG system for a technical Q&A assistant covering API documentation. The documentation chunks are 1,500 tokens each. Engineers observe that when users ask narrow questions (e.g., 'What is the default timeout for the connections API?'), retrieved chunks return surrounding context that buries the specific answer in irrelevant content, causing Claude to sometimes hallucinate the value or hedge excessively. What chunking strategy most directly addresses this retrieval precision problem?
A legal research platform uses semantic search over a corpus of 500,000 case law documents. They embed documents using OpenAI's text-embedding-ada-002 model. Legal professionals report that the system fails to retrieve cases with unusual legal terminology or specific statute references (e.g., 'Dormant Commerce Clause challenges to state tax apportionment under Complete Auto Transit'). Standard semantic search retrieves topically similar cases but misses the specific statutory citation. What retrieval architecture improves retrieval of highly specific legal terminology?
A pharmaceutical company builds a RAG system over clinical trial reports. Each report is a 300-page PDF containing executive summary, methodology, results tables, adverse events section, and regulatory submissions. The team uses fixed-size 512-token chunks. Retrieval quality for questions about adverse events is poor, while retrieval for executive summary questions is good. Investigation shows adverse events sections use highly structured tabular data that loses meaning when chunked at 512 tokens (tables are split mid-row). What document-aware chunking strategy resolves this?
A technical support platform builds a RAG system over a product knowledge base. They use cosine similarity with a threshold of 0.75 to filter retrieved chunks. Support engineers report that the assistant sometimes refuses to answer valid questions, saying 'I don't have information about this topic' — even when the knowledge base contains the relevant article. Analysis shows the relevant article exists but scores 0.73 cosine similarity because the user's question uses different vocabulary than the article. What retrieval quality improvement addresses this without degrading precision?
A news organization builds a RAG system over their article archive (10 million articles from 1990 to present). Users ask both historical questions ('What happened at the 1999 WTO protests?') and current questions ('What are the latest developments in the trade war?'). They use a single embedding index with no temporal metadata. Current events questions return outdated articles because older content on the same topic has higher embedding similarity from more training data. What retrieval architecture best handles temporal relevance?
A biomedical research platform uses RAG over PubMed abstracts for a literature review assistant. The system retrieves top-10 chunks and injects them into Claude's context. Research scientists report that the assistant occasionally cites findings that are the opposite of what the retrieved paper says — for example, claiming a drug 'increases efficacy' when the abstract says it 'decreases adverse effects but does not affect efficacy.' What RAG architecture change most directly reduces citation hallucination?
A software documentation platform builds a RAG system over code repositories. Each repository contains source files, docstrings, inline comments, README files, and API reference documentation. They currently embed all content uniformly. Engineers report that when users ask 'How do I use the authentication middleware?', they retrieve source code chunks rather than the README usage examples, which are more useful for onboarding. What content-type-aware retrieval strategy improves answer quality for usage questions?
A manufacturing company builds a RAG system over equipment maintenance manuals. Documents are 200-page PDFs with dense technical illustrations, schematics, and parts diagrams. Text extraction from PDFs produces incomplete content because critical maintenance steps are embedded in illustrations (e.g., 'Turn valve A clockwise 3 times' is written inside a schematic diagram, not in the surrounding text). Text-only RAG misses these steps. What chunking and indexing strategy captures this content?
A production RAG system for enterprise legal research is experiencing accuracy issues. Users report that Claude provides confident answers citing specific case law, but lawyers verify that the cited cases don't support the stated proposition — the case law exists in the corpus but the holding is being mischaracterized. Retrieval metrics show high recall (relevant documents are being retrieved) but the grounding step is failing. The vector similarity scores for retrieved documents average 0.72. Which architectural change most directly addresses this specific failure mode?
A pharmaceutical company deploys a RAG system over 500,000 clinical trial documents. Researchers query the system for drug interaction information. The team discovers a critical production issue: when a researcher asks about Drug A + Drug B interactions, the system correctly retrieves interaction documents but Claude's response occasionally incorporates information from Drug A + Drug C interactions retrieved in the same batch (documents about Drug A are semantically similar regardless of the co-drug). Precision on the Drug A + Drug B specific queries is 61%. What retrieval architecture change most directly improves precision for compound-specific queries?
A news organization deploys a RAG system over an archive of 10 million articles spanning 20 years. The system is used by journalists to research stories. A journalist asks: 'What is the current policy on immigration detention?' The system retrieves the 5 most semantically relevant articles — which happen to be from 2019 — and Claude provides a confident answer based on outdated policy information. No disclaimer is provided. The retrieval system has no temporal metadata filtering. What architectural combination best addresses knowledge freshness in this production system?
A SaaS company operates a RAG system with a corpus of 2 million technical support documents. The chunking strategy uses 800-token fixed-size chunks with no semantic boundaries. Customer success reports that Claude's answers to troubleshooting questions often contain correct information but in the wrong order — instructions from step 3 are presented before step 1. Retrieval precision is high (correct documents retrieved) but step ordering is broken. What is the root cause and correct chunking fix?
A global enterprise deploys a RAG system with Claude across 15 countries. The knowledge base contains documents in 8 languages. Users query in their native language. Current architecture: all documents are stored with their original language embeddings; queries are embedded in the query language; similarity search runs in the query's embedding space. The system performs well for English but significantly underperforms for Japanese and Arabic queries even when relevant documents exist in those languages. What is the architectural root cause and fix?
A RAG system serves a Fortune 500 company's internal Q&A platform with strict access control: employees should only receive answers grounded in documents they are authorized to access. The document corpus has 50,000 documents across 200 access control groups. Current architecture: all documents share a single vector index; access control is enforced by post-retrieval filtering (retrieve k=50, filter to authorized, pass top-5 to Claude). Security audit finds that with k=50 retrieval and heavy access filtering, Claude occasionally synthesizes information from filtered-out documents that appeared in earlier sessions and were cached in Claude's context window. What is the correct architectural fix?
A RAG system for financial services processes real-time market data alongside static regulatory documents. The static corpus (200K documents) is indexed in a persistent vector store. Real-time data (current prices, recent filings) arrives continuously and must be queryable within 60 seconds of arrival. The current architecture indexes new documents in batch every 4 hours, causing a 4-hour knowledge gap for time-sensitive financial queries. What architectural pattern best achieves near-real-time RAG while maintaining stable retrieval quality for the static corpus?
A global investment bank's production RAG system returns inconsistent answers to the same query when run on consecutive days. The underlying document corpus has not changed, and the query and retrieval parameters are identical. The engineering team's investigation reveals: (1) top-5 retrieved documents differ between runs, (2) vector similarity scores for the same document vary by ±0.03 between runs, (3) the embedding model version was updated two weeks ago with a minor patch. No other infrastructure changes occurred. What is the root cause and the correct architectural remediation?
A RAG system retrieves technical documentation to answer developer questions. The embedding model produces 768-dimensional vectors. The corpus contains both short API reference entries (50–100 words) and long conceptual guides (2,000–5,000 words). Developers report that answers to conceptual questions are accurate but answers to specific API parameter questions are vague. What chunking strategy change is most likely to address this?
A legal research assistant retrieves case law from a vector store. Users report that retrieved cases are topically relevant but jurisdictionally mismatched — a California employment case is retrieved for a New York labor question. The vector store supports metadata filtering. What change addresses this with the least retrieval quality regression?
A RAG system for scientific literature uses a reranker to improve retrieval quality. The team is evaluating ColBERT (late interaction) versus a cross-encoder reranker. The corpus has 10 million chunks and the system must return results in under 200ms end-to-end. Which reranking approach is architecturally preferable, and why?
A knowledge base contains product specifications where individual PDFs have been chunked by page. Users complain that answers about multi-page tables are incomplete — the table header appears on page 3 and the data rows on pages 4–5. What chunking strategy directly addresses this?
An embedding model selection decision is being made for a RAG system that will serve multilingual queries (English, Spanish, Mandarin, Arabic) against a document corpus that is 90% English. Users query in their native language and expect documents in English to be retrieved. Which embedding model property is most critical for this use case?
A knowledge management system chunks documents into 512-token chunks with 50-token overlap. Users ask questions that require synthesizing information spread across 8-10 chunks. Retrieved chunks are limited to top-5 by default. Engineers increase top-k to 20 to capture more chunks but find answer quality degrades because the LLM context is now full of marginally relevant chunks. What retrieval architecture addresses this more effectively?
A RAG system for a pharmaceutical knowledge base uses cosine similarity with a threshold of 0.75 to filter retrieved chunks before generation. During QA, pharmacists report that the system returns 'I don't have information on that' for questions where relevant information clearly exists in the corpus. What is the most likely cause and appropriate fix?
A product catalog RAG system stores items with metadata: category (string), price (float), instock (boolean), and releasedate (ISO date). A user query is 'Show me affordable in-stock cameras released this year.' The retrieval pipeline currently embeds the full query and performs ANN search. Results include expensive cameras, out-of-stock cameras, and cameras from prior years. What is the most effective retrieval architecture?
A legal research assistant uses RAG to retrieve relevant case law. Attorneys report that keyword-specific queries (searching for a specific statute number, '42 U.S.C. § 1983') return irrelevant cases that are semantically similar but do not cite the exact statute. The system uses only dense vector retrieval. What change best fixes this?
A RAG system ingests documents from multiple sources: an internal wiki, email threads, and meeting transcripts. The system frequently retrieves multiple versions of the same information (e.g., an email summarizing a meeting that is also in the transcript). This duplication causes Claude to give over-weighted responses emphasizing duplicated points. What deduplication strategy should the architect implement?
A RAG system indexes a 2-million-document corpus. New documents are added daily (approximately 500/day), and existing documents are updated weekly. The current architecture performs a full re-index nightly (4-hour job). As the corpus grows, nightly re-indexing will exceed the 6-hour maintenance window. What indexing architecture handles scale without exceeding the window?
A RAG system retrieves poorly on user queries that use different terminology than the indexed documents. Users ask about 'heart attack' but relevant documents use 'myocardial infarction.' The system misses relevant results because the semantic distance between lay and clinical terminology is larger than the similarity threshold. What is the most effective query expansion approach using Claude?
A RAG system's retrieval component returns the top-5 documents for each query. The team wants to measure retrieval quality before evaluating end-to-end answer quality. Which metrics provide the most complete picture of retrieval performance?
A RAG system indexes financial reports that contain narrative text sections, data tables, and footnotes. Early testing shows that simple fixed-size chunking splits tables mid-row and separates footnotes from the text that references them, causing Claude to misinterpret financial figures. What chunking strategy preserves semantic integrity for this document type?
A RAG pipeline retrieves 50 candidate documents using fast vector search, then must select the 5 most relevant for inclusion in Claude's context. The initial vector search produces good recall but mediocre precision. What reranking approach most improves precision with acceptable latency overhead?
A RAG system retrieves 10 documents (each ~1,500 tokens) and includes them all in Claude's context along with a 500-token system prompt and a 200-token user query. Total context is approximately 16,200 tokens. Response quality degrades for questions whose answers appear in documents 7-10 (later in context). What phenomenon explains this and what is the mitigation?
A RAG system retrieves 15 candidate passages for each user query. Many passages are long (800-1,200 tokens each), resulting in 12,000-18,000 tokens of retrieved context per query. The retrieved context is often relevant but repetitive, with multiple passages covering the same topic with slight variations. How should the architect apply contextual compression to improve both efficiency and answer quality?
A user asks a RAG system: 'Which of the supplier contracts signed before 2022 contain force majeure clauses that explicitly exclude pandemics?' This requires the system to identify relevant contracts, locate force majeure language, and verify the pandemic exclusion. Standard single-hop RAG returns contract titles but misses the clause-level detail. What retrieval architecture addresses this multi-hop requirement?
Your team is selecting a vector store for a RAG system that will serve 10,000 concurrent users, store 50 million document vectors, require sub-100ms query latency at p99, and need to support metadata filtering (date range, department, classification level). The system requires no maintenance downtime. Which criteria should most heavily weight the vector store selection decision?
A RAG system has been in production for 18 months. Users are reporting that answers to questions about current company policies are frequently incorrect — the system returns outdated answers confidently. The corpus was indexed 18 months ago and has not been updated. The embedding model was not changed. What is the most likely cause, and what is the remediation?
Your RAG system achieves 78% answer accuracy on an evaluation set. Analysis of failures shows two clusters: (1) questions where the correct passage is retrieved but the answer is wrong (generation failure), and (2) questions where the correct passage is not retrieved (retrieval failure). They split roughly 40/60. The team has budget to address only one. Which intervention has higher expected impact?
An organization indexes technical documentation using sentence-transformer embeddings from 24 months ago. The documentation domain has since expanded to include new product lines with distinct terminology. Users report that searching for new product terms returns results about legacy products. What is the technical diagnosis and remediation?
A RAG system for a healthcare organization needs to answer clinical questions from medical literature. The system retrieves passages from 500,000 clinical papers. During evaluation, the team discovers that when the correct answer requires integrating information across three separate papers, the system either answers with the most statistically common finding or refuses due to insufficient context. What architectural addition directly addresses this multi-document synthesis gap?
Your RAG system is being used for customer support at a software company. Retrieved passages come from technical documentation, community forums, and internal knowledge base articles. Users report that answers about current software behavior are sometimes incorrect because forum posts from 2019 are retrieved with higher similarity than recent documentation updates. What architectural control directly addresses this temporal relevance problem?