A buy-side research team wants to build a system that allows analysts to query the firm's proprietary research database — comprising 15 years of internal memos, model outputs, and annotated earnings transcripts — using natural language. The firm's CTO is evaluating whether to use Retrieval-Augmented Generation (RAG) or to fine-tune a foundation model on the internal corpus. The data is updated daily with new research. Which architectural choice is most appropriate, and what is the primary justification?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Imagine having to send your entire research library to a printing press every time a new memo was added, versus just adding the memo to a searchable shelf. Fine-tuning is the printing press — expensive, slow, and you start over with every update. RAG is the searchable shelf — the model stays constant, and you just keep adding documents. For a daily-updating corpus, RAG is the obvious choice. Option B is correct.
Full explanation below image
Full Explanation
The choice between RAG and fine-tuning is one of the most consequential architectural decisions in enterprise AI for financial services, and understanding the correct use case for each is essential for an AI architect role.
RAG (Retrieval-Augmented Generation) separates knowledge storage from model inference. A retrieval system (typically a vector database using dense embeddings) identifies the most relevant document chunks for a given query, and those chunks are passed as context to the foundation model alongside the user's question. The model then generates an answer grounded in the retrieved content. This architecture has three decisive advantages in the scenario described: (1) Daily data updates require only re-embedding new documents into the vector store — no model retraining; (2) Outputs are grounded in source documents that can be cited and audited, which is critical for compliance in investment research workflows; (3) The system can be updated, rolled back, or expanded without touching the foundation model.
Fine-tuning, by contrast, bakes knowledge into the model's weights through additional training on a target corpus. This is appropriate when you want to change the model's behavior or style — for example, training it to produce structured output in a specific format, to adopt firm-specific terminology, or to reason in a domain-specific way. Fine-tuning is not appropriate for knowledge injection in a dynamic corpus for three reasons: (1) it requires periodic retraining as the corpus updates, which is expensive and time-consuming; (2) fine-tuned knowledge degrades over time relative to new information (catastrophic forgetting); (3) the model cannot cite specific source documents because the knowledge is distributed across weights, not retrieved from a specific chunk — a compliance failure in regulated research environments.
Option A is wrong because while fine-tuning does embed knowledge into weights, this is a disadvantage for dynamic daily-update use cases, not an advantage. Query latency differences are marginal and not the primary consideration. Option C is factually wrong — foundation models can absolutely access external databases via RAG without retraining; the knowledge cutoff is a property of the model's parametric memory, not its ability to process new content via context. Option D contains an important error: RAG does not eliminate hallucination entirely. The model can still hallucinate by misinterpreting retrieved documents, generating claims not present in the context, or producing plausible-sounding interpolations. RAG substantially reduces hallucination risk relative to pure parametric generation, but practitioners must implement output validation and confidence scoring.