A housing authority's RAG chatbot recomputes document embeddings on every query, and the team wants a cost-optimization change that doesn't hurt answer freshness. Which change addresses this?
Select an answer to reveal the explanation.
Short Explanation
Recomputing the same document's embedding every single time someone asks a question is like re-reading and re-summarizing the same book every time a patron asks about it. Computing embeddings once, when a document is ingested or updated, and reusing that cached result across every future query, cuts out all that repeated, unnecessary work. Freshness stays intact because embeddings only get recomputed when the document itself actually changes.
Full Explanation
Batch-computing and caching document embeddings at ingestion or update time means each document's embedding is generated once and reused across every subsequent query, eliminating the wasteful, repeated recomputation that was driving up cost, while freshness is preserved because a document's embedding needs regenerating only when that document's content actually changes. Switching to a larger foundation model addresses generation quality and doesn't touch the embedding-recomputation cost at all, and larger models typically cost more per call, compounding the problem. Retrieving more documents per query increases the volume of context sent to the model, raising token cost on the generation side without addressing the root inefficiency of recomputing embeddings for content that hasn't changed. Turning off vector search and leaning on the model's training knowledge abandons the RAG architecture's purpose, grounding answers in current housing-authority documents, trading a cost problem for an accuracy and freshness problem. Scope caveat: cached embeddings need a reliable invalidation trigger tied to document updates, or stale embeddings for changed documents will quietly degrade retrieval quality over time. A concrete operational check: confirm the ingestion pipeline recomputes and re-caches an embedding whenever its source document is edited, not just when it's first added.