A municipal open-data portal embeds its public-records document corpus and stores the vectors in Amazon OpenSearch Service to power a RAG-based public-records assistant. Which configuration choice is most important for this vector store to serve accurate retrieval for the RAG pipeline, beyond simply enabling k-NN search?
Select an answer to reveal the explanation.
Short Explanation
Think of it like two people who need to speak the exact same dialect: if your embedding model outputs cosine-similarity vectors but your index expects a different metric, retrieval breaks down. Matching dimensionality and distance metric to the embedding model, and storing enough metadata to rebuild context, is what actually makes it work.
Full Explanation
A RAG retrieval step depends on the query embedding and the stored document embeddings being comparable in the same space, so the index's vector dimensionality and distance metric must match what the embedding model actually produces; a mismatch silently degrades or breaks similarity ranking even though the index appears to function. Storing enough metadata (source text, document ID, section) alongside each vector is equally important, since the retrieved vector is only useful if the pipeline can resolve it back to readable context to feed the generation step. Relying on exact keyword matching abandons the point of embedding-based retrieval — semantic similarity search is what lets the assistant find conceptually related passages that don't share the querier's exact wording, which keyword matching alone misses. Disabling vector similarity scoring to shrink the index removes the very capability that makes this a RAG pipeline rather than a plain search engine. Prioritizing shard count for write throughput optimizes the wrong metric; retrieval accuracy in a RAG system comes from embedding and metadata correctness, not from how many shards ingest fastest, and over-sharding can even hurt query-time relevance aggregation. Scope note: re-embed and reindex if the embedding model is ever swapped, since old vectors from a different model are not comparable to new query vectors. Operational check: run a known test query through the pipeline and confirm the top retrieved passages are semantically relevant, not just present in the index.