A permitting chatbot's RAG pipeline must index a large municipal zoning-code corpus organized into numbered sections and subsections with cross-references. Which embedding and chunking approach best fits this source material?
Select an answer to reveal the explanation.
Short Explanation
Think of chunking like cutting a reference book into chapters instead of shredding it into random-length strips: each piece should be a coherent unit someone could actually cite. A zoning code already comes organized into sections and subsections, so chunking along those boundaries - with an embedding model suited to that size - keeps each retrieved chunk meaningful. Ignoring that structure throws away organization the corpus gives you for free.
Full Explanation
Effective RAG chunking preserves the semantic units a reader would actually recognize, so a legal or regulatory corpus with existing section and subsection numbering should be chunked along those boundaries - each chunk then represents one coherent provision, which both improves retrieval precision and gives the generation step a citable, self-contained unit to reason over. Pairing that with an embedding model suited to that chunk length and to formal regulatory language keeps the similarity scores meaningful. Embedding the entire corpus as a single document collapses distinct provisions into one vector, which destroys the ability to retrieve just the relevant section and instead returns the whole code's generic embedding for every query. Skipping chunking and relying on the context window to hold the full corpus at query time doesn't scale for a large document and defeats the purpose of retrieval - it turns a targeted lookup into an expensive full-document read on every request. Chunking by a rigid fixed character count without regard to section boundaries can split a single provision across two chunks or merge unrelated provisions together, degrading both retrieval and any citation the agent produces. The scope caveat: even well-structured chunking benefits from overlap or cross-reference metadata so a chunk citing another section doesn't lose that link. A concrete check: spot-check a handful of retrieved chunks to confirm each one corresponds to a single, complete zoning section rather than a fragment.