A water utility must embed long lab narrative PDFs without exceeding model token limits. What design step is required?
Select an answer to reveal the explanation.
Short Explanation
Long lab PDFs are a loaf of bread—slice them into chunks that fit the model’s mouth instead of forcing the whole loaf down in one bite.
Full Explanation
Long lab-narrative PDFs at a water utility routinely exceed embedding model token limits, so designers must choose chunk sizes that keep each embedding call within those limits. DP-800 treats chunking as a required design step for long municipal documents rather than an optional optimization. Correct chunk sizing preserves embeddability of lab narratives without forcing an impossible single-call ingestion of multi-hundred-page PDFs. Pasting each entire multi-hundred-page PDF into one embedding call with no splitting fails when the model rejects or truncates oversized input, silently losing the bulk of the lab narrative. Dropping all lab narratives from the database before any embedding work removes the evidence the utility needs for semantic search instead of solving the token constraint. Converting every PDF to a single emoji and embedding that instead destroys content fidelity and produces meaningless vectors for regulatory or operational retrieval. Exam caveat: chunk size must account for model-specific token limits and for any prompt wrapper tokens the embedding pipeline adds; a chunk that fits in isolation can still overflow once system text is prepended. Operational check: measure token counts for representative lab PDFs after the chosen chunker runs, confirm every chunk embeds successfully within the registered model’s limit, and spot-check that consecutive chunks still cover the full narrative without dropping tables or conclusion sections.