A customs filing document contains both a dense structured table of HS codes and tariff rates, and several paragraphs of prose explaining exemption conditions. Should these two content types be chunked and prepared the same way?
Select an answer to reveal the explanation.
Short Explanation
A tariff-rate table and a paragraph of exemption rules are different animals - the table wants to be structured, queryable data, while the prose wants to be a normal semantic chunk. Don't run them through the same cookie cutter just because they're on the same page.
Full Explanation
Treating the structured table as extracted structured data and the prose as text chunks sized for semantic retrieval is correct because these two content types serve different retrieval needs: a dense table of HS codes and rates is best queried through exact or structured lookup (a specific code mapping to a specific rate), while the exemption-condition prose is naturally suited to semantic chunking and embedding, since a user's question about exemptions is conceptual rather than a precise code lookup. Applying identical size and overlap settings to both is wrong because a chunking configuration tuned for one content type rarely serves the other well - a table split by an arbitrary character count can sever rows mid-entry and produce meaningless fragments, while prose-tuned settings may not be granular enough to keep a table's row-level precision intact. Discarding the HS-code table as irrelevant is wrong and directly contradicts the customs-compliance agent's stated purpose of looking up HS codes and validating declarations - that table is core, load-bearing content for the very application this data prepares for. Discarding the prose as irrelevant is equally wrong because exemption conditions are exactly the kind of nuanced, conceptual content that a customs-compliance agent needs to answer correctly, and it cannot be recovered from the table alone. The exam-relevant point is that a single document can require more than one preparation treatment depending on which parts are structured versus prose.