After deploying the customs-compliance retriever, an engineer discovers the query filter on hs_chapter is rejected even though the column exists in the source Delta table. The Delta Sync index was created listing only the primary key and the clause text column. What explains the failure?
Select an answer to reveal the explanation.
Short Explanation
Your Delta Sync index contains only the columns selected when it was created, not every column in the source table. Since hs_chapter was not synced, it cannot be used as an index filter; include it when creating or updating the index.
Full Explanation
A Delta Sync index syncs the source columns selected for the index; it is a separate serving structure, not a live view of every column in the Delta table. Because hs_chapter was omitted from the synced columns, the query engine cannot use it as a filter, even though it exists in the source table. Include hs_chapter among the synced columns by updating or recreating the index. The claim that filters must be applied to the source table misunderstands the query path: the retriever filters against the index, where the synced data is served, rather than querying the source table for each retrieval. The claim that a field must be concatenated into the embedding text confuses two roles: the text column provides content for embedding, while synced metadata columns can provide filterable attributes; a field need not be part of the embedding text to serve as a filter. The claim that filters support only numeric columns invents a type restriction; string metadata such as chapter identifiers can also be filterable when synced. Exam caveat: a column’s presence in the source schema alone does not guarantee that it is available to index queries. As a practical check, compare the filter fields your application needs with the columns configured for sync, including hs_chapter, before deploying the index.