A city permits office wants a grounded assistant that can use permitting PDFs, PNG diagrams, and training videos together. What should the retrieval pipeline do first?

The city permits office wants a grounded assistant that can use permitting PDFs, PNG diagrams, and training videos together. The retrieval pipeline should first ingest and index the PDFs, PNG diagrams, and training videos for grounding so later questions can retrieve across that multimodal corpus. Without ingestion and indexing, the assistant has no searchable evidence locker for civic permitting guidance.

Loading the PDFs, PNG diagrams, and training videos into prompt context for grounding skips searchable indexing and will exceed context limits instead of building a retrievable multimodal corpus. Summarizing the PDFs, PNG diagrams, and training videos into FAQs for grounding creates a lossy digest that drops the page-level detail, diagram specifics, and video segments needed for grounded retrieval. Embedding the PDFs, PNG diagrams, and training videos for fine-tuning the model bakes content into model weights instead of building a searchable grounding index with citable sources. Those options fail to prepare the full multimodal permits corpus for grounded answers.

Exam caveat: grounding requires ingest/index before query time; summarizing sources into FAQs alone is not a retrieval pipeline. Multimodal indexing may involve different processors per file type, but the stem’s first step is still ingest and index the stated artifacts.

Operational check: ingest sample permit PDFs, site-plan PNGs, and training videos; verify hybrid retrieval returns the right artifact types for test questions; enforce ACL so unpublished drafts stay out of the public assistant; and refresh the index when ordinance packets change. Confirm citations show which PDF page, diagram, or video segment supported each answer.