A municipal library chatbot must ingest mixed HTML pages, PDF guides, and catalog records for FM consumption. Which approach best handles that heterogeneous corpus?
Select an answer to reveal the explanation.
Short Explanation
HTML, PDFs, and catalog rows are different ingredients — you don’t dump them in the same blender. Type-specific processors clean each format into the same kitchen-ready shape the FM can actually use.
Full Explanation
A municipal library corpus mixes HTML pages, PDF guides, and catalog records. Heterogeneous formats need type-specific processors that parse each source correctly, then normalize into a consistent FM-ready representation (clean text, metadata, and chunk boundaries). That pattern preserves meaning from each format—HTML structure, PDF layout text, and catalog fielded records—while presenting one ingestion contract to the chatbot and retrieval layer.
Concatenating all file bytes into one binary blob fails conceptually: the FM cannot interpret opaque mixed binaries as coherent library knowledge, and you lose format-aware parsing. Requiring staff to retype every PDF into chat before indexing fails operationally and does not scale for a public library corpus. Using only keyword search on filenames while avoiding body transformation fails because titles alone cannot ground answers in guide content or catalog fields the resident actually needs.
Exam caveat: AIP-C01 data-preparation items often reward format-aware normalization over “one blob fits all.” Homogeneous FM inputs are produced by processors, not by ignoring document types.
Operational check: maintain per-type parse success rates, sample normalized outputs for HTML/PDF/catalog fidelity, alert on empty extractions, and verify retrieval hit rates improve after normalization compared with filename-only baselines.