A city open-data AI uses scraped PDFs without OCR quality acceptance criteria. What should the auditor recommend?
Select an answer to reveal the explanation.
Short Explanation
Scraped PDFs can look fine until OCR turns street names into alphabet soup. Set quality gates before that messy text trains or feeds the AI. Open data is not a free pass on preprocessing QA.
Full Explanation
Requiring preprocessing quality acceptance criteria, including OCR accuracy thresholds, before scraped civic PDFs feed the open-data AI works because unstructured municipal documents are not inherently machine-ready. Scanned agendas, stamped permits, and legacy plan sets often contain low-contrast text, rotated pages, and handwritten annotations. Without documented OCR fitness gates, street names, parcel IDs, and ordinance citations can be corrupted into training or retrieval noise that looks fluent but is factually wrong. The correct control ties ingestion to measurable extraction quality, rejection handling, and reprocessing paths so only text that meets civic fitness standards enters the pipeline.
Assuming scraped municipal PDFs are always machine-readable enough for AI without checks fails because publication format does not imply extraction accuracy; many civic PDFs are image-heavy or poorly digitized. Skipping OCR quality gates merely because the open-data portal already publishes the PDFs fails conceptually: portal availability proves accessibility of the file, not fitness of extracted text for AI use. Measuring only download speed of PDF scrapes and ignoring text extraction quality fails because throughput metrics address acquisition logistics, not semantic integrity of the corpus.
Exam caveat: OCR thresholds must match use risk—retrieval for resident Q&A may tolerate different error rates than models that recommend zoning actions—so a single universal percentage can still mislead if not risk-tiered. Operational check: sample scraped PDFs against a labeled gold set, record character/word error rates by document class, block promotion when thresholds fail, and retain rejected batches with remediation tickets before any open-data AI retrain or index refresh.