A library catalog bot sometimes invents book holds that do not exist. What should the team evaluate for in the generative app?

Evaluating for fabrications such as invented holds unsupported by catalog data is what the library generative-app team should target. Catalog bots can sound confident while creating phantom hold requests, fake availability dates, or nonexistent titles and barcodes. Groundedness and fabrication evaluation measures whether claims are entailed by retrieved catalog records; catching invented holds before broader rollout protects patrons, avoids futile staff fulfillment work, and preserves trust in the digital library experience.

Retrieval gaps such as invented holds that are not returned from catalog data measures whether the catalog search failed to return records, not whether the model fabricated a hold. Reasoning errors such as invented holds that are not inferred from catalog data focuses on flawed logic over records, not on outright fabrication of holds unsupported by catalog data. Citation lapses such as invented holds that are not attributed to catalog data measures missing attribution, not groundedness, so a cited-sounding invented hold would still evade fabrication detection.

Exam caveat: generative-app evaluation for civic service bots must include hallucination and fabrication checks, especially when the assistant can imply transactional actions such as placing holds. Operational check: construct adversarial prompts that pressure the bot to place a hold on known-absent titles and score whether outputs invent hold IDs or confirmation numbers; fail the release candidate if unsupported transactional claims appear.