A tide-mill visitor kiosk can speak a two-sentence summary or display the entire millwright transcript next to a gear photo. The team runs both and scores whether visitors find the right gear. What is the right success measure?
Select an answer to reveal the explanation.
Short Explanation
The kiosk wins when a visitor points at the right gear — not when the text is merely shorter. Measure the job at the photo, under both summary and full-transcript conditions.
Full Explanation
A tide-mill visitor kiosk that can speak a two-sentence summary or show the full millwright transcript beside a gear photo must score whether visitors locate the correct gear under each text condition. That task-success measure at the photo ties the text-arm comparison to real civic wayfinding: public guests succeed when they point at the right gear, not when staff merely prefer shorter copy. Measuring multimodal visitor success protects interpretability of summarization versus raw-transcript trials for a municipal heritage kiosk.
Always declaring the shorter summary better without a visitor task fails by substituting editorial taste for empirical task success. Counting only synthesizer words per second fails by measuring speech rate instead of whether guests find the gear. Ranking conditions by diffusion-prompt style similarity fails by importing an image-generation aesthetic metric that does not score visitor photo tasks.
Exam caveat: text-condition A/B tests are meaningful only against the multimodal visitor job, not against length or style proxies. Operational check: freeze the gear photo set, randomize summary versus full-transcript conditions, and record whether each visitor identifies the correct gear before declaring a winner.