A transit disruption assistant needs continuous signal on whether riders find explanations helpful. Which evaluation mechanism best captures that user-centered feedback?
Select an answer to reveal the explanation.
Short Explanation
Riders are the ones stuck on the platform—ask them. A simple thumbs-up or thumbs-down on disruption explanations is a continuous report card the transit team can actually steer with.
Full Explanation
Collecting rider thumbs-up/down (or similar ratings) on disruption explanations and feeding them into continuous improvement is the evaluation mechanism that best captures continuous, user-centered signal for a transit disruption assistant. Riders experience whether an explanation helped them reroute or wait with confidence; lightweight ratings convert that experience into an ongoing quality stream the transit team can trend, segment by line or incident type, and act on. Closing the loop from rating to prompt, retrieval, or content fixes keeps the assistant aligned with rider needs. Instrumenting only GPU temperature on unrelated training clusters fails by concept: training-hardware telemetry does not measure whether disruption text helped riders. Banning all rider feedback to avoid biased ratings fails by concept—bias concerns call for careful sampling and moderation, not deleting the primary user signal. Judging quality solely by how many characters the model generates per reply fails by concept: verbosity is not helpfulness and can worsen rider cognitive load during disruptions. Exam caveat: user-centered GenAI evaluation privileges explicit feedback loops over infra proxies, feedback bans, or length heuristics. Operational check: place thumbs controls on disruption explanations, store ratings with trip and incident metadata, review weekly aggregates in transit ops, investigate sharp downs after content or model changes, and feed themes into a backlog for explanation improvements.