A tool-library chat is judged by how often a browser adds a paid weekend reservation. The team is not judging it by BLEU. Which value metric is this?
Select an answer to reveal the explanation.
Short Explanation
Think of how often a chat turns into a paid weekend reservation. That is conversion. BLEU, RMSE, and a token invoice without the booking are not the action.
Full Explanation
Conversion rate is the right value metric when a tool-library chat is judged by how often a browser adds a paid weekend reservation. It counts the desired next action after engagement with the helper, which is how Domain 2 frames business value for customer-facing GenAI: did the visitor do the thing the organization cares about? That booking event is auditable in the reservation system, unlike a lab language score.
BLEU does not apply because a reservation is not a translation exam; n-gram overlap with sample replies does not prove someone paid for a weekend slot. RMSE on chat text misuses a regression error idea on free-form dialogue that is not a numeric forecast. A token-price invoice with no action attached only shows consumption cost; without a conversion numerator, spend cannot justify the helper as a booking tool.
Exam caveat: conversion still needs a defined funnel and attribution window so walk-up reservations that never used chat are not credited to the bot. Operational check: log chat sessions that end in a reservation within a fixed window, compare to a non-chat baseline, and review a sample of converted threads for misleading availability or fee promises before scaling prompts.