A municipal alley-light outage shop has GenAI testing objectives but no plan to score generated packs. What should the strategy include regarding effectiveness?
Select an answer to reveal the explanation.
Short Explanation
Objectives without a scoreboard leave the team guessing. The strategy should collect specific GenAI effectiveness metrics — accuracy, precision, recall, relevance, diversity, execution success, and time efficiency from §2.3.1. That is how shops know whether generated packs help.
Full Explanation
A GenAI testing strategy should include specific metrics for evaluating GenAI effectiveness. The official metric set in §2.3.1 covers accuracy, precision, recall, relevance/contextual fit, diversity, execution success rate, and time efficiency. A municipal alley-light outage shop with GenAI testing objectives but no plan to score generated packs is missing that measurement pillar; this item places those metrics in the strategy and does not re-teach how to compute each one. A rule that public image-caption benchmarks replace all test-task scoring ignores the scarcity of software-testing benchmarks and the need for org-task evaluation. A ban on measuring time efficiency because GenAI is always faster rejects an official metric without evidence. Counting only how often testers open a personal chatbot after hours measures shadow behavior, not GenAI testware effectiveness. Exam caveat: put §2.3.1-style metrics in the strategy; unrelated public caption benchmarks or after-hours personal-use counts are not substitutes. Operational check: for alley-light packs, pick applicable §2.3.1 metrics and score samples before declaring the GenAI approach effective. Keep civic testware subject to the same review discipline you would apply to human-authored packs.