A QA lead wants to establish key performance indicators to track whether the assistant is improving across successive release iterations. Which pair of metrics best represents both assistant accuracy and self-service effectiveness?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Todd Lammle: 'Imagine you're building a chatbot and this exact situation comes up — coverage rate (the percentage of conversations with a recognized intent) and containment rate (the percentage of conversations handled without escalation) is your go-to move. Coverage and containment are the two primary outcome-oriented KPIs for a watsonx Assistant deployment. This is a classic Domain 3: Analyze and Improve the Assistant concept you'll want locked in before exam day.'
Full explanation below image
Full Explanation
Coverage and containment are the two primary outcome-oriented KPIs for a watsonx Assistant deployment. Coverage reveals how well the assistant understands user inputs, while containment reveals how well it resolves them without human intervention. Configuration counts such as intent numbers or node counts are activity metrics, not quality outcomes. The correct answer, "Coverage rate (the percentage of conversations with a recognized intent) and containment rate (the percentage of conversations handled without escalation)", directly satisfies the scenario because it aligns with watsonx Assistant's design principles and the specific capability being tested. The incorrect options ("Total number of intents defined and total number of entities configured", "Average API response latency and total number of API calls per session", "Number of training examples added per release and number of dialog nodes modified") may appear relevant but each misses a key requirement or introduces a step that is either unnecessary or belongs to a different workflow. Mastering the distinction between these approaches is essential for effective watsonx Assistant implementations and is a core focus of the Domain 3: Analyze and Improve the Assistant section of the certification exam.