A company deploys a generative AI chatbot on its public website to answer visitor questions. Before any chatbot response is shown to a visitor, the company wants an automated check that screens the generated text for hate speech, violence, and self-harm content and blocks or flags anything that crosses a defined severity threshold. Which capability satisfies this requirement?
Select an answer to reveal the explanation.
Short Explanation
You need a filter that scores the chatbot’s generated text for hate, violence, and self-harm and gives you severity levels you can use to block or flag it before a visitor sees it. Tone, topics, and language checks don’t tell you whether the text is actually harmful.
Full Explanation
The correct answer is a content safety service that scores generated text against harmful-content categories and severity levels. A content safety service is built specifically to inspect a passage of text, including AI-generated text, against categories such as hate, violence, self-harm, and sexual content, and return a severity score the company can use to block or flag output before it reaches a visitor. Key phrase extraction that summarizes the main topics covered in the chatbot's response is incorrect because it only identifies the main topics discussed in a response; it has no mechanism for judging whether the content is harmful, so a hateful message and a benign one could share the same extracted phrases. Sentiment analysis that labels the chatbot's response as positive, negative, or neutral is incorrect because it measures whether text reads as positive or negative in tone, which is a different axis than harmfulness; a coldly clinical description of self-harm could score as neutral in sentiment while still being unsafe to display. Language detection that verifies the chatbot's response matches the visitor's language is incorrect because it only confirms which language the text is written in and has nothing to do with the content's safety. Screening generated output for harmful categories before display is precisely the job content safety filtering was designed for.