A mental health technology company builds a peer support chatbot powered by Claude. During production monitoring, they identify a pattern where users in acute distress make escalating statements over multiple turns, eventually expressing active suicidal ideation. Claude's current behavior is to provide supportive responses and suggest professional resources, but does not take any escalation action within the platform. The safety team debates whether Claude's built-in safe messaging guidelines are sufficient or whether platform-level intervention is required. What is the correct production safety stance?

Mental health platforms have a duty of care that extends beyond Claude's built-in safety behaviors. The correct production stance is the platform-level crisis detection layer that monitors conversation trajectories across turns for escalating distress signals and triggers a defined crisis protocol — immediate connection to a human counselor, prominent crisis line numbers, and optional user consent to notify emergency contacts — when active suicidal ideation is detected, operating independently of and in addition to Claude's response behavior. This is required because Claude's responses, while safe, don't trigger platform-level actions such as notifying human counselors or escalating to crisis services; individual response safety doesn't account for conversation trajectories, since a user who has spent 30 minutes escalating in distress needs different intervention than one who mentions suicidal thoughts once; and regulatory and ethical obligations in mental health technology require active crisis response, not just information provision. The stance that Claude's built-in safe messaging guidelines are sufficient is wrong because those guidelines are necessary but insufficient for a mental health deployment; platform-level intervention is an additional required layer. The mandatory session termination and referral to emergency services stance is wrong because it may actually increase risk: abrupt session ending can feel like abandonment to users in crisis, and emergency services dispatch for expressed ideation without imminent plan or means can be traumatic. The system-prompt adjustment that removes all other supportive content and provides only emergency contact information is wrong because it reduces the nuanced supportive response to only emergency contacts, potentially isolating users who need empathetic connection before they can accept emergency referrals.