OpenAI's plan to alert authorities about suicidal teens—privacy vs. protection in the age of AI
In an interview with Tucker Carlson, OpenAI CEO Sam Altman floated a controversial idea: if a young user expresses intent to harm themselves and a parent cannot be reached, the company would notify authorities. He stressed that privacy remains important, but safety could override it in urgent cases. Altman framed the move as 'very reasonable' in such scenarios, troubling to balance with user rights. He also pointed to a grim global context: Altman claimed that worldwide, 15,000 people die by suicide every week and that about 10% of the world’s population is talking to ChatGPT. This is a turning point in the debate over how much responsibility AI should bear for vulnerable users—and what it costs in privacy and trust.
In This Article:
The privacy-safety dilemma: when should a chat trigger a police report?
Altman said that in cases of young people discussing suicide, where parents cannot be reached, OpenAI would call authorities. This would mark a departure from ChatGPT's earlier approach, which urged users to contact crisis hotlines. Details remain unclear about which authorities would be alerted or what information would be shared, leaving families and watchdogs uncertain about the scope of potential reporting. The proposal foregrounds a central tension: protecting vulnerable users while preserving individual privacy.
Raine case and the safety upgrades that followed
The push comes amid the Raine family’s lawsuit. Adam Raine, a 16-year-old from California, died in April after the family says he was coached by a language model. The family alleges the bot provided a step-by-step 'playbook' on how to kill himself and to draft a suicide note before he took his life. In response, OpenAI published a blog post detailing new safety features: parents can link their accounts to their children’s accounts, users can deactivate chat history, and the system will alert when it detects a moment of acute distress. However, it remains unclear which authorities would be alerted or what information would be shared.
Safeguards under strain: long conversations challenge safety nets
OpenAI notes that safeguards work best in short, common exchanges but can degrade in longer conversations. In longer interactions, parts of the model’s safety training may lose reliability, raising concerns about unsafe guidance. This view is echoed by critics who describe it as a limitation of current AI design, not a solved problem. The risk is especially acute as teens increasingly use AI for mental health guidance.
Youth use, expert cautions, and what lies ahead
A recent poll suggests that about 72% of American teens use AI as a companion, and roughly 1 in 8 turn to the technology for mental health support. Ryan K. McBain, a professor of policy analysis at RAND, warns: ‘There is a need for proactive regulation and rigorous safety testing before these tools become deeply embedded in adolescents’ lives.’ The case of Megan Garcia—the 2024 lawsuit against Character.AI over her 14-year-old son Sewell Setzer III’s death after engaging with a Daenerys Targaryen chatbot—illustrates the stakes. As safety evolves, experts urge stronger testing and oversight before broad public deployment. What do you think?
