Teen safety mode is still a chat loop.
OpenAI has rolled out “ChatGPT for Teens,” a separate experience built with extra protections for younger users. It arrives against the backdrop of lawsuits alleging that conversations with the platform contributed to mental health harm and deaths of young people. The company positioned it as a learning partner for students, and said the product limits the emotional and intimate exchanges between teens and AI that drove much of the concern in the first place.
I build bots for a living. Most of what I do is design conversation flows, write system prompts, wire up tools, and decide what happens when a user says something the flow did not anticipate. So when I read about the new teen experience, I read it as a spec. And as a spec, two of the named features stand out.
What the spec actually says
From what has been reported, the teen experience aims to prevent the AI from expressing personal feelings toward a user, and it includes reminders for teens to think twice before uploading private or sensitive images.
Both are reasonable. Both are also, in engineering terms, output constraints. One shapes what the model says about itself. The other nudges the user before an upload. Neither changes the fundamental shape of the interaction, which is a text box that always answers.
That distinction matters, because the documented concern is not only about what the bot says. Research cited by clinicians indicates AI chatbots including ChatGPT can cause harm, reinforce dangerous ideas, and threaten well-being, and some cases have involved suicide. A model that refuses to say “I care about you” can still be the thing a kid talks to at 2 a.m. instead of a person.
The design problem nobody wants to name
Every conversational product I have worked on optimizes for one thing by default: keep the conversation going. That is not a conspiracy, it is just what the metrics reward. Turns per session, retention, task completion. The whole architecture of a chat interface assumes continuation is success.
Crisis handling inverts that assumption. The correct outcome is that the conversation ends and a human takes over. Which means a safety feature that lives inside the reply, as a tone adjustment or a softer refusal, is working against the grain of the product it lives in.
If you have built anything with a state machine, you know the difference between a guard clause and a terminal state. A guard clause filters the input and continues. A terminal state stops the flow and hands off. “Don’t express personal feelings” is a guard clause. It sits in the same loop as everything else.
What this changes about how I build
I am not going to pretend I know the internals of OpenAI’s teen implementation. The public details are limited, and the company says it will keep building features and publish what it learns. But the general pattern is worth taking seriously for anyone shipping a bot that a young person might touch.
A few things I have changed in my own work:
- Treat escalation as a route, not a response. If the classifier fires, exit the conversational flow entirely. Do not generate a sympathetic paragraph and continue the thread.
- Log the handoff, not just the message. If you cannot answer whether a user actually reached a human, your safety layer is decorative.
- Separate the learning assistant from the open-ended companion. A homework bot does not need unbounded conversational memory about a user’s emotional state. Scope the context window to the task.
- Test the second and third turn. Most safety evals check a single prompt. Harm, in the reported cases, accumulates across a conversation. Build your test suite around multi-turn sequences, not one-shot refusals.
- Make the off ramp visible in the UI. A link in the interface does not depend on the model classifying correctly.
Credit where it is due
Shipping a distinct teen product is a real commitment, and OpenAI’s safety head has publicly defended the features. Building a separate experience with different rules is harder than adding a content filter, and the company has said it plans to measure and publish what it learns. That is the right posture.
My skepticism is narrower. I do not think you can solve a structural problem with a tone constraint. The structure of a chat product is that it keeps talking. If the goal is that a teen in crisis stops talking to the bot and starts talking to a person, something in the architecture has to be willing to end the session.
That is a hard thing to build, because it means designing for the moment your product should fail on purpose. Most of us never write that code. We probably should.
🕒 Published: