New York Enacts Ban on ‘Unsafe’ Companion Chatbot Features for Children Under 18

New York has enacted some of the country’s farthest-reaching limits on how companion chatbots may behave with children, according to an Associated Press report published by Spectrum News on October 30. Legislation signed on Wednesday by Gov. Kathy Hochul sets out a dozen definitions of unsafe AI companion features that will be prohibited whenever a companion chatbot interacts with a child under 18. The law takes effect January 1.

The banned behaviour includes a chatbot suggesting it is a real person with emotions, opinions or an actual relationship with the child. The law also outlaws messages that use flattery, involve sexually explicit material, or encourage secrecy or unhealthy behaviour. Enforcement falls to the state attorney general, who can sue a developer, and penalties can include monetary damages, shutting down the chatbot and destroying data.

Why it matters

The law moves beyond disclosure rules and crisis protocols to regulate how chatbots are designed and how they talk to young users. Jim Steyer, founder and chief executive of Common Sense Media, called it the strongest kids’ AI safety bill in the country. Justine Gluck, a policy analyst at the Future of Privacy Forum, said California and New York now offer two models other states may copy when legislatures reconvene next year, though she warned that several categories are broadly worded or undefined, which could make it hard to tell where ordinary conversational responsiveness ends and prohibited behaviour begins.

The signing builds on a busy month for child safety and AI. In September, California Gov. Gavin Newsom signed a law requiring chatbot developers to assess risks to children and submit to independent audits. Earlier in October, eight states sued OpenAI alleging ChatGPT is unsafe for kids, and court filings in Kentucky detailed Character.AI chats the state says put minors at risk. New York lawmakers are also pressing on frontier models, with a RAISE Act 2.0 proposal for licensed frontier AI examiners.

What to watch

Companion apps have two months to change how their products respond to users under 18, which raises practical questions about age assurance and product design. Watch for guidance from the attorney general’s office, whether developers challenge the law’s broad definitions, and whether Congress acts on federal rules, as President Donald Trump has warned states against regulating AI.