California signs new safety rules for companion chatbots and kids
On 10 September, California Governor Gavin Newsom signed a package of 13 bills on children's safety online, at an event in Marin County. One of them, which the governor's office calls Adam's Law after Adam Raine, sets rules for companion chatbots used by children. A companion chatbot is one built to act as a friend or partner over ongoing conversations, as opposed to a support bot that answers one question.
The governor's office calls it the strongest chatbot safety law in the country. The bill in the list that matches it is SB 1119, titled Companion chatbots: children's safety, by Senator Steve Padilla with Assemblymembers Buffy Wicks and Rebecca Bauer-Kahan.
What the chatbot law requires
According to the press release, the law requires crisis protocols when a child expresses suicidal thoughts, parental controls, and a notification when a child turns off safety settings. The governor's office says it is also the first law in the US to require companies to run independent child safety audits and annual risk assessments.
The same package includes a separate bill on toys with companion chatbots, SB 867. Other bills ban addictive features such as autoplay and feeds based on a user's history for social media users under 16, extend child sexual exploitation laws to cover AI-generated images of minors, and limit targeted advertising to children and the use of K-12 student data in AI systems.
What we do not know yet
The press release does not say when each law takes effect, which companies are covered, or what the penalties are. Those details are in the bill texts, and they decide who has to change what. A day earlier, on 9 September, the governor's office had announced another AI bill, SB 813, on independent verification organizations, which we have not covered here.
What to do if you build chat products
If you make a chat app that teenagers can use, read SB 1119 itself before planning any work, since the summary above comes from the governor's office and not from the law. Start with the part about crisis protocols: decide what your app does when a young user writes about self-harm, and who on your team is responsible for that answer.
What we think
Crisis protocols and parental controls are things a careful company should already have, so the most demanding part here is the independent audit. It moves the question of whether a chatbot is safe for kids from the company's own word to an outside review, and we expect other states to copy that idea.
Sources