Between 988 and 911: A New Policy Framework for Violent Planning on AI Chatbots

Between-988-and-911_Final Between 988 and 911: A New Policy Framework for Violent Planning on AI Chatbots Anna Vinals Musquera and Scott Babwah Brennen AI systems may encounter private signs of violent planning before anyone has made a public threat, before law enforcement is aware of the risk, and even before the conduct is clearly illegal. The report asks what should happen when chatbots surface private signs of violent planning before there is a public threat or clearly illegal conduct. It proposes a framework separating provider intervention from government reporting: human-reviewed referrals for credible, specific, and imminent threats of serious violence; provider discretion for concerning but not clearly illegal content; and 988 as a possible support pathway for crisis-linked cases. At its center is a requirement that covered AI chatbot providers maintain, disclose, and follow a violence escalation policy, with privacy protections, no general monitoring duty, and accountability tied to good-faith implementation. Download Report (PDF)

60 Bills in 60 Days: State AI Companion Proposals

60 Bills in 60 Days: State AI Companion Proposals Scott Babwah Brennen, Anna Vinals Musquera, and Afnan Abbassi.   In 2025, two states, California and New York, enacted laws regulating companion chatbots. In the first two months of 2026, nearly 30 states have introduced almost 60 bills proposing to regulate companion chatbots. It is clear that regulating companion chatbots has quickly become one of the highest priorities for state lawmakers. But what exactly are they proposing? What trends are emerging across the dozens of introduced bills?  In the table below, we map out some of the most important provisions across the dozens of introduced state bills this year. The table offers a snapshot of where things stand two months into the 2026 state sessions.  At a high level, AI companion proposals include at least one of the following three provisions:  Ban AI companions. While several bills would ban AI companions for all users, far more would prohibit minors from accessing companions. Set safeguards on the use of chatbots. Some safeguards apply to minors; others to everyone. Table 1 details many of these key provisions. Impose restrictions or requirements to protect consumer privacy.    Defining Companions While each of the bills included in the table address companions, there is surprising variation in how legislators define the term. We observed four major categories of definitions. Importantly, these are not mutually exclusive, and elements of these four dimensions appear across definitions.  Social needs: First, following California’s law enacted last year, at least 8 bills this year define companions as AI systems intended to “meet a user’s social needs” in particular by doing three things:  Providing “Adaptive, human-like responses to user inputs” Exhibit anthropomorphic features. Sustaining a relationship across multiple interactions–that is having a memory.    Social Relationships: Second, a handful of bills define companions as systems meant to simulate, build, or sustain social relationships. Some of these follow the definition used in New York’s law enacted in 2025, where a companion is “designed to simulate a sustained human or human-like relationship with a user” by:  Remembering interactions/information across sessions Asking unprompted “emotion-based questions.” Sustain “ongoing dialogue” about “matters personal to the user”    Other proposals, not based on New York’s law, define “social AI companion” as AI models “specifically designed, marketed, or optimized to form ongoing social or emotional bonds with users.”  Social Conversation or interaction. Rather than focus on social needs or social relationships, a handful of bills define companions as models that facilitate or “simulate human conversation and interaction.” Here, the focus is less on sustaining relationships, and more on immediate emotionally resonant dialogue or communication.  Social Mimicry. Finally, several bills include defining companions through their capacity to “mimic” humans. Some proposals include this directly in definitions of companions, such as an Oregon bill that defines companions as models that “mimic written or spoken natural language or social interaction that is adaptive, anthropomorphic and capable of meeting some users’ social needs.”  Other bills prohibit features through which a companion mimics humans, either in appearance, voice, or mannerisms, personality, language, emotions or desires.   Provisions  The Table below maps out many of the central provisions across companion bills. But not all provisions are equal: several are likely to be far more consequential if implemented.  Enforcement How proposed regulations are enforced has a massive impact both on compliance rates and compliance costs. Over the last several years, most state privacy, child online safety, and AI laws leave enforcement to the state Attorney General. Almost half of the companion bills in 2026 include a private right of action, permitting consumers to sue companies for violations. For example, last year’s California’s SB 243 creates a private right of action that allows injured individuals to seek damages of at least $1,000 along with injunctive relief and attorneys’ fees. PRAs compound the regulatory risk for AI companies: not only can they result in very high penalties, but can also require significant effort to respond to a large number of legal actions. Furthermore, several proposed bills establish criminal penalties for certain violations, for example, Tennessee’s SB 1493 would criminalize training AI companions that encourage suicide or form emotional bonds, and authorizes damages up to $150 000- something that we have not yet seen widely in state tech regulation.  Age Verification Many of the proposals we analyzed establish requirements only for minors. But there are important differences when, or how laws require states to verify the ages of (all) users. About a quarter (26%) of laws explicitly state that companion deployers must verify the ages of all users. While the Supreme Court recently blessed age verification for adult content websites, they have also expressed skepticism over age verification for social media, following precedent that goes back more than 20 years.  Other proposals establish a certain knowledge standard of when deployers have to enact safeguards for minors. Some use the “actual knowledge” standard that has been in place for COPPA compliance for decades, and that likely does not mean deployers have to proactively verify the ages of all users. Others use other knowledge standards, including “reasonable certainty” or “reason to believe” or “reasonably should have known.” At this point, it is unclear if these knowledge standards require proactive age verification.  Audits and Reporting Many proposals would require deployers to audit their systems for risks before deployment, complete regular assessments after deployment, or report certain incidents to the government. Pre-deployment risk assessments – especially when combined with requirements to mitigate identified risks – have been at the center of debate in other areas of tech. Proponents suggest these measures are important to mitigating risks; opponents stress the high level of uncertainty and compliance burden to these audits.  Data Privacy Finally, about a quarter of bills include data privacy protections. Most commonly, they include broad data minimization standards. These limit the ways that deployers can use the data they collect for other purposes. Data minimization has become a major focus in both consumer privacy and child online safety legislation more broadly. At the