Florida Woman Charged After AI Chatbot Flags Threat

A Bonita Springs, Florida, woman faces a second-degree felony charge after statements she typed into Anthropic’s Claude chatbot were flagged by the company’s safety systems, reviewed by a human contractor, and forwarded to law enforcement as a credible threat of violence.

Carli Michelle Heller was detained without incident at her home on September 26 following an investigation initiated by the Lee County Sheriff’s Office. An LCSO intelligence detective subsequently took over the case. Heller told investigators she used the artificial intelligence platform as a personal diary, but the content she entered triggered automated safety protocols that ultimately led to her arrest.

Under Florida Statute 836.10, transmitting a written or electronic record that threatens to kill or injure someone, carry out a mass shooting, or commit an act of terrorism constitutes a second-degree felony. The statute requires that the communication be made in a manner where another person may view it — a legal threshold increasingly tested by interactions with cloud-based AI services where data is processed on external servers and routinely audited by corporate safety teams.

Anthropic’s terms of service reserve the right to share user information in limited emergencies when disclosure is deemed necessary to prevent death or serious physical injury. Like other major AI developers, the San Francisco-based company employs a layered moderation approach. Automated classifiers scan inputs for policy violations, and high-severity flags are escalated to human reviewers who assess context before deciding whether to contact authorities.

This escalation pipeline operates under intense legal scrutiny across North America. AI companies face mounting pressure from both directions: lawsuits for failing to report dangerous users, and criminal cases built entirely on their internal moderation logs.

Last month, the Canadian province of British Columbia filed suit against OpenAI and CEO Sam Altman, alleging the company could have prevented a mass shooting. The shooter, 18-year-old Jesse Van Rootselaar, had previously been flagged by OpenAI’s safety team for conversations about gun violence. OpenAI did not alert police, determining the exchanges did not meet its internal threshold for legal referral.

Similarly, Florida sued OpenAI and Altman in June, alleging ChatGPT contributed to real-world harms, including the 2025 Florida State University shooting. These civil actions are pushing AI firms to lower the threshold for reporting user behavior to law enforcement, increasing the likelihood that ambiguous or emotionally charged prompts will trigger police intervention.

Users frequently misunderstand the privacy architecture of commercial chatbots. Unlike local journaling software or encrypted messaging applications, standard web-based AI assistants process text through remote infrastructure. Prompts are retained for model training, abuse monitoring, and quality assurance. Human contractors regularly access this data across the industry. Recent reports revealed that reviewers evaluating Microsoft Copilot’s image editor can see user prompts, uploaded photographs, and AI-generated edits — including sexual, disturbing, or potentially illegal material — while assessing output accuracy.

Heller remains charged with making a written threat of violence. The Lee County Sheriff’s Office has not released further details regarding the specific language used in the prompt or whether any weapons were recovered during the search of her residence.