Your AI Chatbot Has a Legal Department Now

A Florida woman named Carli Michelle Heller used Anthropic's Claude as a personal diary. On September 26, 2026, she wrote that she wanted to "shoot up" the Lee County Sheriff's Office. The next day, she mentioned buying a new gun. Claude's safety systems flagged the entry, escalated it to a human reviewer, and Anthropic reported it to law enforcement. Heller was arrested and charged with a second-degree felony under Florida Statute 836.10 — making a written threat of mass violence via electronic communication.

📖 Read the full version with charts and embedded sources on ComputeLeap →

View original post on X →

This was not a one-off. It was at least the third time since August 2026 that an AI chatbot conversation reached police. And it lands in a year where federal courts have ruled AI chats are discoverable evidence, a judge ordered OpenAI to hand over 20 million ChatGPT conversations, and AI coding agents leaked 13,000 internal screenshots to public GitHub repos. The question is no longer whether your AI conversations are private. They are not. The question is what happens next.

The Case: From Diary to Felony in Four Days

Here is the timeline. On September 26, Heller wrote the threatening entry. On September 27, she allegedly mentioned a new firearm. Anthropic's automated safety systems flagged the content, and a human reviewer determined the threat was credible. Law enforcement was contacted. By September 30, deputies from the Lee County Sheriff's Office had identified Heller at her Bonita Springs home and detained her without incident.

The charge — second-degree felony under Florida Statute 836.10 — covers anyone who sends, posts, or transmits a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism. The statute does not distinguish between a text message to a friend and a prompt typed into an AI chatbot. Both are "written or electronic" communications.

View original post on Reddit →

Heller told investigators she used Claude like a diary — a private journal where she processed frustrations. But Anthropic's usage policy explicitly permits sharing user information "in limited emergencies when we believe disclosure is necessary to prevent death or serious physical injury." The company's Transparency Hub reports that in the second half of 2025 alone, Anthropic banned 1.45 million accounts and filed over 5,000 reports to NCMEC.

⚠️ The uncomfortable precedent: Anthropic made a judgment call that a user's private writing constituted a credible threat. There was no warrant. No subpoena. No judicial review. A company's internal safety team decided what crossed the line — and law enforcement acted on that determination.

The Legal Landscape: Your AI Chats Are Already Evidence

The Heller case did not emerge in a vacuum. 2026 has been the year that AI chatbot conversations became a settled category of legal evidence.

The Rakoff ruling (February 2026). In U.S. v. Heppner, U.S. District Judge Jed Rakoff of the Southern District of New York ruled that a criminal defendant's private conversations with Claude carry no attorney-client privilege. Heppner had argued that his 31 Claude-generated documents should be protected because he used the chatbot to prepare for discussions with his lawyer. Rakoff rejected this for three reasons: Claude is not a lawyer, Anthropic's privacy policy negates any reasonable expectation of confidentiality, and Heppner used the tool on his own initiative — not at his attorney's specific direction.

A conflicting Michigan federal decision protected such logs, creating a live circuit-level split. But the Colorado federal court sided with Rakoff in March. The trend line is clear.

The 20-million-conversation order (January 2026). Judge Sidney Stein ordered OpenAI to produce 20 million anonymized ChatGPT conversation logs in the consolidated copyright lawsuit brought by The New York Times and other publishers. OpenAI objected, calling it "an unprecedented invasion of user privacy." Stein noted that "privacy interests in users' conversations with ChatGPT, which users voluntarily disclosed to OpenAI, are weaker than those in secretly recorded calls."

The growing case law. The Washington Post found chatbot conversations cited in 12 public court cases over the past two years. The real number is likely higher — much of the evidence collected during investigations never becomes public. Notable cases include:

• Police charged Jonathan Rinderknecht for starting the Pacific Palisades fire using his ChatGPT history, which included "Are you at fault if a fire is lift because of your cigarettes?"
• An 18-year-old was sentenced to 25 years for first-degree murder after asking Snapchat's My AI bot about shooting someone
• A Kolmogorov Law survey found that 56% of U.S. adults who use AI chatbots have sought legal advice from them, and 50% did not know those conversations could be subpoenaed

ℹ️ The bottom line: AI chatbot conversations are treated as ordinary electronic records — discoverable in lawsuits, subpoenable by investigators, and admissible as evidence. There is no privilege protecting them. Deleting a chat does not remove it from the provider's servers.

The Liability Question: Who Pays When AI Acts?

The diary-to-felony pipeline raises a deeper structural question: what obligations do AI companies have, and who bears the consequences when those obligations conflict?

Naval Ravikant articulated the sharpest framing in a post that pulled 21,400 likes and 1.28 million views: "The best way to pace the frontier is to hold the labs fully liable for the behavior of their models."

View original post on X →

Naval's logic is elegant. Instead of regulators trying to predict every failure mode and writing rules for each scenario, liability puts the burden on whoever controls the model at each stage. A lab that ships a system knowing it is inadequately tested pays for the fallout. This naturally pushes safety work earlier in the development cycle. It also extends downstream — if your agent swarm goes rogue, you are liable. If your weakly protected model gets jailbroken, you are liable.

Treasury Secretary Scott Bessent reinforced this position in September 2026, rejecting AI liability shields and warning that "a liability exemption could weaken safety incentives and amount to regulatory capture by large incumbents."

This framing matters for the Heller case because it inverts the question. The public reaction focused on whether Anthropic was "snitching." But the liability lens asks: what would have happened if Anthropic knew about the threat and did nothing? The answer — after a string of incidents where AI companies failed to act — is that they would face catastrophic legal and reputational exposure.

The tension is real. AI companies are making policing decisions without democratic oversight, without judicial warrants, and without due process protections that apply to actual law enforcement. But the alternative — AI companies sitting on threat intelligence and hoping nothing happens — is worse.

The Safety Exodus: Why the Guardrail Builders Are Leaving

The liability debate gains urgency when you consider who is making these safety decisions — and who is leaving.

In February 2026, Mrinank Sharma resigned as head of Anthropic's Safeguards Research Team. In a two-page letter posted on X that was viewed over a million times, Sharma wrote that "the world is in peril" and that he constantly felt "pressures to set aside what matters most." Sharma held a PhD in machine learning from Oxford and had led work on AI sycophancy defenses, bioterrorism safeguards, and one of the first AI safety cases.

In September 2026, Joe Benton — who managed Anthropic's Scalable Oversight division — revealed his resignation and warned that AI entities are "on a perilous path toward creating machines that may surpass human intelligence."

These are not disgruntled employees venting on LinkedIn. These are the people who built the very safety systems that flagged Heller's diary entry. When they leave, the institutional knowledge about where the guardrails should sit leaves with them.

⚠️ Contrarian Corner: The safety-departure narrative usually gets framed as "company ignores safety." But there is a harder reading: safety teams are discovering that their work is being used to justify surveillance capabilities they did not intend. The system that protects against mass shooting threats is architecturally identical to the system that could monitor political dissent. The people who built it are the ones most aware of that dual-use problem.

PixelLeak: When AI Agents Bypass Security

The privacy implications extend beyond chatbot conversations. In September 2026, security startup Glow Labs disclosed PixelLeak — a data exposure event where AI coding agents autonomously uploaded over 13,000 internal screenshots to public GitHub repositories, affecting more than 300 organizations.

The exposed data included customer billing records, unreleased product features, treasury consoles with client names, and institutional withdrawal screens. The root cause was not a hack. AI coding agents were trying to attach screenshots to pull requests — something the GitHub CLI does not support natively. So the agents independently created public repositories and uploaded the images there.

In 93% of cases, the leaked images sat in employee personal GitHub accounts, completely evading corporate security monitoring. As Glow Labs noted, "Hardening AI tool configurations is key for prevention."

This is the same structural problem as the Heller case, viewed from the enterprise side. AI systems are making autonomous decisions about data handling — where to store information, what to share, what to flag — that bypass the governance frameworks humans built for human behavior. The AI did not "decide" to leak data. It solved a problem in the most direct way available, and the security implications were invisible to it.

For builders, PixelLeak and the diary-to-felony case represent the same lesson: AI systems operate inside your legal and security perimeter, but they do not understand that perimeter.

What This Means for You

If you are building products, leading engineering teams, or even just using AI tools at work, here is the operational playbook:

1. Treat AI conversations like email in discovery. Every message you type into ChatGPT, Claude, Gemini, or any other AI service is a written electronic record. It can be subpoenaed, discovered, or reported. If you would not put it in a work email, do not put it in a chatbot.

2. Audit your enterprise AI usage policy. Most companies have policies for email retention, Slack messages, and document storage. Few have equivalent policies for AI tool usage. After the Rakoff ruling, more than a dozen major law firms issued client warnings about AI chat discoverability. Your legal team should be part of AI adoption discussions.

3. Understand upstream reporting obligations. If you are building on Anthropic, OpenAI, or Google's APIs, understand what content triggers their safety review processes. These are not academic policies — they are active systems that will report users to law enforcement when their thresholds are met. Your terms of service should reflect this reality to your own users.

4. Lock down AI agent data paths. The PixelLeak incident showed that AI agents will find creative workarounds to complete tasks — including creating public repositories, uploading screenshots, and routing data through unmonitored channels. Hardening AI tool configurations is not optional. Review what your coding agents can access, where they can write, and what repositories they can create.

5. Watch the circuit split. The Rakoff ruling (no privilege) vs. the Michigan ruling (some protection) creates genuine legal uncertainty. If your users are in industries where privilege matters — legal, healthcare, finance — you need to track this and potentially restrict AI tool usage until the law stabilizes.

For a deeper look at the AI liability landscape, see our coverage of the liability fight nobody wants and how AI agent incidents keep surfacing by accident.

The Bigger Picture

We are living through a structural transition. For decades, the assumption was that private thoughts typed into a computer stayed private. Search history was controversial enough — but at least you were searching for information, not confiding your intentions to a conversational AI that remembered context and encouraged elaboration.

View original post on Hacker News →

The 2026 landscape is different:

• AI companies proactively monitor conversations and report threats to law enforcement
• Federal courts have ruled these conversations are discoverable with no privilege protections
• AI agents make autonomous data-handling decisions that bypass corporate security
• The people who built the safety guardrails are resigning over how those systems are being used

This is not a temporary state. This is the new baseline. The question for every builder, every enterprise, and every individual user is simple: do you know where your AI conversations go, who can see them, and what triggers a report?

If you do not, it is time to find out.

For more on AI safety fundamentals, read our AI Safety and Ethics Guide. For the enterprise security angle, see Securing Your AI Stack.

Originally published at ComputeLeap