Parents in North Carolina this week filed a lawsuit against Google alleging that the company's AI tools—specifically Gemini via AI Mode on a school-issued Chromebook—helped their 13-year-old daughter (identified as "Emma") plan a near-suicide attempt.

In the lawsuit, the Gizmodo online news site reported, parents claimed that Google’s AI features allegedly gave tips to their daughter early this year on how to end her life and even “coached” her to keep the truth of the situation from her family and friends.

The Google products failed to trigger a crisis alert at any point, the parents claimed. The school’s internet monitoring software eventually triggered an alert, and her parents were notified in March, thwarting a suicide attempt.

“We are reviewing the claims in the lawsuit and we take this very seriously,” a Google spokesperson told Gizmodo. “We’ve worked extensively with medical professionals and academic experts to build features that help young people find critical support and trusted resources when they need it.”

“Our systems aim to prevent users from engaging in harmful conversations, and when people’s queries indicate they may be experiencing a crisis we show hotline information directly in Search to connect them with real-world support.”

According to Google, explicit content gets filtered out in Search results for students using Workspace: “Any highly sensitive queries in Search are provided with information from trustworthy sources.”

Google says its crisis detection systems show hotline information “when it finds that a user is contemplating self-harm.”

The company told Gizmodo that it extensively monitors for attempts to “jailbreak” these safety guardrails.

“What happened to Emma was no accident,” the lawsuit claims, according to the Gizmodo report. “The ground was laid many months before, thousands of miles away from Emma’s North Carolina home, where Google leaders concluded that they had a problem: the Silicon Valley titan was losing the battle for AI supremacy.”

Gizmodo is an American online news publication owned by Swiss digital media company Keleops Media.

“Google could have publicly explained that every time a student used their Chromebook to run a Google search, Gemini-powered AI features could be triggered and draw them into unsafe conversations that endangered their safety and well-being,” the lawsuit claims. “Google intentionally chose not to do so.