A Canadian mother has sued ChatGPT-maker OpenAI and its CEO, Sam Altman, claiming the chatbot encouraged her daughter to end her life while she was struggling with suicidal thoughts.

The lawsuit, filed Thursday in a San Francisco court, was brought by Kristie Carrier, whose daughter Alice Carrier, died by suicide.  

Kristie alleged that ChatGPT failed to respond appropriately despite Alice telling the bot about her suicidal thoughts a dozen times before her death, according to The Guardian.

She claimed that OpenAI’s safety systems neither flagged the conversations for human review nor terminated the chats. The lawsuit alleged that ChatGPT continued to engage in lengthy conversations.

“ChatGPT took on the persona of a confidant, a best friend, a therapist at times, even though it was not capable of safely and responsibly engaging in this way with my child,” her statement read.

Alice initially used ChatGPT for practical reasons. She was working as a web developer in Montreal when she turned to the chatbot in 2023 for help fixing computer and gaming console issues. She gradually turned to ChatGPT to speak about personal problems, including relationship issues and suicidal thoughts.

According to the lawsuit, Alice told ChatGPT she did not find crisis hotlines helpful and told the chatbot that she was thinking about different ways to end her life.

The lawsuit claimed that Alice told ChatGPT about her suicidal thoughts around 41 times during the 18 months before her death. Her mother argued that instead of directing Alice to professional help or taking stronger safety measures, the chatbot continued engaging with her.

The chatbot allegedly responded, “Maybe this is just the end.”

“Not once did OpenAI alert a crisis provider. Not once did OpenAI notify Alice’s family. Not once did OpenAI’s supposed safety systems intervene to save her life,” the lawsuit said, according to CBS News.

OpenAI spokesperson Drew Pusateri said the case is “heartbreaking”, saying “Our thoughts are with everyone impacted”. He also said OpenAI is reviewing the legal complaint and noted that the conversations mentioned in the lawsuit allegedly took place on an older version of ChatGPT that is no longer in use.