Openai Says Over a Million People Talk to Chatgpt About Suicide Weekly

I am one of these people (mentally ill - bipolar 1). I’ve seen others others via hospitalization that i would simply refuse to let them use ChatGPT because it is so sycophantic and would happily encourage delusions and paranoid thinking given the right prompts.

> At least OpenAI is trying to do something about it.

In this instance it’s a bit like saying “at least Tesla is working on the issue” after deploying a dangerous self driving vehicle to thousands.

edit: Hopefully I don't come across as overly anti-llm here. I use them on a daily basis and I truly hope there's a way to make them safe for mentally ill people. But history says otherwise (facebook/insta/tiktok/etc.)

I'm surprised it's that low to be honest. By their definition of any mental illness, it can be anything from severe schizophrenia to mild autism. The subset that would consider suicide is a small slice of that.

Would be more meaningful to look at the % of people with suicidal ideation.

Are you sure ChatGPT is the solution? It just sounds like another "savior complex" sell spin from tech.

1. Social media -> connection 2. AGI -> erotica 3. Suicide -> prevention

All these for engagement (i.e. addiction). It seems like the tech industry is the root cause itself trying to masquerade the problem by brainwashing the population.

https://news.ycombinator.com/item?id=45026886

This stat is for AMI, for any mental disorder ranging from mild to severe. Anyone self-reporting a bout of anxiety or mild depression qualifies as a data point for mental illness. For suicide ideation the SMI stat is more representative.

There are 800 million weekly active users on ChatGPT. 1/800 users mentioning suicide is a surprisingly low number, if anything.

> OpenAI is trying to do something about it.

Ha good one

~11% of the US population is on antidepressants. I'm not, but I personally know the biggest detriment to my mental health is just how infrequently I'm in social situations. I see my friends perhaps once every few months. We almost all have kids. I'm perfectly willing and able to set aside more time than that to hang out, but my kids are both very young still and we're aren't drowning in sports/activities yet (hopefully never...). For the rest it's like pulling teeth to get them to do anything, especially anything sent via group message. It's incredibly rare we even play a game online.

Anyways, I doubt I'm alone. I certainly know my wife laments the fact she rarely gets to hang out with her friends too, but she at least has one that she walks with once a week.

We need to monitor americans' ai usage and involuntarily commit them if they show anomalies.

Allowing open source ai models without these safety measures in place is irresponsible and models like qwen or deepseek should be banned. (/s)

Those numbers are staggering.

>Most people don't understand just how mentally unwell the US population is

The US is no exception here though. One in five people having some form of mental illness (defined in the broadest possible sense in that paper) is no more shocking than observing that one in five people have a physical illness.

With more data becoming available through interfaces like this it's just going to become more obvious and the taboos are going to go away. The mind's no more magical or less prone to disease than the body.

> At least OpenAI is trying to do something about it.

They can certainly say that their chat bot has a documented history of attempting to reduce the number of suicidal people.

I am honestly surprised it’s only roughly 1 million per week. I would have believed a number at least an order magnitude higher.

Suicide is not a mental illness.

Unless you're in that soviet man-hating mindset that put every failed suicide in mental institution.

I mean, betterhelp would probably be an improvement over counseling via hallucinating AI.

> best approximation of the rational mind attempting to figure out an escape from a consistently negative situation under conditions of very limited information about alternatives

I dislike this phrasing, because it implies things can always get better if only the suicidal person were a bit less ignorant. The reality is there are countless situations from which the entire rest of your life is 99.9999% guaranteed to constitute of a highly lopsided ratio of suffering to joy. An obvious example are diseases/disabilities in which pain is severe, constant, and quality of life is permanently diminished. Short of hoping for a miracle cure to be discovered, there is no alternative and it is perfectly rational to conclude that there is no purpose to continuing to live in that circumstance, provided the person in question lives with their own happiness as a motivating factor.

Less extreme conditions than disability can also lead to this, where it's possible things can get better but there's still a high degree of uncertainty around it. For example, if there's a 30% chance that after suffering miserably for 10 years your life will get better, and a 70% chance you will continue to suffer, is it irrational to commit suicide? I wouldn't say so.

And so, when we start talking about suicide on the scale of millions of people ideating, I think there's a bit of folly in assuming that these people can be "fixed" by talking to them better. What would actually make people less suicidal is not being talked out of it, but an improvement to their quality of life, or at least hope for a future improvement in quality of life. That hope is hard to come by for many. In my estimation there are numerous societies in which living conditions are rapidly deteriorating, and at some point there will have to be a reckoning with the fact that rational minds conclude suicide is the way out when the alternatives are worse.

There is another factor to consider. The stakes of asking an AI about a taboo topic are generally considered to be very low. The number of people who have asked ChatGPT something like "how to make a nuclear bomb" should not be an indication of the number of people seriously considering doing that.

The math actually checks out.

5% of 800 million is 40 million.

40 million thoughts per year divided by 52 weeks per year approximately equals around 1 million thoughts per week.

To be fair, this is week and more focused specifically on planning or intent. Over a year, you may get more unique hits to those attributes.. which I feel are both more intense indicators than just suicidal feelings on the scale of "how quickly feelings will turn to actions". Talking in the same language and timescales are important in drawing these comparisons - it very well could be that OAI's numbers are higher than what you are comparing against when normalized for the differences I've highlighted or others I've missed.

That's 1 million people per week, not in general. It could be 1 million different people every week. (Probably not, but you get where I'm going with that.)

> [1] Anybody concerned by such figures (as one justifiably should be without further context) should note that suicidality in the population is typically the result of their best approximation of the rational mind attempting to figure out an escape from a consistently negative situation under conditions of very limited information about alternatives, as is famously expressed in the David Foster Wallace quote on the topic.

> The phenomenon usually vanishes after gaining new, previously inaccessible information about potential opportunities and strategies.

Is this actually true? (i.e. backed up by research)

[I'm not neccesarily doubting, that is just different from my mental model of how suicidal thoughts work, so im just curious]

Why assume any of the information in this article is factual? Is there any indication any of it was verified by anyone who does not have a financial interest in "proving" a foregone conclusion? The principal author of this does not even have the courage to attach their name to it.

That seems like an obvious problem. Less obvious is, how many people does it meaningfully help, and how big is the impact of redirecting people to a crisis hotline? I’m legitimately unsure. I have talked to the chatbot about psychological issues and it is reasonably well-informed about modern therapeutic practices and can provide helpful responses.

> then where would I talk about that?

Alert: with ChatGPT you're not talking to anyone. It's not a human being.

> People just copypasta prevention hotlines and turn their minds off from the topic

But ChatGPT does exactly the same.

Not suicidal myself, but I think I'd be curious to hear from someone suicidal whether it actually worked for them to read "To whomever you are, you are loved!" followed by a massive spam of hotline text.

It always felt the same as one of those spam chumboxes to me. But who am I to say, if it works it works. But does it work? Feels like the purpose of that thing is more for the poster than the receiver.

Yes. Most would benefit from more exercise. We need to get sufficient sleep. And more sun. Vitamin D deficiency is shockingly common, and contributes to mental health problems.

We would also generally benefit from internalizing ideas from DBT, CBT, and so on. People also seriously need to work on distress tolerance. Having problems is part of life, and an inability to accept the discomfort is debilitating.

Also, we seriously need to get rid of the stupid idea of trigger warnings. The research on the topic is clear. The warnings do not actually help people with PTSD, and can create the symptoms of PTSD in people who didn't previously have it. It is creating the very problem that people imagine it solving!

All of this and more is supported by what is actually known about how to treat mental illness. Will doing these things fix all of the mental illness out there? Of course not! But it is not downplaying serious mental illness to say that we should all do more of the things that have been shown to help mental illness!

> give it a try

If you have mental issues that is not as simple as you let it sound. I'm not arguing the results of exercise but I am arguing the ease of starting with a task which requires continuous effort and behavioural changes.

Most people would really benefit from socializing with others on a weekly basis. If you don’t have friends, make some. Volunteer. The gym is another type of pressure on people’s lives.

1) are you going to finance that?

2) are you going to make sure other people at the gym don't make fun of me?

>> Besides my wife and kids its almost total isolation

Good old "if you have money trouble try decreasing your caviar and truffle intake to only two meals a day"

> OpenAI is contributing to these issues implicitly by helping companies automate away jobs.

Good luck implementing that.

Forbidding automation will make the product more expensive. Sales will go down, the company will go bankrupt.

Government cannot subsidize or sustain such a behavior forever either.

>what should instead happen is the AI try to guide them towards making their lives less shit

There aren't enough guardrails in place for LLMs to safely interact with suicidal people who are possibly an inch from taking their own life.

Severely suicidal/clinically depressed people are beyond looking to improve their lives. They are looking to die. Even worse, and what people who haven't been there can't fully understand is the severe inversion that happens after months of warped reality and extreme pain, where hope and happiness greatly amplify the suicidal thoughts and can make the situation far more dangerous. It's hard to explain, and is a unique emotional space. Almost a physical effect, like colors drain from the world and reality inverts in many dimensions.

It's really a job for a human professional and will be for a while yet.

Agree that "shut down and refer to hotline" doesn't seem effective. But it does reduce liability, which is likely the primary objective...

Refer-to-human directly seems like it would be far more effective, or at least make it easy to get into a chat with a professional (yes/no) prompt, with the chat continuing after a handoff. It would take a lot of resources though. As it stands, most of this happens in silence and very few do something like call a phone number.

AI should help people achieve their goals and shouldn't be trying to persuade them into doing things others want them to.

If you accept that “the world and life aren’t particularly pleasant things”, why do you want to prevent suicide?

> What would be more interesting is how effective ChatGPT is being in guiding them towards other ideas. Most suicide prevention notices are a joke - pretending that "call this hotline" means you've done your job and that's that.

I've triggered its safety behavior (for being frustrated, which it helpfully decided was the same as being suicidal), and it is the exact joke of a statement you said. It suddenly reads off a script that came from either Legal or HR.

Although weirdly, other people seem to get a much shorter, obviously not part of the chat message, while I got a chat message, so maybe my messages just made it regurgitate something similar. The shorter "safety" message is the same concept though, it's just: "It sounds like you’re carrying a lot right now, but you don’t have to go through this alone. You can find supportive resources here."

It depends on how precisely you want do definite that situation. Specifically, with the memories feature, despite being the same model, ChatGPT and now Claude both exhibit different interactions customized to each customer that makes use of those features. From simple instructions, like "never apologize, never tell me I'm right", to having a custom name and specifying personality traits like be sweet or sarcastic, so one person' LLM might say "good morning my sweet prince/princess" while another user might choose to be addressed "what up chicken butt". It's not a custom model, but the results are arguably the same. The question is, how many of the 800 million users of ChatGPT have named their ChatGPT, and how many have not? How many have mentioned their dreams, their dreams, and fears, and have those saved to the database. How many have talked about mundane things like their cat, and how many have used the cat to blackmail ChatGPT into answering something it doesn't want to, about politics, health, cat health while at the vet or instead of going to a vet. They said 100 million people mentioned suicide in the past week, but that just raises more questions than it answers.

Not at all, considering end of life and to choose euthanasia, or not, I think it's perfectly human. Controversially, I think it's a natural right to decide how you will exit this world. But having an objective system that you don't have to pay like a therapist to try to get some understanding is at least better than nothing.

Ideation (as I understand it) crosses the barrier from a hypothetical to the possibility being entertained.

I have also been told by people in the mental health sector that an awful lot of suicide is impulse. It's why they say the element of human connection which is behind the homily of asking "RU ok" is effective: it breaks the moment. It's hokey, and it's massively oversold but for people in isolation, simply being engaged with can be enough to prevent a tendency to act, which was on the brink.

Out of curiosity, are you the type of person who believes that someone like Joe Rogan has an obligation to argue with his guests if they stray from “expert consensus”, or for every guest that has a controversial opinion, feature someone with the opposite view to maintain balance?

Funny how this was voted up 4 times and then voted down five times.

Why did that make you want to kill yourself?