Published on July 31, 2024 2:24 AM GMT
Imagine a doctor discovers that a client of dubious rational abilities has a terminal illness that will almost definitely kill her in 10 years if left untreated.
If the doctor tells her about the illness, there’s a chance that the woman decides to try some treatments that make her die sooner. (She’s into a lot of quack medicine)
However, she’ll definitely die in 10 years without being told anything, and if she’s told, there’s a higher chance that she tries some treatments that cure her.
The doctor tells her.
The woman proceeds to do a mix of treatments, some of which speed up her illness, some of which might actually cure her disease, it’s too soon to tell.
Is the doctor net negative for that woman?
No. The woman would definitely have died if she left the disease untreated.
Sure, she made the dubious choice of treatments that sped up her demise, but the only way she could get the effective treatment was if she knew the diagnosis in the first place.
Now, of course, the doctor is Eliezer and the woman of dubious rational abilities is humanity learning about the dangers of superintelligent AI.
Some people say Eliezer / the AI safety movement are net negative because us raising the alarm led to the launch of OpenAI, which sped up the AI suicide race.
But the thing is - the default outcome is death.
The choice isn’t:
- Talk about AI risk, accidentally speed up things, then we all die ORDon’t talk about AI risk and then somehow we get aligned AGI
You can’t get an aligned AGI without talking about it.
You cannot solve a problem that nobody knows exists.
The choice is:
- Talk about AI risk, accidentally speed up everything, then we may or may not all dieDon’t talk about AI risk and then we almost definitely all die
So, even if it might have sped up AI development, this is the only way to eventually align AGI, and I am grateful for all the work the AI safety movement has done on this front so far.
Discuss
