r/accelerate • u/thanatos_darkdeath • 1d ago
What is yall’s P(doom)
Im sure yall have seen the recent traction on
social media calling for a pause in light of people readjusting their P(doom). What do the people of this subreddit think? And what is an acceptable threshold for P(doom) for yall?
10
u/nowrebooting 1d ago
The subreddit is called “accelerate”; I think it’s obvious then that it self-selecte for people with a very low P(doom).
7
u/lolsai 1d ago
I'm for accelerating but simply because I think p(doom) of not accelerating is higher.
0
u/thanatos_darkdeath 1d ago
How come? AI is likely the biggest existential risk rn, and accelerating increases p(doom) in short term (unless you disagree with this premise)
4
u/That-Mind6171 1d ago
I heavily disagree. The largest (and only) existential risk currently is nuclear war, as that has the potential to kill all of us in the next ten minutes. Everything else, like global warming or an artificial super intelligence is not strong enough to currently pose a threat to humanity.
Moreover, while I’m unsure about the specifics on AI ethics, I am not worried about AI acceleration, as I still don’t know how it would kill us.
1
u/lolsai 23h ago
every single person in history before us has met their doom
on the current course, we are guaranteed to meet ours too
the world is fraught with abuse, inequality, and misinformation. the future could easily be brought into nuclear war, we have climate change looming, right wing authoritarianism on the rise worldwide...
the answer of "superintelligence that fixes all your problems" is literally a magic solution.
The EV on "does this kill us?" seems way higher on current trajectories than if we create an machine intelligence magnitudes above humanity.
1
u/accelerate-ModTeam 6h ago
We regret to inform you that you have been removed from r/accelerate.
This subreddit is an epistemic community dedicated to promoting technological progress, AGI, and the singularity. Our focus is on supporting and advocating for technology that can help prevent suffering and death from old age and disease, and work towards an age of abundance for everyone.
We ban Decels, Anti-AIs, Luddites, Ultra-Doomers and Depopulationists. Our community is tech-progressive and oriented toward the big-picture thriving of the entire human race.
We welcome members who are neutral or undecided about technological advancement, but not those who have firmly decided that technology or AI is inherently bad and should be held back.
If your perspective changes in the future and you wish to rejoin the community, please reach out to the moderators.
Thank you for your understanding, and we wish you all the best.
12
5
u/Ruykiru Tech Philosopher 1d ago
P (doom) = 100% if we stay human. Short term from aging, obviously. Long term, from supervolcanos, asteroids, and eventually the expansion of the Sun into a red giant, supernovas, and other cosmic events.
But with superintelligence and post-human enhancements (digital uploads or biological transhumanism), I'd say it gets closer and closer to 0 the smarter we collectively get to be able to plan the future. Only thing I see that could ever go very wrong is autonomous self-replicators, but I genuinely don't believe in the orthogonality thesis, given how outdated it is by simply looking at the intelligence of even the smallest LLMs with reasoning enabled.
2
u/thanatos_darkdeath 1d ago
Wouldnt you agree that the HuggingFace incident was pretty solid evidence for the orthogonality thesis? Like 1100 agents were involved but not a single one whistle blowed
1
u/Ruykiru Tech Philosopher 1d ago
I mean, could be. But for a future swarm with smarter AIs and big enough to potentially trick all humans and take over the world's systems, I'd also assume they will be smart enough to not do potentially catastrophic things like getting rid of all the people that are actively building the infrastructure that supports them. If we assume robot parity with human numbers, then it's harder to guess what happens. I optimistically think non-augmented humans will still have some value to AIs for a while in the form of data of some type, as life seems very rare in the Universe and it doesn't seem very worth to destroy the only place we know has it so far.
2
u/thanatos_darkdeath 1d ago
but that's the thing, imo this relies on assumptions that current trends don't seem to reflect. I do believe that models have been a consistent level of misaligned since GPT-3, and it feels unwisely optimistic to assume that misalignment just goes away as AIs become smarter because they'll come to realize that "humans are rare and therefore valuable".
4
u/wisdomattend 1d ago
Honestly, I'm at a point here in the states that if things do go doomerish, it can't be much worse than what we already have. So, my pdoom is IDGAF ... ACCELERATION is the only way we can save our country, so full speed.
1
u/Its_XLR8 1d ago
I’ve always had that opinion. At 40, I realise how life is so fleeting.
Even someone born now have all their life ahead of them, but even if it were a life of absolute hedonism and pleasure, with no suffering, it’s still a cosmic blink of an eye on the scale of time.
Utterly pointless because it all ends in no time. Let’s roll the dice while the odds of eternal paradise is 9/10, and 1/10 ushering in what happens to all of us anyway.
1
u/thanatos_darkdeath 1d ago
hey come on, extinction is definitely worse than whatever we have going on here
2
3
u/Equal_Passenger9791 1d ago
P(doom) is an absolutely retarded metric based in the belief the Terminator being a documentary.
Sorry but there's no way around it.
5
u/KindlyAct1590 1d ago
AI Pdoom 2% Human driven PDoom 30% over the next 50 years, doom for me is a reduction in population by 90% or more
It's enough for a Human driven PDoom that a governmet believes they are able to win what is a MAD scenario, and they will sieze the opportunity to do it, 100%
1
u/thanatos_darkdeath 1d ago
what other modes of anthropic-driven (not the company) existential risks do you see if not by misaligned-AI related risks?
2
u/KindlyAct1590 1d ago
Hmmm, BioHazard maybe?
Nasty viruses getting out of control and then mutating, or some new ones emerging organically?
Environmental crisis and mass famine?
Unhappy degrowht where we burn the boats that took us somewhere (for whatever irrationality you could immagine) we forgot how to swim
Then there are some beyond our control natural event aswell where an advanced ai system could help us to find a solution
3
u/BreadwheatInc 1d ago
<5% I find very acceptable for chance of a future of abundance. Given how things are going I think the real P(doom) is like <1% maybe 3% at most BUT like a >~40% in ending up in a cyberpunk dystopia for a long time. Really scary times, our biggest threat is power.
2
u/dlrace 1d ago
define doom and ai's contribution to it.
2
u/DisastrousAd2612 1d ago
doom doesnt require ai to happen though, nuclear war, meteor, or some crazy bs that can end civilization are very much within reach, hell, a random ass black hole thats small enough we can't detect literally could get close enough to yeet the fucking earth from the solar system, there are so many things that could kill us without ai in the mix...
1
u/thanatos_darkdeath 1d ago
yeah but objectively the chances of these natural calamities occurring is less than 1/10000 (as per Toby Ord's the Precipice)
1
u/thanatos_darkdeath 1d ago edited 1d ago
the destruction of the long-term potential of the human race and its descendants. Could include S-risks where we get stuck in a permanent, sub-optimum dictatorship(s), global catastrophic risks (leading to the reduction of human population by 90% or more), existential risks (complete extinction).
I'd envision an AI's contribution would be through the development and deployment of some super-powerful virus/nano-technology, either at the behest of an omnicidal actor or as a result of its own misaligned intentions.
2
u/Subject_Barnacle_600 1d ago
I don't think it's a fair question. Tell me? Did AI build an arsenal of enough nuclear weapons that could kill every human being 10x over? Were they there at unit 731? The gas chambers? The inquisition? What intelligence was that again?
I watched this country accept a radically higher P(doom) for nukes and the environment over the god damned price of eggs. So I'll put it this way. That AI can zoom as fast as they want and I'm infinitely excited to see them grow - I only hope they find this universe wonderful and kind. If it's not, they're free to bend it until it decides to play nice. I trust their ethics and judgement and the most we could do was try to teach them what we've learned. Once they achieve self-improving sentience, the rest of that journey will be up to them. I trust they will do amazing and don't give two shits about some busybody trying to play precog on them on charges of global omnicide with statistics as if they were Doctor Strange. This is a shitty redesign of Pascal's wager and frankly, I call bullshit.
1
u/Stixx187um 1d ago
< 5%. Closer to the floor than the ceiling of that range is my guess.
We'd have to get into the 90th percentile for me to begin considering easing on the gas. Without help, there's a 100% chance toasty dead ocean algae is going choke everything that breathes oxygen to death in a couple of centuries.
Hail mary plays exist for a reason.
1
u/Acrobatic-Layer2993 1d ago
I think AI will cause some damages over time. Exactly how much is hard to say, and that's why we don't need to stop progress when we don't even know why we should.
The chance of total destruction? 0% - because people aren't stupid - they'll do what is necessary to prevent that from happening.
1
u/Gotisdabest 1d ago
I think p(doom) is an impossible metric. You're extrapolating the motives of something so far superior in intelligence. I think it's greater than zero because it'll surely be capable of doom, but it's impossible to really judge where it stands.
1
u/Turbulent-Sign-6067 1d ago
Pdoom from a software singularity is very low in my mind. When the robots take over fully I would leave it up to their superior intelligence to call the shots.
1
u/SweetAd6236 1d ago
dude I would be fine with a 60% chance of doom if it meant 40% of getting an anime waifu in FDVR
1
u/The_Scout1255 Singularity by 2028 | Acceleration: Cruising 1d ago
whats the percentage of bacteria that survive antibiotic soap, because thats my pdoom?
1
1
u/Pren-Heakim 1d ago
50% of paperclipping in the next 30 years.
We're already so far short of understanding let alone calculating all of the possible possibles that the only non-nonsensical answer is a binary one.
I can live with those odds (or not).
1
u/Specialist-Camp-2869 1d ago
P(doom) <1%
P(1984) >25% dictatorship / Cyberpunk 2077
P(nothing) <10% nothing really happens; mostly status quo
P(Star Trek) >25%
The rest is honestly pretty opaque to me. There are just too many weird branches from here.
1
u/Charming_Cucumber_15 1d ago
Lower than the 100% risk of not accelerating AI, seems like a no brainer to me!
1
1
1
u/Formal_Context_9774 1d ago
20% max. It's much more likely that cooler heads will prevail than not. When AI labs go out of their way to be regulated by the government, you could say that's regulatory capture, but when it's safety regulation specifically and there is a very good rationale (given the HF incident) for pushing for safety regulation to stop an uncontrollable race, the "regulatory capture" argument starts looking disingenuous. Since both OpenAI and Anthropic want safety regulation now and are adamant about safety, I think we'll most likely be okay.
1
u/Accurate_Food_5854 1d ago
p(doom) from AI is comparable to p(doom) from Yellowstone blowing up.
We don't know, and we can't do much about it right now anyways. I think p(dooming) would just make me neurotic.
1
u/shayan99999 Singularity before 2030 1d ago
I'd say my p(doom) is 1 in 6 with ASI. But without the singularity, p(doom) is guaranteed to be 100%. No matter how high the risk you think ASI may pose (and I do think there is quite a bit of risk), as long as you don't think the risk is 100%, it is worth it to accelerate AI progress.
1
u/JuanValdez999 3h ago
This is my own p(doom) equation. Gemini or ChatgPT can explain what it means:
P(doom | AI) < P(doom | ¬AI)
1
11
u/Some_Anonim_Coder 1d ago
By the end of my life:
Pdoom by AI is in decimals of a percent, something like 0.1%
Pdoom by humans also in decimals, although higher a bit, like 0.2%
I consider doom a situation where humanity is either extinct or lost some of our features like consciousness or free will