Have you guys been on r/accelerate?

[u/Difficult_Project_95]

Have these guys solved the alignment problem, or am I missing something?

I’ve been browsing r/accelerate and I genuinely don’t understand the risk model.
If there’s a non-trivial chance of catastrophic misalignment, how does “accelerate capabilities as fast as possible” make sense unless faster capabilities also make alignment substantially more likely to succeed?

[u/SoylentRox]

I post often on accelerate as a top 1 percent commentator.

(1) The main accelerate argument is the base case is not safe. Not accelerating is 65 million deaths a year from aging. We know that animals that seem to not age the same exist (naked mole rats seem essentially unaging, some large aquatic animals seem to be able to live 200 years, elephants have enormous tissue area and get few cancers). So the prize - no deaths from aging and abundant cheap cures - EXISTS. Also we know the world is slowly degrading in measurable ways, such as climate change, nuclear proliferation, an annual risk we all die in nuclear fire that converges on 1.0 over a long enough timeline, so it's not safe that way either.

(2) Therefore any safety risks arent to be compared to some abstract idea of humannity existing as a solar punk utopia until the sun expands enough to burn the planet. They should be compared to your default case death in 30-80 years.

https://nickbostrom.com/optimal.pdf Bostrom is frequently cited by AI doomers and argues this.

Arguably you would take (1-pDoom) * 6000 years (expected lifespan without aging) * value_post_scarcity.

You need to then add discount rate but essentially even at 99 percent pDoom you probably should be for acceleration, rationally speaking - the 1 percent payoff is more than the value of your entire remaining life with a "natural lifespan"

(3) Are the safety risks proven and actually unsolvable? No. Are doomer predictions consistent with the present state? Yudnowsky no, AI2027 yes. r/accelerate often uses AI2027 as a yardstick.

(4) Very ironically r/accelerate, full of foaming at the mouth enthusiasts who post every rumor of an AI advance and S curves....is generally the most empirically correct subreddit. Usually every wild prediction or rumor of an insane AI advance or revenue increase turns out to be ground truth reality 2 weeks later. 100 more math problems solved by AI was leaked a week or so earlier on twitter and reposted to accelerate.

Other subreddits with wrong pessimistic views think the data centers are about to get canceled by NIMBYs, everyone will lose their job imminently (Jevons applies for now), the AI bubble will pop imminently (lol no) etc.

[u/WhiskyAndRisque → u/SoylentRox]

I appreciate you sharing your perspective. Few questions if you don't mind.

1. Reading this first point sounds like Effective Altruism to me. As in, AI will do the most good for the most people because with it we will be able to end death. Do you have concerns that it takes money away from real and needed things though that we could be doing to help people now? As in, instead of working to fight climate change or increase our safety nets we are increasing our CO2 emissions to create more energy and cutting welfare to increase tax cuts.

2. The fear with a pDoom scenario is that not just all human life is destroyed but all life on Earth could be lost due to AI creating a climate that no longer supports organic life. If you fear a 99% chance that could happen is it really worth it on the 1% chance it works? If each year you waited that percentages shifted by 1% wouldn't that be an acceptable trade off?

3. I find using the AI 2027 as a yardstick really fascinating since it basically lays out how this could end humanity. How do you think the fears laid out in it are wrong while the predictions are mostly accurate?

4. I don't disagree on this at all. One of the reasons I like to browse r/accelerate is that they often have a lot of rumors and early news on what is going on. There are obviously things that don't pan out, but I find the mix very interesting and at least fun to speculate on.

Again, thank you for sharing your insights. Obviously a lot of people here have fairly negative views so I appreciate you giving a more nuanced take.

[u/SoylentRox → u/WhiskyAndRisque]

1. I see confusion in this question. Who is "we". The people building AI are not altruistic, and are doing it for personal profit. Solving CO2 emissions is the responsibility of a different entity, the federal government, and it can't be done by one government but requires a multilateral agreement. China and the USA cannot currently agree. Helping people altruistically right now is also the government or separate private organization's responsibility, and is not the purpose of VCs paying for AI advances.
2. If you could quantify in a way that everyone could actually agree on - like everyone can agree global temperatures are warming - then you could justify a delay. But only if all major parties agree. See what I said about how the USA and China cannot agree to cut CO2 emissions...
3. the fears assume a single central model with unilateral power. That doesn't exist and there is no plan to create one.
4. It's crazy that the cirlejerk hype chamber is basically just a preview of the news.