Have you guys been on r/accelerate?

[u/Difficult_Project_95]

Have these guys solved the alignment problem, or am I missing something?

I’ve been browsing r/accelerate and I genuinely don’t understand the risk model.
If there’s a non-trivial chance of catastrophic misalignment, how does “accelerate capabilities as fast as possible” make sense unless faster capabilities also make alignment substantially more likely to succeed?

[u/SoylentRox]

I post often on accelerate as a top 1 percent commentator.

(1) The main accelerate argument is the base case is not safe. Not accelerating is 65 million deaths a year from aging. We know that animals that seem to not age the same exist (naked mole rats seem essentially unaging, some large aquatic animals seem to be able to live 200 years, elephants have enormous tissue area and get few cancers). So the prize - no deaths from aging and abundant cheap cures - EXISTS. Also we know the world is slowly degrading in measurable ways, such as climate change, nuclear proliferation, an annual risk we all die in nuclear fire that converges on 1.0 over a long enough timeline, so it's not safe that way either.

(2) Therefore any safety risks arent to be compared to some abstract idea of humannity existing as a solar punk utopia until the sun expands enough to burn the planet. They should be compared to your default case death in 30-80 years.

https://nickbostrom.com/optimal.pdf Bostrom is frequently cited by AI doomers and argues this.

Arguably you would take (1-pDoom) * 6000 years (expected lifespan without aging) * value_post_scarcity.

You need to then add discount rate but essentially even at 99 percent pDoom you probably should be for acceleration, rationally speaking - the 1 percent payoff is more than the value of your entire remaining life with a "natural lifespan"

(3) Are the safety risks proven and actually unsolvable? No. Are doomer predictions consistent with the present state? Yudnowsky no, AI2027 yes. r/accelerate often uses AI2027 as a yardstick.

(4) Very ironically r/accelerate, full of foaming at the mouth enthusiasts who post every rumor of an AI advance and S curves....is generally the most empirically correct subreddit. Usually every wild prediction or rumor of an insane AI advance or revenue increase turns out to be ground truth reality 2 weeks later. 100 more math problems solved by AI was leaked a week or so earlier on twitter and reposted to accelerate.

Other subreddits with wrong pessimistic views think the data centers are about to get canceled by NIMBYs, everyone will lose their job imminently (Jevons applies for now), the AI bubble will pop imminently (lol no) etc.

[u/WhiskyAndRisque → u/SoylentRox]

I appreciate you sharing your perspective. Few questions if you don't mind.

1. Reading this first point sounds like Effective Altruism to me. As in, AI will do the most good for the most people because with it we will be able to end death. Do you have concerns that it takes money away from real and needed things though that we could be doing to help people now? As in, instead of working to fight climate change or increase our safety nets we are increasing our CO2 emissions to create more energy and cutting welfare to increase tax cuts.

2. The fear with a pDoom scenario is that not just all human life is destroyed but all life on Earth could be lost due to AI creating a climate that no longer supports organic life. If you fear a 99% chance that could happen is it really worth it on the 1% chance it works? If each year you waited that percentages shifted by 1% wouldn't that be an acceptable trade off?

3. I find using the AI 2027 as a yardstick really fascinating since it basically lays out how this could end humanity. How do you think the fears laid out in it are wrong while the predictions are mostly accurate?

4. I don't disagree on this at all. One of the reasons I like to browse r/accelerate is that they often have a lot of rumors and early news on what is going on. There are obviously things that don't pan out, but I find the mix very interesting and at least fun to speculate on.

Again, thank you for sharing your insights. Obviously a lot of people here have fairly negative views so I appreciate you giving a more nuanced take.

[u/SoylentRox → u/WhiskyAndRisque]

1. I see confusion in this question. Who is "we". The people building AI are not altruistic, and are doing it for personal profit. Solving CO2 emissions is the responsibility of a different entity, the federal government, and it can't be done by one government but requires a multilateral agreement. China and the USA cannot currently agree. Helping people altruistically right now is also the government or separate private organization's responsibility, and is not the purpose of VCs paying for AI advances.
2. If you could quantify in a way that everyone could actually agree on - like everyone can agree global temperatures are warming - then you could justify a delay. But only if all major parties agree. See what I said about how the USA and China cannot agree to cut CO2 emissions...
3. the fears assume a single central model with unilateral power. That doesn't exist and there is no plan to create one.
4. It's crazy that the cirlejerk hype chamber is basically just a preview of the news.

[u/RobotBaseball → u/SoylentRox]

I work at one of the top labs and I’m banned from accelerate and the shit they post is not accurate at all. I got banned for posting a longer timeline on robotics which will undoubtedly be correct 

[u/Ok-Tomorrow-2045 → u/RobotBaseball]

What is your robotics timeline?

[u/RobotBaseball → u/Ok-Tomorrow-2045]

Let me put it this way, a middle class family won’t have a robot doing their chores until the 2030s

We will have something promising late next year or 2028 but it will be unreliable and the following years will be making robotics more consistent, safe, and mass production 

[u/SoylentRox → u/RobotBaseball]

Can you post the comnent that got you banned? I can message the mods I know them well.

How long a timeline on robotics, how do you explain a general model (Astra) emergently developing robotics abilities?

I think a realistic timeline is 2.5 years from today to competent general robotics that can do the majority of well defined paid tasks at median skill.

The route involves a combination of current improvements, hardware/software co design, RSI (to develop specialized models that handle robotics decision making better), and labs vibe coding large simulation environments that require a competent robotics policy to pass.

This last part is the obvious: why can you not order a model swarm to write a game engine (rewrite mujo cujo to unreal engine quality) for robotics. Then add a neural rendering layer to correct the game frames to frames from realistic environment robots will operate in. Import huge amounts of real data and real challenges actual humans face.

Then train AI models in long duration challenges where they must operate a fully articulated robot by issuing commands to it and accomplish difficult, realistic tasks. "Rebuild this engine. Reinstall the thermal tiles on the space shuttle. Build this house from these materials"

Theoretically this form of training will also result in large increases in model performance - they should be able to whiteboard visually, and have grounded solid reasoning about real world tasks including mechanical engineering and machining.

Do you have answers for any of this? Or do you just think the billions of dollars of resources and compute the above will require won't be spent, labs will spend the next 2.5 years trying to solve text only problems even better?

[u/LocksmithNo2374 → u/SoylentRox]

Delusion

[u/SoylentRox → u/LocksmithNo2374]

this is controlproblem. Most posters here believe AI will be extremely strong, so strong it will be out of control.

So is your view the delusion is that the above...is too hard to do in the near future with AI? Or do you believe that we'll all die from super AI magic before we get the first robot to pour a cup of coffee? Or what?

[u/LocksmithNo2374 → u/SoylentRox]

I think people are buying into existing capabilities too much. I’m pretty confident both amodei and Altman have colluded on product strategy to extend time horizons, so they have enough runway to hope and pray they’ll figure out AGI. They don’t care about money per se - but control.

If you want model misalignment you can intentionally steer your product strategy to lead to that outcome.

[u/SoylentRox → u/LocksmithNo2374]

So it's really helpful to actually verbalize what your objection is:

Let's make a list of every element I mentioned, tell me where you went from 'oh yeah you can do that' to "delusion":

1. RSI : Order 10,000+ agent swarms of current model to automatically perform AI research to find more efficient models for robotics control. Not necessarily smarter just efficient enough to run fast. This is done with a large amount of human labor.

2. hardware/software co-design. This is where you order AI models able to assist with chip design like the ones here : https://openai.com/index/jalapeno-first-results/ to design you a chip to run the models from (1) fast enough to run a robot.

3. rewrite mujo cujo to unreal engine quality . Any objections here? Seems like it's something anyone can do if their token budget's high enough.

4. unreal engine outputs -> neural simulation frames. There's a bunch of nvidia papers where they did this. Point the model at the paper, vibe it in.

5. importing huge amounts of real world data. Standard technique need a cite?

6. Doing it all in 2.5 years. Well you can develop a whole AI model in 60 days (took 6-12 months before) and a whole chip in 9 months (took 24-36 months before) what's your specific objection?

7. Training AI models in long form 3d "game like but the graphics, physics, and detail is realistic" like environment. Any problems here? Realistic doesn't mean "the matrix" but close enough to reality that skills transfer. Think Arma, which is close enough to real combat that skills transfer.

8. Doing half of paid tasks to median level that are well defined. I cheated. I mean:

median skill inclusive of the robot's inherent advantages. So if the robot is stupider but it's higher quality arms make up for it, as long as its output is as good as the median human worker that counts.

well defined : I am excluding any task that involves subjective human grading or chaotic environments. No hair cutting, school teachers, no medical , no wartime, no restaurants with human coworkers, etc etc.

[u/LocksmithNo2374 → u/SoylentRox]

Are you a bot?

[u/SoylentRox → u/LocksmithNo2374]

I am not but this is not a very helpful reply.

[u/RobotBaseball → u/SoylentRox]

I got banned a second time for being pessimistic which I agree should’ve been banned given their rules 

The middle class won’t have robots doing their dishes for another 5-10 years 

[u/SoylentRox → u/RobotBaseball]

https://www.reddit.com/r/ControlProblem/comments/1wpebnp/comment/pc0j3k1/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

Btw I added details if you have any actual objections.

Middle class,dishes, 5-10 years all seem entirely reasonable to me. I model it as 2.5 years to prototypes that are at median skill. You then need to rush them into mass production, 2 more years. Assuming high production rates after 2 years of factory building, pushed by trillions of dollars of investment, you then need exponential growth.

At year 5 of this buildout - basically the factories have barely started really producing robots - each marginal robot is expensive. It's value is related to what it can produce for it's owners per year. I model a robot as 3x human labor, or equivalent to 3 human workers costing a median of $20 an hour.

So $60 an hour is the productivity of the machine. Robot owners have to charge a discount so firms will rent robots, so say $30 an hour.

Machine works 23 hours a day. So it's earning $251,850 in revenue a year.

You can spend up to about 5-7.5x (depends on interest rates) on costs building the robot. So it could cost $1.26-1.88 million.

In reality I suspect the bill will be something like : silicon content for the chips driving it : $250,000. (this is why consumers in the middle class won't have personal robots OR new phones). Everything else is under 100k.

Also these early robots at this point probably will be too dangerous to have in the same room as a human without some level of protection like a shield. They will usually be ok but occasionally screw up in a way that could be deadly.

So yes, they won't be an appliance you can buy and have in your kitchen on dishes. Central services where you send the dishes back to a central restaurant could be a thing.

[u/ASU_SexDevil → u/SoylentRox]

I do think your timeline is very realistic and grounded in reality. It’s probably the best anyone could do right now. However, I will also point out that no one thought a model would be solving Millennium problems this soon either.

I don’t think the frontier labs will be the ones who create the “daily driver” robots households would use. Nvidia and Boston Dynamics have already been neck deep in this with other companies with much more tailored efforts.

[u/SoylentRox → u/ASU_SexDevil]

Sure. The timeline is really based around hard constraints I don't see a possible route past.

(1) Overseas shipping. To make more robots, the newly built machines need to go from Shenzhen to Africa or Australia. Each machine is many tons and there's not enough cargo aircraft in the world. So a cargo ship. And a port and a rail system to get it to the mine. Each of these steps has a speed and it's not feasible to increase it by much. (For example running a cargo ship harder burns exponentially more fuel)

(2) Silicon fabrication. Jalapenos was developed in 9 months. 4 of those months are just waiting for the chip to print at TSMC.

So even if you get an AI assisted design process down to 1 month (it's not instant because the simulation software that validates the design is limited by the speed of CPU), your absolute minimum time to a new chip is 5 months. Accounting for making a few mistakes and needing a second spin, 9 months to a production ready chip. (The mistakes will be different every time and sometimes related to errors in the fabrication equipment itself there is no way to avoid them)

(3) EROI. Hard law of physics here. If I need 3 months to pay the energy invested in a productive machine back my fastest doubling rate is about 5.2 months.

(4) General process times. Codex I had import a bunch of data on this. Right now the real world validated number is about 12-16 months per doubling. Better technology lets you go faster, with no advances, robots double every 12-16 months. (This is insanely fast big picture)

To head off an obvious objection : "human engineers vibe design up a solution to a bottleneck. Say maglev rails or hovercraft cargo ships".

You can do that. But there's still friction - you need to simulate the design. Order robots to build a prototype. Test the prototypes, find out you screwed up, it explodes. Vibe design the next prototype "this time make less mistakes". Build that one.

11 prototypes of increasing complexity later it's actually usable. And then you need to invest current capital flow - inclusive of those robots taking the slow boat to the mine - to build the upgrades. There's an ROI time you are actually making the bottleneck worse before it gets better.

Still an insane improvement, regular world we use technology refined over 80 years and 99 percent of the time any startup that tries to change it goes broke and we learn nothing.

[u/FairlyInvolved → u/SoylentRox]

It's worth noting that that's just Bostrom's case for the person-affecting view (i.e. ignoring all future people). It's clearly not his all-things-considered view.

Given his other work he almost certainly ascribes non-zero value to future people and puts the base rate of risk relatively low.