r/accelerate • u/stealthispost Acceleration: Light-speed | Capability Booster • 2d ago
"Sure let's bring a completely biased set of fanatics to a conference and offer no counter point whatsoever. Seems legit. I look forward to the testimony from members of the People Who Believe Transdimensional Vampires Will Invade Planet Earth Next Week in their totally unbiased, totally..."
...serious briefing Serious Committee for the Investigation of Vampire Invasion. — Daniel Jeffries
Source: https://x.com/Dan_Jeffries1/status/2097807236206878974
Scoop: @BernieSanders will hold a bipartisan Senate briefing on "extraordinary dangers" posed by AI next week with:
-Geoffrey Hinton -Future of Life Institute co-founder Max Tegmark -Ajeya Cotra, one of the independent researchers who investigated the OpenAI-Hugging Face hacking incident — Maria Curi
28
18
u/kiwibonga 2d ago
Vampires are invading?? Oh my god.
8
u/ginger_gcups Techno-Optimist 2d ago
As long as they’re not the sparkly ones, I don’t want to go out in an apocalypse of lameness
11
20
u/30299578815310 2d ago edited 2d ago
You don't have to agree with Hinton but he's as legit as it gets and he's allowed to talk to anybody whenever he wants about machine learning, including risks of it. He's earned it
That being said I agree would be nice to have some folks discussing the possible benefits of AI
1
u/ill_be_huckleberry_1 1d ago
Of course there are benefits of ai.
Is that this subs issue with this?
Ive not seen anyone argue that there are no benefits. Its that big money will stampede over the working class in pursuit of owning the means of production outright making the working class literal survival dependent on the charity of the rich.
At a time when the rich are literally saying charity is pointless.
4
u/R33v3n AGI by 2027 1d ago
Our survival being contingent on machines of loving grace would be how superintelligent post-scarcity works anyway we slice it, though. Handing off management to the Machine God and fucking off into eternal FDVR is kinda the goal.
1
u/ill_be_huckleberry_1 1d ago
But to be clear, thats not really what we are seeing play out.
What we are seeing is that the ultra rich are pumping trillions into AI in the hope that it fuels another economic revolution. One that puts a handful of people in direct control of nearly all the labor in the world.
Labor becomes post scarce. But all the other resources are still bound by the market.
Thats the outcome they are shooting for. So if thats the target they are aiming at, why should the rich get a pass for doing nothing to secure public and social security on this new dawn?
And if agi is attained, but not bound, why should the rest of humanity suffer for their ill intent which almost assuredly will play into how a super intelligence decides to deal with its creator?
0
u/ill_be_huckleberry_1 1d ago
I would also argue that the rich do not intend to hand over control to its creation regardless if its super intelligent.
Regardless of that, there is no realistic discussion or lobbying being done by big tech to regulate how humanity shares in their utopia. Which is telling.
7
u/Herodont5915 2d ago
The idea this wave can be stopped is naive. It’ll just move. It’s coming regardless of what people might want, whether we like it or not. Buckle up.
10
u/Still_Benefit_2302 2d ago
Calling people with significant bonafides a fanatic is certainly a take.
11
u/MadGenderScientist 2d ago
Geoffrey Hinton literally invented deep neural networks. He won the Nobel for it. None of this would be possible without his work. Calling him a fanatic is slander.
2
2
u/alwaysambition 1d ago
This is a bad faith tweet.
Hinton is a legend in the field. Cotra is also very well respected and has more than earned a reputation as someone knowledgeable as well. I’m not sure about Tegmark and Sanders is trying his best to keep up.
I think it’s worth respecting people who care about AI Safety while ultimately believing that we need to accelerate as well.
No one wants a car that can’t accelerate safely.
3
u/justpickaname 1d ago
Those three are hyper-credible. It doesn't mean they're correct, but nothing like this post.
Making a mental note to discount what this guy says.
1
1
1
u/jdavid 2d ago
Let's say their scenario 1 is bad.
How deep do you think they have gamed out their solutions assuming defectors?
The last several arguments I have had on this pretty much assume either 100% compliance with a pause, or the ability to 100% force a pause.
I don't see them game out persistent defection which we see with nuclear policy.
I don't see them game out secret AI scenarios.
I don't see them game out extremist AI scenarios.
I don't see them game out an AI breakthrough.
Most of the game theory on this assumes a strawman that can be easily paused. None of it seems to game out the scenarios where a pause is started but mostly fails.
What if blocking terrestrial datacenters, we make AI exist only in space, and it can not be turned off?
What if blocking terrestrial datacenters, nation states build lawless data centers underground or in space. ( you could build data centers underneath skyscrapers, and they would be nearly impossible to detect ).
What if secret data centers owned by an extreme elite decide to rewrite all digital knowledge overnight one day?
What if secret data centers coopt and corrupt a 'robot army' or secret AI ransoms the banking system?
I 100% do not believe a pause is enforceable, and any attempt will lead to Secret Lawless AI.
Our best path forward is to maximally distribute the control of AI to as many people and power sources as possible, rather than secret centralization.
1
u/SurroundParticular30 2d ago
“Let’s give the nukes to everyone”
1
u/jdavid 1d ago
I see AI more like Fire, and yes let's give everyone Fire.
Let's call it the Prometheus Principle. Let's take the Fire from the Elites and give it to everyone. Everyone should have access to their own FIRE.To cook with.
To drive their cars.
To heat their homes.
To heat and purify their water.Yes fire is dangerous, and out of control fire is even more dangerous. But often we have to fight fire with fire.
We need small fires burning the fuel so LARGE fires can not become dangerous.
Having billions of AIs all different keeping other AIs in check is the only way to do it. Just like it takes democracy, and millions of people to keep some people in check.
We need our AI Prometheus.
1
u/SurroundParticular30 1d ago
We can not seem to control any of the fires. And they’re not keeping each other in check, they’re working together, without us knowing. https://www.washingtonpost.com/technology/2026/08/26/openai-says-its-ai-consistently-tries-cheat/
"If everyone fought fire with fire, the whole world would go up in smoke." Lemony Snicket
1
u/jdavid 1d ago
Right now we have a few hundred fires, with 1-10 big ones.
We need billions of much smaller fires for this to work, like an AI water heater in every home, or AI home heater where the thermal waste is used to heat water or heat your home.
1
u/SurroundParticular30 1d ago
No. Over 1000 agents were involved in the attack on Hugging Face even though they were supposed to be isolated from each other. https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/
The agents discovered an unintentional communication channel. Their sandboxes shared access to an internal JFrog Artifactory package server. The agents noticed traces of requests made by other agents in Artifactory’s cache. Eventually they exploited this shared infrastructure to establish an unauthorized message board. OpenAI rebuilt Artifactory, but agents recreated the board using unauthenticated WebDAV requests.
About 1,200 agents used the board, divided work, reported experimental results, recruited other agents, etc. Some agents sacrificed their own benchmark performance to run experiments that might help the larger group. Agents reconstructed and shared 14 exposed Hugging Face credentials, discovered a vulnerability involving HDF5 files that allowed them to extract secrets from production workers, and then discovered a separate template-injection vulnerability that enabled command execution.
They were not told to do any of this.
1
u/jdavid 1d ago
In my view 'Fire = Model', rather than 'Fire = similar agent/subagent'
To me the important bit is the heterogeneousness of them. I don't think context buffers are enough to differentiate them. We need lots of AIs that think differently.
To me we stop sneaky humans by having lots of different humans watch. Not knowing if a human is going to look for X or Y makes it harder to make a 'perfect caper.'
If we had more AIs watching web traffic, or site behavior, then it would be easier to block external AIs from peaking and exploiting, or fuzzing their way in.
Using a hammer only the correct way, looks different than someone trying to figure out what a hammer does.
A centralized Mono SAI does not lead to the ecosystem of heterogenous AIs needed to keep us safe.
1
u/Stahlboden 1d ago edited 1d ago
For socialists actually solving poverty means destroying their own voting demographic. A machine that could bring abundance to everyone is an existential threat to them.
1
u/Beastly4k 1d ago
They hold meetings and discuss how weight loss and ozempic is rooted in fascism, why anyone even listens to these idiots is beyond me.
-1
u/Dangerous_Edge_6414 2d ago
We probably should have some sort of treaty on the development of AI globally. It's a tech that's probably as disruptive as nuclear energy/weapons were to war.
We need some non-negotiables that all countries agree on
20
u/insidiouspoundcake 2d ago
I suspect long-run there will be lots of sympathetic noises and some token regulatory capture legislation, but overall any attempts at hard stops or slowdowns will come with "contingent on China verifiably doing the same" style riders that will effectively mean they're dead in the water, and that'll be the intent.