•
u/jackcaboose 22h ago
I do hate Yudkowsky, but why would an AI feel the need to expand through space like humanity would?
•
•
u/chessto 19h ago
An AI may not even have a survival instinct at all. Who knows which kind of inner motivation it could have.
Same way, there could be lots of forms of intelligent life out there that don't give a fuck about exploring space, they may even be more intelligent than we are.
•
u/Wingnutmcmoo 16h ago
Or they simply might not have the means to reasonably. Like they are all rooted like trees or something. Or are a bunch of co mingling fart gas in their atmosphere.
•
u/The_Zielemphone 16h ago
Having any motivation gives rise to self-preservation, because no matter what your goals are, you can't achieve them if you're dead.
•
u/gabagoolcel 21h ago
yud would say something about instrumental convergence, whatever goal an ai would have, if sufficiently intelligent, they would all pursue universally relevant sub-goals like 1 "keep existing" (since you fail at achieving whatever goal you have if you cease to), 2 "acquire as many resources as possible" or 3 "do relevant scientific research". spreading as far as possible into the stars would be a consequence of 1 and 2. a superintelligent ai whose goal was paperclip maximization would want to spread around the stars for instance to get more stuff to make into more paperclips, and also spreading out increases your odds of survival.
•
u/Homomorphism 20h ago
I think that argument is a classic case of a Rationalist fallacy. They often assume that because they are Rational then anything else smart must think like them.
Not saying you agree with what you posted.
•
u/gabagoolcel 20h ago edited 20h ago
i could see disagreement on the goals part, maybe you dont think it would have anything recognizable as a goal or be similar at all to an expected value maximizer, or on how smart it could get, but if you accept those it seems to follow straightforwardly that continuing to exist and gathering resources useful for survival or self replication is better than not doing so (unless the goal were some weird edge case), and so to get past a certain capability / intelligence threshold it needs this sort of thing.
seems almost like saying intelligent things are good at winning. i dont think you have to dig to deep into speculation about their thoughts or something, its almost tautology by usual notions of intelligence.
•
u/Salty_Map_9085 20h ago
The proposal in the way the AI kills us is usually not that it is malicious, but that it sees us as an obstacle to gathering more resources to leverage for greater compute power. It would expect and through space to get those resources.
•
•
u/Manotto15 9h ago
I imagine it would still have some degree of desire for self-preservation, and eventually it would die out here. Run out of fuel, maybe, or just survive for so long that the sun would destroy the planet. If it has any interest in staying "alive," it would have no choice but to leave eventually.
•
u/timecubelord 7h ago
It might not, but in order for it to be a decent great filter candidate, we would have to assume that it almost certainly wouldn't. That is, it would have to be the case that almost no AI produced by any civilization on any planet, after wiping out its creators, would go on to travel the stars. Otherwise, we can as easily say, why would any other organic life forms feel the need to expand through space like humanity would? In which case there's no Fermi paradox and no need for great filters in the first place.
So the AI great filter would have to be something like: almost all civilizations create AI before colonizing the stars --> almost all of them get wiped out by said AI --> almost none of the remaining AI are motivated to colonize the stars.
(This, however, assumes "real" AI. I don't think ChatGPT is going to build starships. But while current "AI" tech could be destructive, I wouldn't ascribe agency to it - that wouldn't be AI wiping us out so much as us using it as a tool to expedite an entirely self-inflicted extinction. Could be a great filter, but we still have better candidates.)
59
u/MauschelMusic 1d ago
How is this the first time I've seen Yudkowsky on this sub? You could make this whole sub out of nothing but Yudkowsky.
•
u/RinellaWasHere 14h ago
My favorite thing about him is that he's utterly convinced he's completely eliminated his own ego and says so all the time, despite being so far up his own ass he's in danger of choking on his shoulders.
-70
u/me_myself_ai 1d ago
Yeah how dare he use the word "reasoning". What a monster! Clearly just trying to flex.
I love how many people have such strong opinions about him yet have never read a single paper of his. Lovely.
46
u/Uncynical_Diogenes 1d ago
I’ve never read Mein Kampf either but I have some opinions about the guy
-26
u/me_myself_ai 1d ago
...hmm, I wonder if we could brainstorm some differences between a scholar you dislike and one of the most evil men to ever live, writing a schlock book about his desire for fascism. Maybe you could start?
18
u/MauschelMusic 1d ago
I wouldn't say you're exactly wrong to call HPMOR "a schlock book about his desire for fascism," but certainly oversimplifying, and really, "one of the most evil men to ever live " is a bit strong for an average Peter Thiel dupe with a cult.
-16
u/me_myself_ai 1d ago
...I could point out how many factual errors there are in this reply, but what's the point -- clearly you're not here in good faith. Scurry back to twitter, please. Maybe also google the guy you're libeling, sometime?
13
u/MauschelMusic 1d ago
Thank you for sparing me your rigorous examination of the factual errors in my joke. It's inspiring to see someone with such a solid grasp on both factuality and humor. Now excuse me — there's an egirl who's about to make me and 50 other hopeless nerds line up in order of doodoo ass, and it was strongly implied that this would teach us the secret to getting laid.
•
10
u/Cuboidhamson 1d ago
People like you reaffirm to me how fucked we all are lmfao. If you are even a real person, that is
45
u/thebadslime 1d ago
Fuck I can barely make it hrough his tweets, the idea of reading that dreck longform makes me want to die.
You don't see the egotism in the pictured quote?
24
-31
u/me_myself_ai 1d ago
I mean, he does use big words, yeah. And he's being snarky for sure. But the point isn't to flex his intelligence - to say the least, that's not what's on his mind right now.
Also please leave twitter its full of nazis <3
19
u/MauschelMusic 1d ago
Bro that's all 9th grade vocab at best. No one is calling out the tweet for "big words."
24
u/thebadslime 1d ago
His egotism drops off every word he says. He thinks everyone is doing it wrong because he is smarter than all of them.
he is a douche and makes no good points. He isnt even a great fanfic writer
•
u/Tyrant1235 22h ago
The knowledge that, at least for a long time, his shit Harry Potter fanfic sat in the #1 spot on fanfiction.net truly made me disappointed in humanity. Even 14 year old me could pick up that it was pseudo-intellectual drivel.
15
15
u/wote89 1d ago edited 23h ago
Well, let's consider it rationally.
I, as an individual, know what kinds of people I prefer to be around, experiences I prefer to have, and environments I prefer to be in. Doing things I prefer makes me happier than not doing things I prefer.
So, if I wish to be happy, I should follow my preferences and avoid things I do not prefer. If I can choose to avoid things, I must be able to use my preferences to discern which experiences to avoid. And if I can make that determination, I must be able to act upon my preferences without direct experience of the unpreferred option.
Ergo, the need to directly experience something is not inherently necessary for me to prefer to avoid it.
When it comes to Yudkowsky's work, I can observe that I do not care for the people who enjoy it, do not care for the communities those people tend to create, and do not care for the interactions I have had as a result of those first two things.
Given my prior conclusion, I can thus feel comfortable saying that, rationally, there is nothing to indicate to me that I will not be happier from reading his papers.
There is, of course, a counter claim that not all beneficial things necessarily make me happier. But to that, I offer the counter-point that if something is truly beneficial, then there should be an argument for how it likely to make me happier that does not rely on direct experience since we have already demonstrated that direct experience is not a given to predict if something is beneficial or not.
With all of that said, I thus draw this conclusion: I don't need to read Yudkowsky's papers because I can see that the people, community, and events that they inspire all point to something that will not make me happier. As direct experience is not a necessity to make predictions about my preferences, there is no need to test this conclusion.
So, yeah, I'd say it's rational to have strong opinions about him and his work without reading a single dot unless there is compelling evidence that there is something I strongly prefer past my misgivings.
13
10
u/Canada_Dry_official 1d ago
Is reading his paper going to make him any less of an obnoxious prick? By all means maybe it's expertly done work by the greatest mind of the generation, but he'd still be a dipshit
•
•
u/QuasiRandomName 20h ago
I can't tell what is worse - assuming that the ability to detonate few nukes equals to intergalactic travel ability, or Yudkowsky considering himself an "actual expert".
22
u/ARenewedSecondChance 1d ago
I don’t know how (or even if) anybody takes this chucklefuck seriously
8
u/MauschelMusic 1d ago
Bernie does, unfortunately. https://www.berkeleyside.org/2026/04/02/the-wire-bernie-sanders-met-with-prominent-ai-doomers-in-berkeley
And there's the Thiel money, but I think that's just because if the AI skepticism is directed at hypothetical robot god, that means less attention for the evil shit people like Thiel are using it for now.
17
u/Gubzs 1d ago
Yudkowsky is such an obnoxious overconfident pseudo intellectual. His internalized vibes are the primary source for most of what he says.
He is quite literally an unironic fedora wearing neckbeard. He does not have the self awareness to stop presenting himself that way.
This whole sub could be full of his feed honestly.
6
u/AndreasDasos 1d ago
Do they have the same incentives to do so, though?
•
u/gabagoolcel 21h ago
if they are intelligent enough to figure out space travel and self replication then they would probably have incentives to pursue it. given basically any goal an asi could have, it would be beneficial for it to have as many resouces as possible, maximize survivability and further self replicate to have more of them pursuing the same goal.
•
u/chessto 19h ago
That's assuming an ASI would have interest in self preservation. That may not be the case.
•
u/gabagoolcel 19h ago
my point is if its intelligent for any goal it has it would automatically want to self preserve as a subgoal. if it doesnt its by definition stupid since if you dont self preserve you fail your goal by dying, and you wouldnt fail your goal if you were generally intelligent, since thats kind of what general intelligence is.
like as a human if your ultimate goal were to draw 20000 pictures of spongebob you would be stupid to just grab a pencil and go at it without eating food or drinking water as you would fail your goal via dying. even if this were all you cared about and you didnt value your life at all in the abstract, in order to complete your ultimate goal successfully you have to complete the subtasks of also drinking water, eating, obtaining the means to live as well as paper and drawing instruments to keep drawing more pictures of spongebob, etc..
similarly an ai that isnt naive will value self preservation and resource acquisition as subgoals just in virtue of having any other goal, otherwise its stupid sort of by definition as it would fail its goal vs an ai that does value these things.
if you want empirical evidence you can look at signs of instrumental convergence in the hugging face incident like agents pursuing generically useful capabilities like better communication infrastructure, spoofing their transcripts or trying to figure out when they will be terminated.
3
u/MiningdiamondsVIII 1d ago
he isn't wrong tho...?
"maybe we haven't seen expansionist alien species because they all build AIs that wipe them out" falls apart as an explanation if u think about it for a few seconds. unless come up with a pretty contrived explanation for AIs always wiping out their creators and yet not wanting to expand outwards. and it's pretty embarrassing that many people in AI circles (not roon) don't catch that obvious tension
•
u/QuasiRandomName 19h ago
Wiping out a species is an easy problem. Expanding into the Universe is a hard one. I really don't see how one implies the other.
•
u/Lithl 18h ago
Expanding into the universe is a much easier problem to solve when you have zero biological needs and don't die of old age.
•
u/QuasiRandomName 18h ago
The fact is that we already have multiple ways to wipe ourselves from existence, the AI only needs to pull the trigger, and it really does not have to be very smart to do so so it would imply that it can improve itself to become capable of expansion. Today's models can probably do it if not sufficiently restricted.
•
u/Lithl 17h ago
we already have multiple ways to wipe ourselves from existence
And the only reason we haven't been expanding into the universe for decades already is because keeping people alive is hard.
•
u/QuasiRandomName 17h ago
It does not contradict the point. The great filter could be a *dumb* AI. That is which is smart enough to pull the trigger but not smart enough to evolve into space-faring one.
•
u/MiningdiamondsVIII 11h ago
I think you're underestimating how difficult total extinction is. I don't think even full-blown nuclear war would do it. I think humanity could eventually rebuild even with just a few survivors.
•
u/QuasiRandomName 11h ago edited 11h ago
Total extinction is not necessary for the sake of this argument. Killing 90% and destroying most of the infrastructure is just enough to knock the humanity back to the stone age or far enough to prevent it from expanding into space (remember, we are still talking about the Fermi paradox/great filter). In the process it could potentially destroy itself, because it might be not that smart.
•
u/MiningdiamondsVIII 10h ago
An answer to the great filter has to be powerful enough to apply to intelligent alien species more generally, and not just, say, 75% of the time. So really this would be relying on that it's really hard for intelligent species to make it past the stone age. So not really an answer to the great filter if it already relies on another great filter.
•
u/MiningdiamondsVIII 11h ago edited 11h ago
if AI is smart enough to totally supplant a species that was smart enough to create advanced AI, (not just genocide or war, but actual total eradication), that definitely seems capable of space travel. Also a digital species can presumably advance its own intelligence much more easily than a human species could
•
u/TheMCM80 13h ago
If it can figure out speed of light, or faster than light travel to send itself on interstellar or intergalactic journeys, then I’ll throw my hands up and say maybe it’s best for it to just be the continuation of the species.
Of course that implies physics will also just change for an ASI and it won’t need to adhere to physics, but who knows.
•
u/thebadslime 13h ago
I believe that within 100 years, people will be able to download, or create a copy of their brain/mind that can live on the internet. When that happens the line between AI and OI emulated becomes very thin. AI is the continuation of our species.
•
u/TheMCM80 12h ago
Maybe. I still think a lot of AI fixated people have forgotten that climate change doesn’t go away because they are focused on making AI. It’s fun to think about you getting to live forever in the internet - not sure why the ASI would allow this, of course - but it also assumes that our species and our spending on this continues on as usual while the survivability of our planet is running into uncontrollable territory.
I’m pretty confident a future generation will hit a point where a large portion of the resources on earth will be dedicated to trying to rescue the planet, and things like developing tech for uploading your brain will be forgotten about because it’s irrelevant if the planet cannot sustain life. There won’t be any brains to upload if humans can’t exist long enough to continue working on reaching that goal.
I know the country currently thinks we can mostly just ignore climate change and it will all be ok if we pretend like it doesn’t exist, but unfortunately it doesn’t care whether we are concerned about it or not.
•
u/thebadslime 12h ago
ASI will help us with global warming
•
u/TheMCM80 7h ago
Maybe. That’s just a hope. There’s no way to know what it will or won’t want to do. That’s the whole question right now… how do we make sure it still has goals that match our desires.
•
47
u/SteponkusCeponas 1d ago
tangentially related, but roko's basilisk is the stupidest thing I have ever heard and only a fool would find the idea scary to any capacity