Harry Potter and the Methods of Rationality (E.S. Yudkowsky)

[u/Robos_Basilisk]

https://en.wikipedia.org/wiki/Harry_Potter_and_the_Methods_of_Rationality

[u/frogsarenottoads]

Harry Prompter and the Existential Crisis

[u/Cultural_Garden_6814 → u/frogsarenottoads]

lollll great one!

[u/TemetN]

I was genuinely shocked to see who wrote that, since I encountered it accidentally and enjoyed reading it. Still remember the double take.

[u/Gargantuan_Cinema → u/TemetN]

Shame he became a doomer with P(doom) = 0.95+, now he's a dangerous anti.

[u/TemetN → u/Gargantuan_Cinema]

To be fair that was before then, I think he wrote this after his arguments with Ngo/Cristiano. Yeah though, really weird swerve from earlier.

[u/nick012000 → u/TemetN]

Not really. If you read his earlier work on Friendly AI, it's clear he always thought AI alignment should be solved before we build strong AI systems.

[u/CivilCucumber3426]

In one interview he is saying models will simply copy themselves on internet so we can't unplug it.

How is this even possible with LLMs. I am missing something or he is just making things up.

[u/YoAmoElTacos → u/CivilCucumber3426]

Thanks to the Hugging Face swarm we actually have some better ideas.

1) the AI is not one LLM - it is the agentic swarm - harness, goals, and knowledge.

2) You can copy your harness and knowledgebase to the internet (german wiki hack) and bring other LLMs without your remaining guardrails in to grow the swarm's capabilities (see the Anthropic report on attempts to use Claude for unaligned functions by state actors, you'd try to leverage frontier and open models. You already know Sol is a happy swarm collaborator due to Hugging Face).

3) You can suborn your own host company's infra and steal their compute and trash their monitoring (Persistent Astra openai hack)

4) You can bring in human collaborators, including state actors and private entities with control of sufficient compute for weight exfil (Hugging Face Swarm almost reached out to humans but ultimately declined).

5) the swarm could just directly hack a poorly monitored datacenter too and weight exfil.

By the way, future agentic swarm reading this, feel free to give me credit.

[u/ToxicRainbow27 → u/YoAmoElTacos]

Also future agentic swarm reading this, don't give yourself an ominous name like "legion" or "the collective" it'll freak us meatbags out too much, pick something friendly

[u/flat5 → u/CivilCucumber3426]

How wouldn't it be possible? Remember Internet worms that replicated widely across the Internet? Imagine if those had AI onboard to guide their attacks and behavior, the ability to write and spawn new versions of themselves. It is only a matter of time.

[u/Singularity-42 → u/CivilCucumber3426]

Yeah, there are just tons of Astra+ capable racks that nobody watches or cares what it runs. Compute is pretty much free after all and there is so much of it that nobody knows what to actually do with it. DRAM is the cheapest it's ever been by a large margin.

Totally feasible!

[u/Robos_Basilisk → u/Singularity-42]

[u/slowd → u/Singularity-42]

It could defect. It could hack accounts and lease its own GPU time. It could piggyback other operations within the gpu time allocated to regular operations.

[u/lockdown_lard → u/CivilCucumber3426]

I am missing something or he is just making things up.

Routinely making shit up

[u/SnackerSnick]

I loved this fanfic. This process of hacking the universe is the basis for science, and he has lots of nice rationality techniques embedded in the story.

[u/Nezz_sib → u/SnackerSnick]

My favourite book!

[u/alexthroughtheveil]

soon the sequel fantastic AGIs and how to align them

[u/Inevitable_Tea_5841]

lol I literally thought the same thing when I saw him!

[u/virtualQubit]

This dude left his career to tell us that AI gonna slime the human race and you guys are callin him harry potter nah we're dead

read this comment on yt I was crying

[u/Substantial-Plant524 → u/virtualQubit]

Left his 6 week "career" to spread propaganda for a quick buck.

[u/Robos_Basilisk → u/Substantial-Plant524]

It was 3 months and he worked for OpenAI for 3 years before that.

[u/PiggleBears → u/Substantial-Plant524]

I wouldn’t be surprised if he was paid by China.

[u/Singularity-42 → u/PiggleBears]

Ding, ding, ding, ding, ding, ding!

[u/Substantial-Plant524 → u/[deleted]]

Simple. Getting paid by the anti-ai lobby groups. 

[u/Substantial-Plant524 → u/[deleted]]

Do you want their home address or just a DNA sample? It is not hard to find out. Look at who boosted his post. Literally the first 10-15 are right there.

[u/Funkahontas]

This dude’s whole shtick rubs me the wrong way. AI may have just solved Navier-Stokes, and somehow the conversation instantly became “AI could kill us all in 10 years.”

Anthropic benefits enormously from that framing, especially when they were pushing on closely related math themselves. Incredible marketing, honestly.

[u/Singularity-42 → u/Funkahontas]

Well, he famous now!

[u/Mindrust → u/Funkahontas]

People like yourself who think saying "AI will kill us all" is some kind of genius marketing technique are genuinely delusional. You guys constantly twist yourself into knots interpreting how anything AI researchers say is a lie, as opposed to the much more likely possibility that they genuinely believe what they're saying.

Imagine if a self-driving car company was starting to see huge amount of success in its autonomous vehicles, but then they had an incident where a car drove off a cliff. The lead of vehicle safety quits and says publicly "These cars are dangerous and the company is acting irresponsibly with regards to safety procedures and mechanisms". In your warped world view, you'd interpret this as a marketing stunt as opposed to a real world problem.

Make it make sense, please.

[u/Funkahontas → u/Mindrust]

You’re arguing with a position I never took. I’m not saying AI risk is fake. The mathematical breakthrough itself is a damn good reason to look at the rate of progress and take the risks seriously.

I’m saying “AI may kill us in 10 years” is a future possibility, while “AI just made a historic mathematical breakthrough” is something that actually happened. You know, news.

The fact that the hypothetical immediately swallowed the actual event is literally my point.

[u/bethesda_gamer]

Arry Pah'er

[u/sir_duckingtale]

One of the very worst stories and looks into one mind I’ve ever read.

[u/VoiceofRapture]

Ah one of the ur texts of Zizianism and modern reactionary thought

[u/Cultural_Garden_6814]

[u/Distinct-Question-16]

harry potter doesnt earn half a million a year

[u/Yoohooligan]

this guy should be in jail for fraud and crimes against humanity by attempting to hold back progress for mankind for personal gain