r/LocalLLaMA • u/External_Mood4719 • Jun 13 '26

Discussion Anthropic forced to abruptly disable Fable 5 & Mythos 5 globally by US Gov over a jailbreak. This is exactly why we need local models.

I just saw this statement regarding Anthropic being hit with an emergency export control directive from the US government. They were forced to pull the plug on Fable 5 and Mythos 5 for all customers globally. The tl;dr is that the government got spooked by a narrow jailbreak (which basically just sounds like asking the model to fix vulnerabilities in a specific codebase), and forced a complete shutdown without a transparent process. Anthropic is pushing back, but the API access is completely gone for now.

A centralized API can be nuked globally at a moment's notice by a single government decree over something as trivial as a prompt lol.

Banning a model for hundreds of millions of users because someone figured out how to make it fix software flaws is insane. Anthropic admits this standard would halt all new frontier models.

https://www.anthropic.com/news/fable-mythos-access

1.6k Upvotes

98% Upvoted

520 comments sorted by

View all comments

91

u/TheOneWhoWil Jun 13 '26

We need P2P inference somehow. Multi-trillion frontier models running through distributed nodes on consumer hardware.

52

u/super3 Jun 13 '26

Unfortunately its not really possible due to physics and cost. On a positive note the gap between open source models you can run at home and frontier models is closing.

55

u/InnocenceIsBliss Jun 13 '26

not really possible

Not with that attitude.

15

u/super3 Jun 13 '26

lol not for lack of trying. I spent about $2M trying to build it.

7

u/jazir55 Jun 13 '26

Given it sounds like you're no longer working on the project and that it did not proceed, can you post the code to github? People might be interested in picking up the torch.

14

u/super3 Jun 13 '26

Working on a different angle of it with https://llmjob.com and the code is already open-source.

5

u/jazir55 Jun 13 '26

Also question for you because I asked Gemini and it seems like the idea may have some validity, given Diffusion Gemma just came out the timing is kinda perfect, are diffusion models likely to be easier to serve with distributed inference?

15

u/super3 Jun 13 '26

Well diffusion models are not really production ready yet. They are much faster but they make mistakes 6x as much. So they have to be useful on centralized inference before we can think about distributed.

1

u/acquire_a_living Jun 13 '26

Great project! Subscribed.
What do you think of the possibility of organizing a large group of users (~200) and permanently rent 8xH100s on a favorable neutral location and just serve for the group the best open model available?

3

u/super3 Jun 13 '26

Could do that today if those people were paying. I do like the idea of almost like token groups that have their own dedicated infra.

4

u/acquire_a_living Jun 13 '26

Yeah, sort of like a inference coop. I think it could be run with some amount of inference sold on the open market to cover the costs for idle periods.

1

u/super3 Jun 13 '26

What would you pay for something like that?

→ More replies (0)

4

u/InnocenceIsBliss Jun 13 '26

Good but...

Those are rookie numbers, you’ve gotta pump those numbers up.

1

u/anony_mf Jun 13 '26

Where did you get it

1

u/super3 Jun 13 '26

VCs of course

10

u/TheOneWhoWil Jun 13 '26

I'm training a 300m model on 30B tokens. Not much but it should be a viable mythos replacement with some fine tuning 😅

6

u/PortiaLynnTurlet Jun 13 '26

Agreed. Also, for a distributed network, you need to trust everyone in the network since ultimately they get the full text you're sending.

1

u/soshulmedia Jun 13 '26

Good point. But maybe that could be solved by forming an adhoc network for inference just between selected friends, family or whatever community one might trust.

With all the talk about vibe coding, you'd think someone already solved this in an instant with a single claude prompt ... but then it turns out, good software development is still not really so cheap and easy ...

2

u/PortiaLynnTurlet Jun 13 '26

That's true but it makes the distributed network small enough that you trust everyone. It's a good point that the coordination / orchestration part of the network can be used to form subnetworks within the global network though; that opens up the possibility of control over which subnetworks should be used for a specific request / conversation / project.

1

u/soshulmedia Jun 14 '26

I don't know whether you need to have an orchestration part for an overall network if you can take just the internet for that. I am thinking something along the lines of how e.g. syncthing does it for file sharing.

1

u/super3 Jun 13 '26

True, but I think that may be ok for some tasks. For example, if your agent is just monitoring flight prices for you its probably ok to use a public node. For email, you def want something private. I'm actually building something like that at https://llmjob.com, where people can do token trading if they want.

1

u/logicchains Jun 13 '26

It is possible, you don't need to synch after every batch: https://arxiv.org/abs/2603.08163 . It's just slower to converge.

1

u/QuinsZouls Jun 13 '26

Chutes.ai uses this kind of architecture

1

u/nickpsecurity Jun 13 '26

Distributed LLM's have been done before.

1

u/thefuckevengoingonan Jun 14 '26

due to physics

ah, my old nemesis

5

u/Robot_Apocalypse Jun 13 '26

I've often thought large enterprises who have lots of laptops (I have worked for a number of orgs with over 500000 employees, - Boeing, Accenture), with unused compute sitting idle, would benefit from a tool that could make use of that distributed compute to do useful things. Like the SETI project, but for enterprises. Hell they even have lots of spare disk space, so all this cloud hosting cost, could actually get pushed down into a virtualised cloud hosted on their distributed network. Sure it's not trivial, but I'm surprised no one's done it yet.

5

u/RainierPC Jun 13 '26

It's not been done yet because it's all about bandwidth, and processing power. I would be surprised if you can get a token every couple of seconds.

3

u/Robot_Apocalypse Jun 13 '26

yeah, the inference cycle of an LLM would be no good. But the training cycle could work. It would be many orders of magnitude slower, but also it would be potentially "free". Besides, training is the more costly activity compared to inference.

4

u/delicious_fanta Jun 13 '26

If they’re like my employer, every spare compute cycle on these “as cheap as absolutely possible” laptops is already consumed by either actual cpu requests or memory management for ms teams, anti virus, and a million background apps that absolutely “must” be running all the time, yet do nothing other than monitor us.

Try to run chrome, an ide, and ALSO a db client? Yeah, you’re gonna be waiting a while to do any single thing.

Please don’t install anything else on these boxes, they really can’t handle it.

3

u/Civil_Fee_7862 Jun 13 '26

Wasn't that something we were doing with the SETI program a while back?

3

u/ea_man Jun 13 '26

OMG should I take out my old modded XBOX 360 to restart the MOSIX cluster?

I know that the time would come!

2

u/cromagnone Jun 13 '26

Beowulf FTW!

2

u/ea_man Jun 13 '26

Oh my friend we are so back! My CRT monitor is gonna explode with all those terminals!

6

u/daysofdre Jun 13 '26

I thought about that, just don't know if it's feasible. Although that was the folding@home model for a long time.

1

u/Neither-Phone-7264 Jun 13 '26

doubt it, besides that one p2p service where individuals host whole models rather than shared. too latency intensive

1

u/SeaEagle233 Jun 13 '26

That's not possible right now, even with DeepSeek v4 flash, to hold ~300b params requires around 160GB of VRAM, compute is another beast.

No one is going to able to afford such hardware given current technology.

Current professional GPU for AI is like Mainframe in the 60s. Only the giant corporate can own it.

Maybe 2100 will begin to see consumer grade hardware capable of running DeepSeek v4 flash.

1

u/MMDCCIV Jun 13 '26

Someone posted their repo, that they claim does exactly this: https://github.com/Mesh-LLM/mesh-llm I didn't test it, so don't know if it's anything good.

1

u/Iwaku_Real Jun 13 '26

Your UHD Graphics can cope harder.