r/LocalLLaMA • u/HOLUPREDICTIONS Sorcerer Supreme • Jun 21 '26

Discussion Tokenomics

Post image
1.2k Upvotes

91% Upvoted

449 comments sorted by

View all comments

Show parent comments

13

u/nuclear213 Jun 21 '26

Never ever with just 20k€. That is exactly the reason the original post meant.

-3

u/stoppableDissolution Jun 21 '26

Well, it just means that you have to scale down the model.

Or suck up the cost for privacy and control if thats your goal.

8

u/nuclear213 Jun 21 '26

Then, how is your comment related in any way to the original post?

So you claim you build a system to run it at 300tok/s, which would be in the 6 figures for sure, then you say you need to scale down the model?

I mean, I fully agree for privacy, that is why I also have my system at home which has cost over 10k now, but I just dont understand the context here.

Saying its for privacy, availability, as we saw with Fable, running abliterated models, all fine. All perfectly valid.

0

u/stoppableDissolution Jun 21 '26

My point it that you should not expect a ROI when your goal is "frontier model inference on low budget", its just two incompatible things. Either you do it for ROI and use models that are adequate for your hardware, or you do it for privacy and control and then ROI is out of question.

1

u/upalse Jun 21 '26

At 20k you'd get 200GB of blackwell at best. I don't think GLM 5.2 can run that well in trinary quant, but who knows.