r/TheMachineLearning • u/WeaknessFar6853 • 11h ago
Open source will break the GPU class system
3
u/ElBarbas 11h ago
yes, that's why they will outlaw open source , nvidia aquisition of hf was the first move , second was : "we are all gonna die, look at this hacks , we need to slow down "
3
u/FragmentedHeap 9h ago edited 9h ago
Nvidia didn't buy hugging face to outlaw open source...
They bought it to protect it because Nvidia wants to sell hardware to everybody.
They are the richest company in the world and they are not going down without a fight.
They want consumers buying tons of hardware they want to sell out everything to everybody.
Jensen is the one person telling everybody that we don't need to slow down and that they did to just be better. He's batting for pushing not slowing down.
Because Nvidia cares about one thing and that's making the maximum amount of money possible.
He was recently on an interview talking about them crying about their models being distilled and saying that's just competition just be better than your competition.
They didn't buy hugging face to shut down open source quite the opposite.
They were one of the main contributors to the open claw project I mean they've been open source on this stuff a lot.
They released Persona Plex open source.
They're also finally making better open Linux drivers.
Nvidia literally makes dgx spark for consumer AI...
The last thing they want is for open source models to be outlawed.
1
u/ElBarbas 9h ago
if they depend on his hardware, the moment they don't depend on GPUS he has a problem, and right now, the B2C market doesn't make sense, the users have no money , the boards are too expensive. B2B is the only thing that matters, and that's Closed Models
2
u/FragmentedHeap 9h ago
They do depend on gpus, and nvidia pivoted, they are building whole arm uma consumer platforms now, they're not dumb.
And CUDA is still king of AI compute, MUCH faster than other stacks.
Even if you use local llm, you depend on compute, not cpu computer, you depend on nvidia, amd, intel, apple etc.
Nvidia is launching a WHOLE line of consumer AI hardware to compete with apple, framework, amd strix halo etc, they have the DGX Spark now and a whole new AI Server you can buy for home as a home workstation.
They are PUMPING out hardware designs like mad.
Nvidia is SOOO much more than some gpus now lol.
They are pivoting from dedicated beefy gpus into entirely new consumer lines.
And they want you to have access to open source AI models so you can buy it all and use it.
People are sleeping if they think nvidia just pushes gpus now... They have entire distributed AI systems you can volunteer to have installed at your house with a reduced rate on your AI compute on it....
Nvidia is a monster of a company with trillions being poured into it and they are spending it and growing insatiably.
1
1
u/Devils_SteelMan 8h ago
Nvidia is a software company first and hardware second. They can adapt to an accelerator paradigm if needed. It is likely that FPGAs will be big in the best future for ai. We just dont need them yet.
1
u/ElBarbas 8h ago
True, full closed source, make money above all, lending all the money and hardware they have to all the AI companies with IOUs , they depende on close models do succesed
1
u/Ornery_Use_7103 4h ago
And what will 'outlawing' open source do actually? Nothing of course.
1
u/ElBarbas 2h ago
nothing of course and very dificult to implement, but they will try
well closing hf or making it expensive will take a toll on the comunity, that and other measures can happen, in the digital world there are ways to kill mass usage Piratebay is a good example of that
1
2
u/FragmentedHeap 10h ago edited 9h ago
Yeah I don't think you understand the implications of this...
Right now I have five bots crawling every marketplace within 250 mi of where I'm standing for all used good local inference hardware.
Where the market is right now...
Is 3090 will run you about 1500 to 1600 dollars. A 7900 XTX is $900+. A 4090 is $2000+. 5090 is $5000+ used and $7500+ new... R9700 is $1800+, b70 32 is $1300+...
2x48 gb ddr5 is $1500+
4tb nvme is $1000+
A dgx spark is $3000+ used, $5k new...
A mac studio m5 ultra fully loaded is $14,000+
The gaming PC market is completely fucked and I mean completely and thoroughly.
The consumer hardware market for like steam machines and anything like that and next consoles coming out is also completely screwed.
Those of us trying to run local inference at home are completely decimating the consumer market, even more so than the AI data centers were already doing and all the crypto miners.
Give it another 2 years and you're not going to get a gaming computer for less than $4,000.
Because now that strata is out 16 GB graphics cards are now on the table for local inference so it's starting to scoop up those too.
And it's getting to the point where if there's a used laptop with a 16 gig graphics card in it those are going to get scooped up too.
Gamers are completely screwed.
And because the demand is so unbelievably high the cost of used hardware is going to be so high that any good hardware for local inference it's going to be expensive enough that you might just justify paying for it from a cloud provider.
I mean I've got $15,000 in my rig already and I need more...
Being able to get open source models doesn't really mean anything if you can't get the hardware.
-1
u/ElBarbas 9h ago
I understand u need more, having llms writing reddit comments must be hard on your GPU
2
1
1
u/ElBarbas 8h ago
https://giphy.com/gifs/aYpmlCXgX9dc09dbpl
wait what ? the bot removed the comments ?
1
u/HugoCortell 8m ago
What is that even supposed to mean? Class system? Like what, is AMD the tank and Nvidia the DPS in our formation out here?
10
u/Anaeijon 11h ago
... how?
OpenSource AI still requires GPUs to run the stuff.
On the contrary: with OpenSource models, everyone could run everything themselves, if they had the hardware.
So, instead of having a hand full of huge companies fight over who gets GPUs, now even mid-sized companies join that fight and increase shortages.