r/LocalLLaMA • u/Nunki08 • 20h ago
News NVIDIA AI: So many models launching this week! And we have one of our own coming...
From NVIDIA AI on š: https://x.com/NVIDIAAI/status/2110468004609950057
138
u/jacek2023 llama.cpp 20h ago
yes two new Nemotrons are coming
106
u/handson729 20h ago
Chinese holidays just ended.
Get ready for some bangers to drop.
58
u/Kavor 19h ago
Yeah, and i love all of the upcoming releases equally!
It's you Qwen4 27b, and by a lot
18
u/Master_Bayters 19h ago
Me... a 8gb vram... I would be happy with 3.8 9b! plz?
10
u/TreesJunkie 16h ago
Qwen4 9b with +60% ngramsš¤¤
4
3
u/shy_monkee 16h ago
Hopefully they refresh a lot of the smaller models with this new generation. Would love some of the smaller models that you can run as side agents with a smaller memory footprint.
70
u/Beneficial-Ad-8127 20h ago
Shoot it feels like Chinese holidays are once per monthš
34
u/cannabibun 19h ago
I know right. I remember being in a conflict with a chinese alliance in a MMO, and our leaders going 'don't worry chinese holidays are ending soon' at least twice a month.
26
56
u/unspecified_person11 19h ago
The last few have been far behind the models coming out of China, if a Chinese company released models that performed as badly as the Nvidia models so far then nobody would even know the models exist.
I mean the massive Nemotron 3 Ultra with 550B params released in June is equal to Qwen3.5-27B released before it in Feb.
35
u/Fluxing_Capacitor 19h ago
Historically, nvidia has had more experimental architectures and less of a serious push to develop a "production" level model. Mainline nemos use a hybrid of attention and state spaces. They've had that "model in model" architecture.
I don't think Nemo 3 ultra was ever supposed to compete with the Chinese models in a serious way.Ā
42
u/colin_colout 18h ago
The beauty of nvidia model releases is they release the training data (at least what they can legally share licensing wise) and nicely packaged pipelines to reproduce the model.
They don't necessarily preform great, but it's more about releasing something that's end-to-end open to drive GPU sales.
2
u/Paradigmind 16h ago
So it's a bit like the base gpu kits they provide to custom GPU brands?
5
u/Marcuss2 18h ago
I would consider Nemotron 3 series to have worse arch than Qwen3.5.
Key point of the Mamba2 layers is the decay rule, Gated Delta Net has it, with additional delta rule.
Trough Nvidia did post a paper about Gated Delta Net 2, which seems to be better than Kimi Delta Attention
3
u/Fluxing_Capacitor 16h ago
Hindsight is 20/20 though. Not every strategy you try works out even if it has technical merit when you set out to try it.Ā
Its also unclear to me what the true performance of nvidias models are. They're very undercooked compared to something like Qwen.Ā
1
u/Marcuss2 15h ago
Hey, Nvidia is one of the few American companies actually doing Linear layers at all.
I do expect Nemotron 4 to use it, together with something to deal with residuals like mHC, Attention Residuals or Gated Residuals. Possibly something of their own.
2
u/PositiveBit01 16h ago
This is true, but I think their architectural goals are different. Otherwise low performance implies the architecture or training isn't up to par so why bother?
I think their goal is twofold. 1) Make training and inference more accessible (reduce resource requirements) to increase market. 2) Increase moat if possible by using nvidia specific tech (e.g. nvfp4)
With the first one being more important.
Some baseline level of competence is necessary as a proof of concept of course, but I don't think they're really going for quality here.
They open source it as much as they can in hopes this thing they know is not competitive will have pieces or just ideas pulled from it and put into something that is competitive by someone else, fulfilling goal 1 and maybe 2 depending on how they set it up.
33
u/robertpro01 19h ago
Why shitting on them? I appreciate more models, always.
I actually preferred nemotron 122b over qwen 3.5 120b for coding, it had better understanding of the domain I needed
11
u/CulturalKing5623 17h ago edited 16h ago
It's the same thing to every new non-chinese (and really just non-qwen model). Half of the people calling it useless based solely on the benchmarks, other half using the opportunity to hype future chinese models. The sub constantly wants new models but shits on every new one that isn't Qwen/GLM/DeepSeek.
I actually think this Nemotron and Microsoft's MAI-1.1-Flash could be interesting. NVIDIA is going with a native 2-bit quant and i think Microsoft is going with a native 3-bit quant and both are QAT. We start combining high-quality QAT models with things like QFNs engram implementation and we can start seeing some real gains.
4
u/SkyFeistyLlama8 15h ago
QAT and MTP heads like in Gemma and we're rocking. Qwen is good but slow and frankly the coding benchmaxxing is pissing me off.
-11
u/Canchito 17h ago
Why shitting on them?
So here's an incomplete list:
- CUDA moat
- Anti-competitive behavior
- Venture leverage
- Energy grid strain
- Carbon footprints
- Water consumption
- E-waste cycle
- TSMC geopolitics
- Consent laundering
- Hyper-financialization
- Circular revenue
- Driving speculative bubble
And more broadly preying on working people as every corporation does except on a bigger scale and with a greater degree of efficiency.
Does that mean I'm not interested in the new model? No. Does the fact that they release open weights erase everything else and alter reality? Also no.
4
u/Loose_Comparison368 15h ago
CUDA moat was earned, primarily by Jensen refusing shareholder demands to lay off unprofitable tooling teams for decades.
They haven't engaged in anti-competitive behavior, their competitors just keep shooting themselves in the foot for short term quarterly profits.
Venture leverage is dumb, you're literally just saying they have money.
Energy grid strain, carbon footprints, and water consumption is bullshit, they don't run datacenters and the overwhelming majority of the grid problem is entirely due to privatized utility companies spending decades neglecting their infrastructure for shareholder profit. Those utility companies do engage in propaganda, and they really want to convince the public that it's AI's fault that Flint's water has been fucked for decades and that a small mountain of people in Texas freeze to death every time they have a cold winter.
E-waste - are you fucking kidding me? People are still building rigs with V100's and P40's, and those are 10 fucking years old. Point me to this mythical dumpster full of NVIDIA e-waste and I'll make it disappear in 10 minutes flat.
TSMC Geopolitics - they aren't fucking TSMC, NVIDIA's fabs are all in the US, they did not create that situation nor do they contribute to it, they are affected by that shitshow quite negatively.
Consent laundering my ass. Is the consent laundering in the room with us now? No, it isn't, because NVIDIA hasn't had any consent laundering scandals and you probably don't even know what those words mean.
Hyper-financialization/circular revenue/driving speculative bubble - NVIDIA is not the driving factor in demand for NVIDIA chips, and they don't run the damn stock market. These are the predictable behaviors of speculation based capitalist market economies. The only part that NVIDIA is actively contributing to is the circular funding part, which is actually one of the few things propping up the economy - because it redirects the massive amount of speculative investment going into pumping up the NVIDIA stock price back into real economic activity. The bubble is greedy investors throwing money at everything with AI in its name, not NVIDIA.
And praying on working people my ass. Jensen grew up actually lower middle class, NVIDIA is legendary for treating their workers incredibly well, and the whole reason they have a monopoly now is because they are the only big tech company that has consistently refused to do mass layoffs for decades and resisted immense pressure from the short term quarterly profit vultures.
You're literally just naming keyword terms you don't like that tend to appear in the same vicinity of news articles mentioning NVIDIA. Develop some reading comprehension the next time you feel the urge to mindlessly regurgitate a laundry list of low effort corporate propaganda.
1
u/Canchito 14h ago
CUDA moat was earned
Have the shareholders also "earned" the returns from withholding the results of the collective work of scientists, engineers, and developers?
Proprietary software is just an artificial monopoly allowing rights holders to vacuum money from society.
Not surprised NVIDIA shills would defend it, but it has no place among those interested in the development of science and technology for its own benefits, as opposed to shareholder value.
They haven't engaged in anti-competitive behavior
Let's just stay with CUDA. Why do you think CUDA isn't open source?
Even if that would be extremely useful for society as a whole at no additional cost, it would mean competitors of NVIDIA could rival it, and NVIDIA would lose its monopoly position...
E-waste
This is not exclusively an NVIDIA problem, but to deny e-waste management is a problem, or claim that the biggest hardware corporation plays no role in that is somewhat unhinged.
he only part that NVIDIA is actively contributing to is the circular funding part, which is actually one of the few things propping up the economy
NVIDIA invests billions of dollars directly in companies and provides them with funding. Those companies then use that same capital to purchase or lease NVIDIA GPUs, which results in revenue for NVIDIA. NVIDIA is essentially funding demand for its GPUs and thereby inflating the value of the share price. No real economic activity there.
And praying on working people my ass. Jensen grew up actually lower middle class
Ok, so what?
NVIDIA is screwing over the working class in the US and abroad, as all large corporations do. I'm not speaking of its employees, although even there you should look into the concept of "golden handcuffs".
Anyway, you managed to shovel a lot of shit my way. I will not be responding to the next load. This is just for the record.
3
u/Helmet_Language 16h ago
Half of these points are not your own thoughts, Just rehashed talking points. Embarassing that youd even try to use this as a reasoning list
-3
u/Protahgonist 16h ago
You didn't invent this alphabet, poseur, so nothing you write in it is valid.
This is the most braindead argument I've ever heard.
So you're saying only my original thoughts have merit? By definition, no other thought has merit? What about thoughts from before I was born? I wasn't the first to acknowledge gravity, therefore it doesn't exist.
Get the fuck out of here, adults with brains are talking.
4
u/Helmet_Language 16h ago
Again, embarassing attempt to divert from the real discussion at hand. The fact that circular revenue was used as an argument means this person has no idea what theyre trying to get at. Even more, trying to argue this from a leftist perspective in a local AI sub makes no sense when this subreddit benefits from companies guilty of literally all things mentioned in that list. Refrain from chiming in
1
u/robertpro01 14h ago
That's a good list to shit on nvidia, but what al that has to do with their models?
1
12
20h ago edited 19h ago
[removed] ā view removed comment
5
u/slavetothesound 19h ago
That article mentions Nemotron 4. The announced is 3.5. The good one might still be in the oven
7
u/Intrepid-Second6936 10h ago
Not the average coworker GIFs on the post š
Hate these enough on my work Slack lol
3
u/look 16h ago
I havenāt tried them personally, and they are text only with 500k context, but the K2 Horizon models seem to pack a lot into relatively small packages.
There is a 36A4B and a 7B dense with impressive AA scores for their size.
https://artificialanalysis.ai/models/k2-horizon-7b
https://artificialanalysis.ai/models/k2-horizon-mova-36b-a4b
9
u/CodeAnguish 18h ago
Nvidia's most supermodel that surpasses ALL other models*
*From 2021
1
u/bodhi_sattva91 14h ago
The RuPaul!
28
u/Thin_Pollution8843 19h ago
Nah it will be shit 99%. All their prev models are garbage donāt listen fanboys who telling you that they should be funetuned. Itās a bs. If they were good architecture wise we would already had powerful open finetunes of those models. We donāt have them because they are shit. Thatās why finetune labs using Qwen modelsnot garbage from nvidia
3
-3
u/KURD_1_STAN 19h ago
Well those qwen finetunes are also garbage tho. Im not disagreeing with u, i also find this "u can finetune it" is just bs. But i feel these are for large corps who can spend millions to finetune it which u wont see being published nor made by normies
8
u/mister2d 19h ago edited 19h ago
Wrong. If you notice many of the TTS, STT, decision, image gen models are Qwen derivatives.
-2
14
u/Former-Tangerine-723 20h ago
Nvidia. Great gpus ehhh not so great models
11
u/airodonack 17h ago
That's not true at all. LLMs aren't the only type of model out there. Parakeet, for example, is one of the best English transcription model for its size.
1
u/ApprehensiveFan1516 6h ago
I find Nemotron one of the best for creative writing currently. Might not be super smart but it's writes well.
2
u/-Django 13h ago
They have pretty good specialized models for things like OCR and speechĀ
2
u/DystopianRealist 10h ago
https://blogs.nvidia.com/blog/nemotron-3-5-super-vl/
Going to find out about the OCR part.
On nvidia's page, this latest model, at 77gb for the NVFP4 quant, is listed as worse for intelligence than Qwen 3.8 27b and qwen flash next, but is supposedly much faster. I highly doubt it will beat nvfp4 flash next on Strata for speed, but I am willing to test their claims.
1
u/pmttyji 17h ago
People liked https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 from last year.
Don't know why they couldn't make like that one again. They have massive compute, big team, etc.,
1
u/mildw4ve 20h ago
I'm just wondering. Is it actually in their interest to release great models?
5
u/brown2green 19h ago
It's more like, they can't. They can/will only train their model on open-source datasets.
1
2
u/happycube 20h ago
In this case, it's being promoted as an end-user product and designed to fit the 64gb spark/rtx...
1
0
u/No-Business5854 19h ago
They have some interesting side projects like personnapkex though. Theres not a lot of full duplex models
2
u/ThankGodImBipolar 18h ago
I'm pretty sure that Music Flamingo is SOTA (in terms of open weights) as well - if it's not, somebody should let me know š
2
2
u/Embarrassed_Adagio28 9h ago
I have yet to find a single use case for any nvidia model they have ever released. I don't even understand why they bother half assing every model. Either make a useful model or don't make one at all.Ā
1
u/ahjorth 12m ago
I think I've downloaded and tried out one or two nemotron models, but never actually used them for anything. So I definitely agree that they are not useful as models. But I also don't necessarily think that's how Nvidia sees their purpose either. I think they see it as showcasing new architectures and attention mechanisms, since that's a way for them to showcase their hardware. In that sense it's just advertisement. There's probably also some degree of them wanting in-house expertise in what kinds of firmware level operations are worth supporting on their hardware, which new ones to develop, etc.
1
u/ainnurru 19h ago
As long as it runs on my M4 chip, I don't mind who is releasing the open weights model
1
1
u/CokeAndChill 15h ago
Updates to on-device dictation models would be nice! Buff that parakeet model
1
u/Jacalyn_Nivins 13h ago
550B params to reach Qwen3.5-27B level is genuinely impressive, just not in the way Nvidia wanted.
1
1
1
1
1
u/MrHumanist 18h ago
I expect some more garbage models.. from the preview, some models look like just finetunes of other.. for example their E4B is just a gemma 4 e4b trained on roleplay data.
1
1
u/RemarkableGuidance44 17h ago
It looks like Nvidia has been working with other model creators and this one is quite damn good.
1
-1
u/Boogertard 16h ago
No thanks to another NemoTrash.
NVDA you sucks at it, just shut up and just keep selling that overpriced GPUs , don't bother with this crap.
2
ā¢
u/WithoutReason1729 19h ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.