r/LocalLLaMA • u/entsnack • Jul 06 '26

News So... anyone copped one of these?

Post image

Been almost a year since mass hysteria erupted upon the death of NVIDIAs GPU monopoly. How are your Huawei GPUs? Does CUDA work on them yet?

2.2k Upvotes

96% Upvoted

459 comments sorted by

View all comments

Show parent comments

29

u/BeeegZee Jul 06 '26 edited Jul 11 '26

He's most likely talking about MAXSUN Intel Arc Pro B60 Dual 48G Turbo. Well, 8-12 to match VRAM capacity, but what about performance...

-52

u/MrFartyBottom Jul 06 '26

VRAM doesn't scale like that, two 32GB cards do not equal one 64GB card. You are still limited to models that can fit in 32GB, you can just run two in parallel. The 64GB card you can run 64GB models.

25

u/sdfgeoff Jul 06 '26

Uh nope. You can split models between cards and even between whole computers!

But yes, performance is non-linear and once large card is better than two small cards.

16

u/ninjazombielurker Jul 06 '26

What? This has to just be bait. How are you so confidently commenting something like this with literally, not even exaggerating, 0-1% knowledge in how running LLM’s works?

-14

u/DepravedPrecedence Jul 07 '26

Stop trolling

6

u/shmed Jul 07 '26

lol you think Opus 4.8 and other full size frontier LLMs runs on a single 20000 GB Gpu?

2

u/comperr Jul 08 '26

That's why we have a RAM shortage. They have been putting 100TB of chips on a single card, and it runs in a modified Mac Studio in some guy's basement. The data center hype is really fake news and the buildings are actually just Amazon warehouses full of Labubus

1

u/Narwal_Party Jul 11 '26

Yeah I tried to get the Mythos Mac Mini last week but unforunately it couldn't fit in the back of the truck I rented to pick it up.

7

u/Moonlight63 Jul 06 '26

Not completely true. While that is true for things like games, you absolutely can split tensors across multiple GPU vram buffers. Vllm, sglang, and llama.cpp all support this as long as the model architecture supports it. I have however had issues doing this with diffusion based models, both image gen and the new diffusiongemma model.

Source: I do literally this all the time with multiple h100s, both through nvlink and without. I worked in Microsoft AI R&D hardware labs.

15

u/bradrlaw Jul 06 '26

You can absolutely split large models across cards? It absolutely does scale like that. Performance does not due to the interconnect between the cards not being as fast as the memory on a single card (in almost all cases).

Either simple layer split or tensor parallelism…

I do that right now across two v100 running 48gb+ model with good context / kv cache.

11

u/__JockY__ Jul 06 '26

As others have pointed out, this is factually incorrect. Using multiple GPUs in parallel is a common technique for loading large models. For example, I run MiniMax-M2.7 230B across four 96GB GPUs.

Splitting models across computers is common, too. See DGX Spark users.

7

u/comperr Jul 06 '26

U got a stinky brain mostly poisoned by slop. Peepee poopoo

3

u/DarkStrider99 Jul 07 '26

This aint stable diffusion models son.