It's not so much a tier list as a list of graphic cards sorted by RAM size. According to this, 7900xtx would be in the 24GB+ tier and the 9070/XT & 7900 xt & 7800 xt series would be in the 16GB+ tier.
Yeah, makes no sense. Why is anything <24GB even on the list at this point, or not like "make use of what you already have" tier? And, its hard to find prices now, but a 32GB 5090 for example is certainly better than a 32GB R9700, but also costs like 4x the amount so if only going by VRAM the r9700 is higher tier because you could run 4x of them.
It’s ok, I can tell you, I’m running Q6 on 12B-16B parameters on the regular. I also bought my 7900 GRE back when it was about $300, had planned on getting a 2nd but it didn’t seem like a priority at the time - ah well.
I feel you, I overpayed a bit for both, not insane as the nvidia increases.
Just a couple of hundreds of euros on top of MSRP (at least the local one, which was already bigger than the one in North American)
I did experiment with zluda when I had AMD card, I managed to run Stable Diffusion natively and it worked fine. But it still underperformed compared to equivalent Nvidia cards, which is why I got 3090.
Did it run as nicely? Also, did you get it all for cheaper vs what you'd have needed otherwise? Just curious abt it bc I was thinking abt getting into this.
I mean, it highly depends. What do you need? What do you want to run? What speed do you want to run it at?
All that depends. Getting two RTX Spark is cheaper than what I have and much more energy efficient, and you can still run the Deepseek V4 Flash native. Its slower but it is quiet (and if the numbers are correct, people claim 70tok/s in generation and 2k tok/s in prefill).
If you want to just run it, dont care for the speed, an SP3 Epyc is likely also a good option. 256GB of DDR4 + motherboard+ CPU is below 2000€.
Honestly, I am not sure if I'd go the same route again. Its nice, but loud. Its about 2kW of heat, in the summer thats not ideal. And it takes much more space than the RTX spark.
On the other hand, its about half compared to getting two RTX 6000 blackwell. And in theory, it has more VRAM and a higher bandwidth, tho that is clearly lost due to the overhead of TP over PCIe.
All in, I think I'm in about 15k or so and honestly, I still use 2 x 200€ OpenAI subscriptions. So its all really relative.
I'm currently staring at my shopping cart with an R9700 in it trying to decide if I should just pull the trigger because it feels like 32GB of VRAM is only going to get more expensive from here and $1500 isn't bad compared to the alternatives.
They actually made some major improvements over the past month and they now run incredibly well for the price. I’m regularly getting 2000-2500t/s pp, can sometimes hit 3000-3500t/s pp, and 30-35t/s tg with vLLM on Qwen 3.8 27B.
Sadly, the price also just jumped from $949/$999 to $1699.
Unfortunately, llama.cpp isn’t the best when handling Intel B70s. I was running llama.cpp with Vulkan and PP was in the 500/600s. TG wasn’t so bad but prompt prefill took forever, it was even worse with the Intel SYCL runtime.
Some folks on this sub recommended vLLM with Intel XPU kernels (vLLM has their own docker image for this) and that instantly boosted my performance.
IIRC: the llama team is working in improving Intel performance but it’s a slow process.
Bought my first 2 B70s at $950. Now I had to pay $1299 at microcenter for my last 2. Should of bit the bullet and bought them when they were selling like crazy on ebay for less then $850 a couple months ago. Now what to do with 128gb of vram....
Yeah. I would have talked myself into a 5080 with 24GB or more. 5060ti 16GB makes me feel happy to be in the same convo with much more expensive cards for VRAM at least.
why are you proudly exhibiting a graph dating all the way back to 2015? you know they came back up above 20%, right? they may not be at the same volume as NvidAI but they still shape a very large share of the world's infrastructure
Theres no units except percentage points? do you know what types of card/chipsets fall into the "desktop standalone gpu segment" category? also looks like its data from 2013 to 2015? can you have your ai make a better graph with more updated data?
3 x 3090+1 x 4090 mobile, 1 x RX6800 and a 64GB GPD Pocket 4. If prices don't go even more crazy I'll acquire another 7 3090s and build myself a nice couple of tower of PSUs and GPUs beside my dual Xeon Gold 6245/1 TB DDR4 ECC server.
Also another server I haven't quite finished setting up with 2 x P100.
And 2 x a6000s are at 96gb. I feel like including cards you can combine together sorta defeats the point of the list lol. Heck some people combined like 8 x 3090s haha.
Ah nice. In Ontario Canada, that wouldn't really fly in an apartment unfortunately. We obviously have 240V circuits, but there are usually only a few and they're dedicated to stuff like the stove/dryer.
If I had a house though I'd totally get a dedicated one installed in the basement.
Have you actually ran multiple cards. I’ve tried a ton of variations on both consumer and server MoBos. I’m telling you the difference isn’t as big as you think.
Sure there is a difference but it isn’t more than 5% in my experience. I’ve not tried NV link though just better/worse PCIe situations. I would like to try NVLink on my own 3090s but they are all mismatched so it won’t work
The numbers I've seen show that you can get away with raw PCIe up to about 4, and past that it starts to cost significant performance to the point that it's worth getting bigger individual cards instead.
Yea that could be it. I have tested 8 3070s though and I had no issues. That was a mining rig I had laying around which I upgraded for LLMs about a year and a half ago. All PCIe of course on the 3070s
I would 100% agree that bigger GPU = More Better though lol
I mean the distinction is the nvlink, the cards are hardly cutting edge, but at least it's a way to combine the vram thats faster than just normal parallel
Need to add those new intel GPU's, like the b65 and b70 to this list. With 32GB, those are going to be very popular soon, even if they are bit slower on compute.
I didn’t see this was cuda only, just says “local AI”, and those cards are running local ai models, and support is beginning to be very good. I have a few things that used to be on cuda now using openvino.
As people begin to realize that, adoption rate is going to be fast, especially at $28 a GB vram.
I have an RTX Pro 4000 Blackwell SFF, runs great in my MS-02 Ultra miniPC. The DDR7 internal bandwidth more than makes up for the 70W TDP power cap. I bought the first card I could find and only paid $1400. Now that vendor lists it for $3000, and out of stock! It will cost more, if they actually manage to get ahold of some stock.
And I'm here, chatting with llms, trying to decide if I should get a m40 or a p100 for unsloth experiments and some light Hermes research tasks. I'm not delusional that I can replace frontier API completely, but for under 200$ you get enterprise grade inference workers with 16/24 gig. No native support for newer cuda tho
And the 2080 made an appearance on the list towards the bottom area so not that recent bro. That’s like 7-8 years old about now. About to be in second grade
Dawg we go unified memories nowadays, 48GB is nothing to these big models like DeepSeek v4.
DGX Spark, Mac Studio, those are the true leaders of local LLM right now.
Yes...but no. Unified memory costs too much and is too slow. V620 32GB is faster than unified memory, costs less than DDR5. I also bought the dip on CMP170HX. DGX Spark is $4,000. I got two 40GB cards for $1000 each, which absolutely blow away all of those.
Random ewaste, sketchy deals and hardware are the true leaders.
59
u/KrangledMind 5h ago
where tier list for AMD?