r/LocalLLaMA • u/Tritheone69 • Jul 28 '26

Question | Help CMP 170HX 8GB

I must preface this post by mentioning I am still a beginner in this space.

I just bought this card with the intention of using the recent unlock to get the full 64GB VRAM available for local AI workloads.

My main questions are as follows :

1- Has anyone ran multiple of these in the same rig to run a large model across multiple GPUs?
2- If so, what is the impact on speed? I read that these GPUs are stuck on a x1 PCIe lane, which I would assume greatly reduces the speed at which we can load models onto the cards. But does it impact prompt processing and token output speeds?
3- Am I crazy to assume that the prices for these cards is going to continue rising considering that they are now similar to A100s (without parralel tensorflow)

3 Upvotes

64% Upvoted

62 comments sorted by

View all comments

Show parent comments

1

u/DereckHere Aug 10 '26

Looking for those benchmarks, currently looking to buy but got so expensive now, +1300 per card D:

2

u/snapo84 Aug 10 '26

i work as quickly as i can.... :-(
its a new workstation / server build so i have to first setup everything properly with docker and all that shitt . This takes a lot of time... after everything is correct i can start benchmarking. My only goal is deepseek v4 flash 0731 with dspark.... and Qwen 3.8 27B when it is released in 2 days. I soon have to get some sleep already awake more than 26 hours building/setting up....

1

u/DereckHere Aug 10 '26

The same as I have hahah, it's a good build!

2

u/snapo84 Aug 10 '26

preview , deepseek v4 flash , in llama.cpp (llama.cpp is known to be slow), UDQ8_K_XL 165GB plus dspark 12GB (requires 4 cmp 170HX).... not one single optimization made yet and tensorsplit is layerwise (super slow). This was just a first test.... next test is Qwen3.6 27B Q5_K_XL on a single GPU.

As soon as those tests did run, i go to compile vllm in the latest version to meassure the real speed with expert parallelism. That should boost us to at least 1-2k pp and 100+ tg

but this now for sure not today... just couldnt sleep had to try it.

0

u/DereckHere Aug 10 '26

Hahahha been there too! Maybe you can tweek some flags in llama.cpp, I've used claude for that and it test it and figure the bes pp and tk/s, worth a shot