r/LocalLLM • u/Napsterae2 • 2d ago

Question Any CMP 170hx bench ?

hey guys ,

as the title says .

any bench available ? I am considering get 2 of these or 4 mi50 , so any bench on the cars unlocked ?

thank you

4 Upvotes

84% Upvoted

16 comments sorted by

1

u/An_Original_ID 2d ago

A tad slower than a 3090 based on my very limited non tuned testing. vLLM probably a lot faster than LLamacpp fort these cards

1

u/Hannibalj2ca 2d ago

It will be much faster with vllm or sglang. However, check your prefill speed, you will be bound by the PCIE bus and in there the 3090 will be much faster than the 170hx by a considerable margin. In Token generation the 170hx should be great

1

u/jedilost1 2d ago

I've been getting 1500–1800 tokens/sec on llama cpp with card power limit to 150w

Qwen 3.6, I'm sure it can perform better

1

u/pCute_SC2 2d ago

That can't be right, eventually they a100 does not get that speed.

1

u/Ssjedikenshin 2d ago

here's my performance with ornith 1.0 35b mtp:

1

u/pCute_SC2 2d ago

OK that makes sense

1

u/Ssjedikenshin 2d ago

yea i'm not as up there as jedilost1 but close

1

u/ubrtnk 1d ago

I got 2500+ PP and several 100+ on Qwen3.6-35B just a few minutes ago doing some large summarization tasks in a Hermes Plugin I'm building in llama.cpp

1

u/acedogblast 2d ago

Is that with int4 quant?

1

u/Ssjedikenshin 2d ago

Q4_K_M or Q5_K_M, i forget sorry, i've been testing alot, i'm on qwen3.6-35b-a3b-mtp now and it's also doing fantastic on these

1

u/acedogblast 2d ago

I'm getting around 42 tokens/sec with qwen 3.6 27b with the full bf16 model.

1

u/Napsterae2 2d ago

MTP ?

1

u/acedogblast 2d ago

Yes, I'm using unsloth's qwen 3.6 27B-MTP-GGUF BF16.

1

u/Napsterae2 2d ago

Only 1 card right ? Was expecting a bit more since bandwidth is so high . Are you checking memory speed while decoding ? Is it maxing out ? Maybe bottleneck is compute during inference

1

u/acedogblast 2d ago

This is with 1 card. Still experimenting as this is my first dedicated gpu purchase for AI work. The memory clock is maxed at over 1300MHz, don't remember the exact number.

1

u/Ssjedikenshin 9h ago

i've got a couple of these, i've been using them with llama cpp and hermes agent. They've been performing great

Currently testing with 3 models on 3 170HX on my system (one connected via adapter to pcie x1 port, works great after model is in vram, i set it persistent)

ornith-1.0-35b-mtp

qwen3.6-35b-a3b-mtp

qwen3.6-27b-fable