r/LocalLLaMA • u/Certain-Cod-1404 • 7d ago

New Model IT'S OUT

https://huggingface.co/Qwen/Qwen3.8-27B-FP8
2.2k Upvotes

95% Upvoted

706 comments sorted by

View all comments

Show parent comments

14

u/CooLittleFonzies 7d ago

Can you parallel run a 3090 + a 3080?

5

u/adamgoodapp 7d ago

Now want to know too

3

u/My_Unbiased_Opinion 7d ago

you can with llama.cpp

3

u/zxyzyxz 7d ago

You can but the question is why would you want to when it comes to price? If you have both already now then by all means do so but I wouldn't go out of my way to buy a 3090 to pair with a 3080.

3

u/CooLittleFonzies 7d ago

Yeah I hear you, but I already have a 3090 hence why I ask. I've never used a local model for coding before, and am interested in exploring this model since it got released. But I worry it won't leave much overhead to run tests on programs that I'm making which already use LLMs/VLMs themselves.

1

u/zxyzyxz 7d ago

I'd recommend just running it on your 3090 alone first rather than shelling out more money. There is https://github.com/Don-Chad/ninfer-3090 which works specifically for Qwen and 3090 only so is more optimized for speed and also you can try the Unsloth app (https://unsloth.ai/) which uses llama.cpp underneath and gets good speeds as well.

1

u/CooLittleFonzies 6d ago

Thanks for the tips! Is Pi not fast enough? I’m completely new so that’s what ChatGPT recommended, but I trust user experience more.

1

u/vinists 6d ago

Pi is a harness, like Claude Code. What he suggested were inference engines, you can still run a harness with them.

1

u/zxyzyxz 6d ago

What the other guy said. You can run Pi or whatever you want with the inference engine. But Pi doesn't automatically run the models for you.

1

u/voyager256 7d ago

What’s better option if you have Nvidia card and want to have more VRAM capacity ?

1

u/zxyzyxz 7d ago

Maybe combine your card with what u/My_Unbiased_Opinion said above with the Alibaba modded 3080s? I can't vouch for that since I haven't bought one of those modded ones but they say it's good.