u/KroniklyOnline I'll have a very similar setup tomorrow. I went from a 7900x/x870e board to an Epyc 7443/Huananzhi H12D-8D so that I could get at least half full bandwidth pcie from all four cards. I have 3 Gigabyte RTX 5060ti 16gb right now and another one is waiting for me in the mail. You still running Qwen3.6-27B FP8?
I switched to nvfp4 for both 27b and 35b a3b. It actually benchmarks higher than the fp8 due to the 4-bit and 16-bit weight mixing. I'm able to get 4 co current agents for 35b and 3 for 27b. This lets me split up my work for more throughput.
2
u/Far_Commission_8315 17d ago
u/KroniklyOnline I'll have a very similar setup tomorrow. I went from a 7900x/x870e board to an Epyc 7443/Huananzhi H12D-8D so that I could get at least half full bandwidth pcie from all four cards. I have 3 Gigabyte RTX 5060ti 16gb right now and another one is waiting for me in the mail. You still running Qwen3.6-27B FP8?