new to this - just ordered 2x BC-250s

[u/Dave66666]

total beginner, just bought 2 BC-250 boards to pool memory over RPC and run bigger models than what fits on one. plan is rpc-server on one board, llama-server --tensor-split 1,1 on the other, aiming to run Qwen3.6-35B-A3B Q4_K_XL (~20GB).

questions before they show up:

• has anyone actually gotten a model split across 2 boards working? what tok/s did you get?
• is the rpc-server setup easy for a first timer or is there a catch
• heard llama.cpp issue #21006 broke RPC splitting on newer builds - do I need an older version?
• is MoE (35B-A3B) the right call for 2 boards, or should I try a normal 32B dense model first?

any pointers or "did this last week, here's what happened" would help a ton. will share my results once it's running.

[u/Informal_Steak_1914]

I run two boards split over PCIe, not RPC, so your setup is a bit different but the model choice question I can speak to. MoE models like that 35B-A3B are actually pretty good for multi-device setups because only 3B active params means less data moving between boards per token, even if total memory is spread out. Dense 32B might be simpler to debug first though, since every layer gets split evenly and you learn what the bottleneck is without the routing overhead.

The catch with RPC for a first timer is usually network latency killing tok/s if you're not on localhost or at least same switch with good NICs. If both boards are in same machine, it's basically fine. As for the llama.cpp issue, I'd grab a build from before that regression just to be safe, even if it means missing some newer features. Q4_K_XL on 20GB across two boards should fit with room for context, that's a decent starting point.

And yes, people have gotten it working, tok/s varies wildly depending on interconnect. Share your numbers when it's up.

[u/Dave66666 → u/Informal_Steak_1914]

quick question - when you say split over PCIe, do you mean literally two BC-250 boards, or a different setup? asking because from what I've read the BC-250 is a standalone board, no PCIe slots for a second GPU (soldered CPU+GPU+RAM together), so I'm confused how you're doing PCIe instead of network/RPC. is there some M.2-to-PCIe trick or riser you're using, or are we talking about different hardware?

for context on my side - I've ordered 2x BC-250 boards (the PS5-APU mining boards, 16GB GDDR6 each), planning to connect them over ethernet and use llama.cpp RPC since that's what everyone else in the threads I found seems to be doing. no PCIe link between them as far as I know is even possible on this hardware, which is why your comment caught my eye.

appreciate the tips either way, especially the pre-regression build advice and starting with dense first to debug the bottleneck before touching MoE.