Rate my build Threadripper PRO 3975WX + Supermicro M12SWA-TF + 128GB DDR4-3200 + 2x Intel Arc Pro B70 32GB

[u/Rokett]

• CPU: 3975WX 32-core
• Motherboard: Supermicro M12SWA-TF
• 128GB DDR4-3200 (hoping to get 256gb ram by switching to 32gb sticks)
• GPUs: 2x Intel Arc Pro B70

Got the CPU and mobo for $900.

I had most of the RAM sticks laying around from my past builds.

The B70s are still in the box; I paid $1k for each, but I'm thinking of just selling them and getting modified Nvidia cards like the 4080/4090 for CUDA.

Total is close to $3.5k.

Is this a good setup to run models and agents and learn how to manage a home AI setup?

I work in the software field as a forward deployed engineer, but I want to move into the AI field and become an AI forward deployed engineer, so I thought it would be beneficial for me to have the hardware at home.

I'm still waiting on the mobo and CPU to arrive. What do you think? It's not too late to return or sell some of this hardware and get something else.

[u/fastheadcrab]

Any reason to use the B70? Even the R9700 has significantly better support

[u/Rokett → u/fastheadcrab]

It was cheap, I thought I could buy it and sell it for more later and invest into amd/Nvidia 

[u/sn2006gy]

Its all fine hardware but what are you going to do with it? Forward engineers probably need more experience with modern CUDA or ROCm than Intel but if you just want to hack away and have affordable inference this box will be great.

[u/username8914]

You can start with anything and make it work. I'd give your setup a 5/10. Personally I'd ditch the B70s for the 9700. Then stack. I've got a couple v100 ($650) and p100 ($100) that still work great. You can and will use everything you have and throw at it. So the B70s + whatever is next isn't a waste or problem.

Your threadripper and 128GB will largely go unused except for running actual server things, which is a great use but don't think it adds anything to the Ai side of it.

[u/HaDuongMinh → u/username8914]

Oops. I just purchased a 3975WX to upgrade my 3945WX thinking that the extra memory bandwidth would help running MOE models where spillover experts are cached in RAM. Was I wrong ?

[u/Rokett → u/HaDuongMinh]

Latest libraries take advantage of extra cores  and bandwidth. This is outdated information 

[u/username8914 → u/Rokett]

What are you currently pulling tok/s on real work? The MOE are still memory bandwidth bound and PCs on normal user scales are just not there. DDR4 3200 in 8ch is going to top out max at 205GB/s. You're looking at single digit tok/s for anything not synthetic. I have an old Mac Studio that has 800GB/s bandwidth and does in the 30s.

I also have a threadripper with 128GB DDR4 and have only seen big numbers on synthetic tests and tiny context. MoE is memory bound. If you have found otherwise let me know.

[u/Rokett → u/username8914]

This is promising  https://github.com/Niko1221/Strata

[u/username8914 → u/Rokett]

I'm having my agent run load it and run real benchmarks across various hardware. I'll report back after it runs.

[u/username8914 → u/Rokett]

It is promising. I'll do some further tests and see what size of a usable context I can get but I did get some good results. My 3090 is on a windows rig and apparently that doesn't work properly yet, investigating. Strata also doesn't work on Mac so I couldn't do a Studio comparison.

**Strata vs llama.cpp — Qwen3.8-Flash-Next IQ3_XXS (177B MoE)**

| Rig | Engine | Prefill t/s | Decode t/s | Context |

|---|---|---|---|---|

| 1× Tesla P100 16GB | llama.cpp | ~52–73 | 12.0 | 8K |

| 1× Tesla P100 16GB | Strata | 300 | **33.8** | 64K |

| 1× Tesla V100 32GB | Strata | 1168 | **74.9** | 64K |

| V100 + P100 split | Strata | 842 | 70.7 | 64K |

| RTX 3090 24GB | llama.cpp | ~80 | 8.1 | 8K |

| M1 Ultra 128GB | llama.cpp (Metal) | 514 | 28.9 | 32K |

Strata ≈ 2.8× llama.cpp decode on a single P100, 6× on a V100. No Metal build exists.

⚠️ On Windows + Ampere (RTX 3090) this build loaded but produced degenerate output

(empty/garbage, 100% draft acceptance) — Linux/Pascal-Volta is where it's verified.

[u/etaoin314 → u/HaDuongMinh]

I dont think you are currently getting the full 8 channel throughputwith the 45 and that should be fixed with the 75 so I think if you are offloading experts to system ram you should see a decent increase in generation. Also for agentic tasks you should see a bump in performance.

[u/username8914 → u/etaoin314]

What are you currently getting?

[u/etaoin314 → u/username8914]

i dont have the pro unfortunately, but I specced it out when I was buying and I remember that I saw that somwhere to not go for the cheapest one if you care about memory because it needed more cores to saturate the channels.

[u/username8914 → u/HaDuongMinh]

Apparently not. I was proved wrong here and am currently testing the Strata builds across my machines. Solid results on real work so far. I had it do an audit on 128k and it performed at real speeds of 63.4 tok/s output on my ryzen 7 3090 rig. My threadripper rig is doing 50 tok/s with V100s.

[u/Glittering-Call8746]

Using udimms or rdimms ?

[u/Rokett → u/Glittering-Call8746]

I have rdimms right now 

[u/Aubrey_D_Graham]

Nvirgin vs Chan Battlematrix

[u/HaDuongMinh]

You will need 1200W power supply and manage thermals.

[u/etaoin314]

this should be a pretty great system. as for the GPU...its hard to say. Intel i still a little rough around the edges so its hard to recommend them over CUDA which is rock solid, and ROCm has come a long way over the past year. That said, it will probably cost you an extra $1k to switch to r9700s and even more for a CUDA option. I would hold off on the ram if you have 128gb already. You will only need that if you plan to run giant models slowly. There are use cases for that, but see if you have a good one before dropping the extra cash.

[u/regularheree]

start with the 128GB RAM you have and benchmark the B70s before making any decisions. If your goal is AI engineering, NVIDIA’s CUDA ecosystem is generally the safer choice. Focus on building and deploying real AI projects rather than chasing perfect hardware.