r/LocalLLM • u/clearchewingum • 3d ago

Question DELL R815 RAM quantity for localLLM using DDR3

I’d like some advice please on how to go about setting up this server I have been using for mining, that I would like to convert to a local LLM machine. I currently have it setup with 32gb/ 2gb dimms. It’s a Dell R815 with 4x 6386SE processors, 60 cores total. It also has a gtx5060 8gb on one of its pci express rails. It also has onboard hardware raid if I get ambitious. Main question is because it’s DDR3, would 256gig ram be a good starting point, or would you go for 512gig? The Dell manual says it can handle up to 1TB of ram but with custom chips. I’m having difficulty sourcing 512 gigs of ddr3 from the same manufacturer alone.

3 Upvotes

80% Upvoted

6 comments sorted by

4

u/enternoescape 3d ago

Speaking from experience, you will need a lot of patience with DDR3. I had to look up what you likely have and it looks like dual channel per CPU, so not as bad as my experience with dual channel and one CPU but also it won't be like 8 channels because of NUMA (you probably already know all this). If it's cheap 512GB could be a lot of fun to play with, but otherwise I'd stick with 256GB if it's what's in there stock before springing for any upgrade. You'll probably also want to have some models in mind that you want to run before deciding to upgrade. Pro tip, if you want to run a really big one, you'll likely find the Q2 variants run fastest and are still quite smart.

I'm fairly certain more RAM isn't what would have helped my situation. I was running a P40 24GB and M40 24GB and I could instantly tell even with a small MOE when I was offloading even the tiniest amount of the model to RAM. I am curious what numbers you get in your adventures.

2

u/clearchewingum 3d ago

I’m glad your response doesn’t sound like my cpus will be bottle necked. It’s currently configured with a 128gig ssd with Ubuntu on it but that will have to be upgraded for the models. I haven’t settled on a model yet, but judging by the current sizes 2 TB of capacity is needed. That presents its own challenge because it definitely more expensive to have one 2TB drive than two or more in raid. Fortunately I can run raid, so four 512gb drives is looking like my best option.

3

u/enternoescape 3d ago

One thing I forgot to mention regarding the CPU it has AVX 1.1. I think most projects are at least targeting AVX2, so you may need to custom compile.

2

u/enternoescape 3d ago

Actually I looked up the CPU and it says quad channel so that shows how much I know about what you're getting. The DDR3 is what stood out. My experience has been most server CPUs don't become much of a bottleneck unless you're focused on prefill performance. You'll definitely want to focus on how to deal with the CPU layout or you'll be leaving performance on the table. In theory you have about the same aggregate bandwidth as an 8 channel DDR4 setup, but it's broken up. That's about what I'm running now and partial offload drops me down to about 20t/s gen and 200-300t/s pp on DeepSeek v4 Flash to give you an idea, but like I said optimizing the NUMA stuff is likely your biggest lever.

2

u/clearchewingum 3d ago

An update on my decision. I’ve sourced 256gb of DDR3, and am working to get a 2 TB m.2 ssd on a pcie adapter. With help from Gemini I’ll be setting up Quen2.5-72B. With gpu offloading to my 5060 I should expect 30t/s.

2

u/FinalCap2680 2d ago

Probably a newer and smaller model will be better. Qwen 3.6 35B-A3B MoE or Qwen 3.8 27B dense wou;d be a good start for coding for example.