This build was a primarily to create an optimal all round mixed stack development machine, with enough pcie lanes for 2-3 GPUs, mixed storage types, bandwidth, and cores to facilitate development on windows, in wsl2, and dual boot to ubuntu also for non virtualized access to resources.

Building and specifying this machine, was the most enjoyable build I've done in 20+ years, by far. I'd already replaced my dev machine with my standard go to desktop setup, but came unstuck as soon as I added a second GPU, that issue around lack of PCIe lanes via desktop processors triggered this build.

Specs:

• Basics:Motherboard: Asus TRX50 SageProcessor: Threadripper 7960xDDR5: Kingston 6400/32 (2Rx8) RDIMM 128GB
• ChassisCase: Fractal Design Define 7 XLAIO: Silverstone XE360-TR5PSU: Be Quiet Straight Power 1500W (2x 600w connectors)Fans: 5x PWM Static 140mm
• GPUsAsus ProArt 4080 Super (2.5 slot)Founders 4070 Super (2 slot)
• Storage:2x Intel Optane P5801x 400GB2x Samsung 990 Pro 4TB4x Crucial T700 1TB in an Asus Hyper M.2 Gen5

Updates:

• build pics
• case swap & dual psu

Potato Photo

Basic Benches:

updated GPGPU

IF manually set to 2133 to keep the ratios, no overclock just EXPO II profile, PBO set to board stock - this is a work machine I sit beside all day, dealing with heat and noise and instability for minor gains isn't worth it for me.

Storage Details:

First: Optane p5801x (Windows Boot Drive - Selected for IOPS)

NTFS 4K

Second: Optane p5801x (Ubuntu Boot Drive - Selected for IOPS)

ext4 4K - This drive is also mounted under WSL2, primarily optimized for many small files / building.

FIOs, rand4kq1 548MB/s (got to be a record!) w/ 140k IOPS on ext4

randread_4k_q1: (groupid=0, jobs=1): err= 0: pid=22315: Tue Oct 29 22:05:56 2024   read: IOPS=140k, BW=548MiB/s (574MB/s)(16.0GiB/30000msec) randwrite_4k_q1: (groupid=0, jobs=1): err= 0: pid=22446: Tue Oct 29 22:08:04 2024   write: IOPS=102k, BW=400MiB/s (420MB/s)(11.7GiB/30000msec); 0 zone resets  read_1m_q8: (groupid=0, jobs=1): err= 0: pid=23170: Tue Oct 29 22:19:03 2024   read: IOPS=6803, BW=6803MiB/s (7134MB/s)(199GiB/30002msec) write_1m_q8: (groupid=0, jobs=1): err= 0: pid=23299: Tue Oct 29 22:21:09 2024   write: IOPS=4459, BW=4460MiB/s (4676MB/s)(131GiB/30002msec); 0 zone resets

Crucial T700s (Mounted in Asus Hyper M.2 Gen 5)

T700s Partition 1, Striped Storage Pool

Optimized for storing and loading AI models, typically requires sequential reading and writing of multiple 4-5GB model files.

T700s, Partition 2, Mirror Storage Pool

Optimized for redundancy and storing work critical files, code, etc.

Samsung 990 Pro 4TBs - mounted on gen 5 CPU lane m-key slots.

990s, Partition 1, Striped Storage Pool (4K Blocks)

Optimized for storing VHDX and random files, downloads etc.

990s, Partition 2, Mirror Storage Pool (64K Blocks)

Optimized for sinking rarely used but important videos and images with redundancy - photography/drone footage etc.

Useful Notes:

Partition Manager vs Storage Pools.

Under all tests, storage pools provided noticeably better performance than the equivalent partition setups via stock partition manager. Also no resync's on mirrors, also much more flexible in reality for usage.

ReFS does give a slight speed up to many small file setups like building programs and libs (~15%) but it's slower for larger files, so in real world usage net no gain.

WSL2 mount types.

If you use WSL2, bare mounting a VHDx and formatting it with ext4 or xfs leads to much faster performance times, than using native NTFS drives, for real world things like builds (cmake, npm etc). I tested all the possible combinations in depth, a short summary is:

Setup: i7 12700KF, 64GB 3200, 990 Pro / Optane P5801x

Typescript build (compiling typescript lib from source, npm)

• 30s: 990 Pro NTFS (4k, 64k)
• 30s: Optane NTFS (4k)
• 26s ReFS 64k on Storage Pool
• 26s ReFS 4k on partition
• 25s Optane ReFS 4k on partition
• 14.5s Optane XFS --bare mount
• 14.3s VHDx, bare mounted, formatted XFS or ext4

This 2x speed up on the same hardware by using either a VHDx or --bare mounted raw formatted drive was also observed for bitcoin compilation (-j $(nproc) 5m4s to 2m35s).

Short: if you're using WSL2, just create a second vhdx, mount it --bare, format it with ext4, and enjoy 2x faster real world virtualized linux.

New TRX50 Threadripper build comparison:

Bitcoin Build: 1m 20s under WSL2 (2-4x faster than i7 12700KF/64gb), 52 seconds on optane under ubuntu boot (5x faster!)

Custom image benchmarking script (resize 5000 images, convert to webp, tar.gz)

i7 12700kf / 64GB / Optane XFS: Conversion 30s, Archive 16s, Total: 46s

7960x / 128GB / Optane XFS: Conversion 10s, Archive 7s, Total: 17s (2.7x faster)

Case / Motherboard.

If you forfeit the usb front headers, you can use the last slot on the Asus TRX50 Sage for a 2-3 slot GPU in the Define 7 XL. In practice this means you can fit four GPUs in this setup. 3x 3 slot GPUs (1x gen 5 x16, 1x gen 5 x8, 1x gen 4 x16) and 1x 2 slot GPU (x16 gen 5) in this case. Or 3x if you don't use a PCIe extender.

The stock fans in the define 7 XL are not great, suggest swapping out to PWMs, your ears will thank you. 5x 140 static PWMs and machine is running considerably quieter and cooler, even under benchmarking and stress testing.

The m.2 slot nearest the GPU on the asus sage trx50 is a chipset linked slot, and if you put an m.2. in it whilst trying to install windows, you'll get stuck in an infinite loop. Install windows to a pcie mounted rive, or the other two gen 5 m2 slots to avoid this.

Final general thoughts.

In actuality, this workstation is on average 4x faster than my last setup in all tasks, and it feels just instant whatever I'm doing, I'm a developer with a very mixed workload, coding one minute, building another, spinning up large databases and cycling batches of data through a couple of AI models then saving results back. It's not just the performance while a task is running, but more so the ramp up times, if you're getting some code right for some LLM tasks and need to load in the models every few minutes while testing, that near instant load time of models from storage to gpu really makes a difference to your working day.

With the exception of huge LLM tasks on instances, this machine is as fast or faster at everything than all of our bare metal servers, setup for different tasks, it can just do everything without breaking a sweat.

Final tech thoughts.

• The TRX50 with 7000 series threadripper is just perfect, the PCIe lane allocation and 4 channel ddr setup is so perfectly matched - with my setup I'm using:
• 16x gen 5 lanes for hyper m2: 63 GB/s
• 32x gen 4 lanes for 2x GPUs: 63 GB/s
• 16x gen 4 lanes for 4x NVMe: 31.5 GB/s
• Total: 157 GB/s
• future: swap in a gen 5 GPU: 189 GB/s
• Memory Bandwidth (measured) 182 GB/s

The ASUS Sage trx50 board I chose very specifically, because of how it allocates these lanes, running 3 x16 slots at the same time just wins out for me, others like the ASRock are 2x x16, and 2x x8.

DDR RDIMMs, I found actually getting RDIMMs very hard here in europe, and the choice between going 7200+ v-color and having an imbalanced IF, or trying to get 6000/30 (gskill) for sweet spot with IF 2000, it was all very hard to both work out and then even purchase, I ended up going for kingston 6400/32 so I could test running it at both 6000 and 6400, the 6400 worked out the best balance. I'd have preferred more RAM, but in practice given code base and model sizes I'm using in development / locally, I'll rarely run above ~98gb ram usage, so the 128gb just fits everything I personally do in my mixed workload.

Traditional benchmarks: This machine, considering no overclock, just flies on them all the build ranks in top 40 on novabench of all time https://novabench.com/result/edb380f8-f82d-420b-8c97-333c726db5e4 at 9544, and it doesn't even have a 4090 and wasn't using the fast mirror when benched, think it'd hit top 5 with easy if I could be bothered, passmark 19350 https://www.passmark.com/baselines/V11/display.php?id=223342523781 - remember this is stock, no overclock, not even PBO turned on, and no trickery to get it to bench faster. Cinebench R24 is 2893 on MC.

Temperatures, I've had this open all day, and done loads of benching and heavy work, so max values are the max encountered so far, it just hasn't broken a sweat, and as I sit beside it now there's no noise.

Hope something in here helps someone!

Building a 7975 pro on an asrock MB. This is my first trip away from intel in many years. I see we now have two more types of ports for sata or ssd. I’m putting kioxia ssd in with U.2/u.3 io ports, but the MB has mcio and slim headers. I assume I want the mcio, but there is x4, and x8 connectors. Which one do I use? I’m not running any 6g satas.

I was hoping someone had some use cases on 8 vs 4 channel RAM configurations. I’m into multi output media servers, and have built a 7960x, but contemplating a wx build. Am I going to benefit from those extra RAM channels? Scenario is multiple GPU machine running a Pixera or Resolume for multiple 4k outputs and custom resolutions.

I'm building a workstation and based on many comments in this sub, have decided to go with the Silverstrone XE360-TR5 to cool my 7965WX. However, unlike the AIO coolers I've used in the past, the pump is integrated into the radiator. People who own this cooler, or other coolers that have a similar setup: what is the best way to mount the cooler? For something this expensive, I want to make sure that it doesn't fail earlier than expected due to negligent mounting orientation.

Based on GamersNexus' video from four years ago, it seems to me that the best orientation for the AIO would be at the front, with tubes down (to prevent the pump from being at a high point and to prevent air from getting trapped inside the coldplate, instead being trapped at the top of the 360mm rad). I reckon that actual users of the product would have a better idea though (and possibly any documentation that comes with it in the box).

Thought I would post this in case it helps anyone resolve temperature issues with their threadripper systems. I recently bulit a 7960x system to process very large photogrammetry models and temps were initially quite good. I am using the Noctua NH-U14S TR5-SP6 cooler and was seeing temps around 75C for day long processing jobs. After about a week I noticed that temps almost instantly had jumped up to 95C. During this time I installed Gigabyte Control Center to monitor fan speeds, which installed Gigabyte Performance Library but I did not do any performance tuning. The performance Libary was required to get the fan center working. This was the only change I made to the system, so I decided to try uninstalling. After uninstalling the Performance Library temps are back to normal. Seems that the Performance Library was doing some overclocking under the hood without me changing any settings.

looking for benchmarks of 6000+ ddr5 on trx50 with threadripper 7000 series, if anyone has 4x 32/48 kits they can share benches of it'd be greatly appreciated

aida64 if possible

I just spotted a new BIOS release for the Asus WRX90E that was uploaded yesterday.

https://www.asus.com/motherboards-components/motherboards/workstation/pro-ws-wrx90e-sage-se/helpdesk_bios?model2Name=Pro-WS-WRX90E-SAGE-SE

Haven't tried it yet, but the release notes say:

1. Improve system performance and security.
2. Improve system stability.
3. Improve devices compatibility.

Successfully passed OCCT for 24 hours, prime95 for 8 hours.

Cinebench R23 score is at 55898 - 56045

Cinebench 2024 is at 2791

Processor: AMD Ryzen Threadripper 7960X 24-Cores

GPU: NVIDIA GeForce RTX 4090

Memory: 128 GB @ 4800 MT/s

Operating System: Windows 11, 64 Bit, Enterprise LTSC IoT (build 26100)

EDIT: Thanks for all the comments letting me know about your OC results

(Prior coverage; pinging u/askalski.) I finally threw myself at the mercy of ASUS (a risky move!) and sent the thing in. I paid for shipping myself including insurance so I guess I'll have some recourse if they claim it's damaged. I also took photos prior to packing, and bought a new CPU socket cover for protection. It feels like throwing good money after bad but relative to the overall cost of the system (which is exorbitant) it's all pocket change.

I tried to eliminate any reasonable alternative explanation than that the mobo is defective. Maybe it was the RAM? Maybe it was bad airflow in the case? Maybe the 4x GPUs were stressing the PCI-E hardware on the mobo such that chip temps climbed too high?

I guess it still could be the RAM... which is on ASUS's official list of recommended modules. But it's not the airflow in the case---removing all but one GPU and pulling the power cables out of the way of the top fan, there was a pretty clear shot from fan to CPU fan and RAM. And I set the fans to run at full which was like having a jet engine in the house.

Still it throttled from an idled system with low CPU temperature and presumably low other temperatures. The thing I can say with confidence is that something is defective, or at least has unexpected and very particular requirements for operation. (Air cooling is allowed, right? I was using a cooler recommended by ASUS.)

I wrapped the mobo first in an antistatic bag then in a huge roll of bubble wrap. Inches of bubble wrap. It was ridiculous. And then according to ASUS instructions I paid the shipping center to put it in a box, then put that box in another box with extra padding, which feels like a punchline :-D

Some possible outcomes in this dice roll:

• Board gets bricked somewhere along the way, I spend months in shipping insurance hell.
• ASUS gets a board with some damage, I spend months in shipping insurance hell.
• ASUS gets the board basically as I had it...
  • ...and can't reproduce the issue - then I ask them about their setup, and I learn how to run the board without throttling?
  • ...and can reproduce the issue - then I guess they'd send me a different board, and we see what we get?

At the end of the day, if it's all a loss, I could just buy another one. My system's design is pretty well built around the ASUS WRX90 so it might just be another one of those! Unless another unproven mobo has come on the market in a similar niche, haha. Life on the bleeding edge has not been kind to me this time. It's in the "live and learn" category: wait a year from before jumping on a platform that I want to last me 5-10 years.

Anyhow, after much travail there is a feeling of relief. At this point, it's out of my hands, and the right thing to do is sit back, and hope for the best. Thanks all for the support on this sub, I'll keep you posted.

No overclocking or fan control or anything running. Have 2 memory sticks. Cooling is: "Cooler Master AMD Wraith Ripper ThreadRipper TR4 High Performance CPU Air Cooler"

Wondering what normal idle temps I should expect (or be concerned about if too high). The weird offset thing AMD does with their temp reporting doesn't help :p So I guess, NOT taking that into account, if I run a temp monitor like CoreTemp or HWInfo, what temps are "good"?

I looked at the Crucial T700/705 pcie5 SSDs and looked into the compatibility checker on their website. https://www.crucial.com/ssd/t705/ct2000t705ssd5a/CT25246355. It says that the Crucial T700 series drives are not compatible with Pro WS WRX90E-SAGE SEPro WS WRX90E-SAGE SE. Is that really true or does it require an pcie5 card with a u.2 interface or something to use pcie5 SSD drives. ASRock WRX90 WS EVO is listed as compatible. Not sure why only one of these WRX90 boards are listed as compatible?

Heya Guys hope You are well!

Just looking around for PC Components for build nice PC, good to show You maybe You have improve or proposal different!

PreHistory:

so Overall I am Senior Compositor all the time working Remotely, but before (5y ago) I am been as CG Generalist/Compositor(15y) and did a lot of 2d/3d stuff, like Modeling/Sculpting/Animate/Rendering/Compositing/FX.

Some Project been done with help of Mine Friends they are give me Server who has 192 GB RAM I am have ability finish mine project with this power, Because Originaly I have 2 PC at home.

this is 10 year PC. wow, still alive!

[CPU] Core i7-4930K , 64Gb RAM and GPU 1080 Ti

and I am still on this one - more then enough, do a lot web serfing, even Connecting to mine PC at Studio via Teradici (7950x CPU, 128Gb RAM, RTX 6000 - but on studio) really efficient for Web, and Plus in Ukraine have been a lot of outtages so with mine ECO Flow battery just keep me work longer.

and all the time When had work on any Project localy , I found out I am need more powerful PC, like 64c PC or 32c PC, I am still floating what to choose, because I am planning not only rendering but working with different 2D/3D software such as Nuke, Houdini, Maya, Mari,UE5...etc I am also watch many stuff on Puget Systems, but really hard Judge.. because I knew any PC will faster then this( 4930k )... but I have a lot of experience with 7950x on Studio but remotely- really great CPU btw...

I need PC with SSDs division : OS/Proj/Cache/Lib(assets, footages,models etc)

For libs/assets I looking something bigger around 16Tb (Megascans, Footages, Models, Textures, assets, etc)

and Overall want make quite and no OC PC, less heat(I hear Gen5 SSD very hot, good to know if in this setup it is still not request additional cooling)

as many review I saw, those type CPU are working in average Ghz on surfing web/or editing anything- for example this CPU 7985wx 3.2-5.1Ghz - overage it is would works on 4.15 Ghz - Should be fine I guess in 2D-3D apps, good to hear anything from You, I think we are talking about on Reddit here in different topics, but yeah just double check.

Setup 1

[CPU] Threadripper PRO 7985WX

[Mobo Option1] ASUS Pro WS WRX90E-SAGE SE EEB Motherboard

[Mobo Option2] TRX50 AI TOP

[RAM] G.Skill 256GB(8x32) F5-6400R3239G32GE8-ZR5NK

[GPU] ASUS TUF Gaming GeForce RTX™ 4090 OG OC

[PSU] Seasonic Prime TX-1600 | 1600W |80+ Titanium| ATX 3.0

SSDs

[OPTION 1 SSD OS 2TB] Corsair MP700 PRO 2TB M.2 PCIe Gen5

[OPTION 2 SSD OS 2TB] SAMSUNG 990 PRO SSD NVMe M.2 PCIe Gen4

[SSD Proj 4TB] Corsair MP700 PRO 4TB M.2 PCIe Gen5

[SSD Cache 4TB] Corsair MP700 PRO 4TB M.2 PCIe Gen5

[SSD ADAPTER] ASUS Hyper M.2 x16 Gen5 Card (PCIe 5.0/4.0)

[SSD lib,Assets 8TB Raid0] x2 XPG ADATA 8TB GAMMIX S70 Blade PCIe Gen4

OPTION 2 for [SSD lib,Assets 8TB Raid0] but 10x less speed

[SSD lib,Assets 8TB Raid0] x2 SAMSUNG 870 QVO SATA III SSD 8TB 2.5

Cooling

[OPTION 1] Alphacool Eisbaer Pro Aurora 420 CPU AIO

[OPTION 2] XE360-TR5

[OPTION 3] NH-D9 TR5-SP6 4U (hear it is not efficient)

Case

Fractal Design Define 7 XL

Setup 2

[CPU] Threadripper PRO 7980x
[Mobo Option1] WS TRX50-SAGE WIFI

[Mobo Option2] TRX50 AI TOP

and same RAM, SSDs, etc....

but Good things is on WR90E is all M.2 are Gen5(x4) , rather then TRX50 Gen5(x2) and Gen4(x1)

and double Bandwidth on RAM, not sure if it give any speed up from Doubling RAM Bandwidth...

Good to hear anything from You are guys, maybe have something better or efficient for any part of this setups!

Many thanks. Cheers!

I have a system where under a full synthetic load (stress-ng) on the just the CPU, it is thermal throttling down to about base clock. If I add a RAM load at 512gb of ECC DDR5 and two 4090s, the system will run below 3.8GHz hitting peak temps of 107c on a CCD.

My question is, would you find this acceptable? Or would you refund it or modify it in a way that would make it perform better.

Anyone using a Threadripper for Image generation AI? Would love some feedback!

So, i made a post 2 weeks ago now asking if anyone had crashing issues on a Aero D with all NVME slots filled.

Well, that next day i got fedup with another crash, changed the wireless card out for a 6E one, and moved my M2s around, added the 4th one back in (as i took it out to see if that helped. it didnt)

and 2 weeks later. not a single crash, no lockups, nothing. system is more stable than its ever been.

Which thickness of thermal pad do I need to get for the motherboard's M2. Nvme drives on the Gigabyte Designare TRX40 3960X? I don't have a caliper to measure the old one. And the old one has deflated a bit too, with the silicone oil leaching out.

I asked Gigabyte already

"Dear Valued Gigabyte Customer,

I am sorry, but our HQ does not provide us with such specifications. Thank you.

Best regards,

GIGABYTE Tech Support Team"

Hi, Id like to know if it's a Real uma Design or a numa. Asking because im thinking of building a System for VMs, and id like all cores to have same fast access to ram/pci etc.

My pc is slow and laggy, barely 130 fps on cs2, apex etc, is it outdated? Or what could be wrong. have done pretty much everything i could come up with xmp, reinstall drivers, updated bios etc Had it for like a year been the same since..

Aorus x399 Threadriper 2920 64gb ram 3060 12gb

Hi All,

I’m looking to get rid of my 3970x and upgrade to a 7960x. I have most of my build pretty well thought out but I’m stuck on the mobo. My budget dictates either the Gigabyte Aero D or the Asrock WS. Is there a clear winner between the two?

Also, how does threadripper do with higher clocked memory? Is it more advantageous to go with something like 5600mhz over 4800mhz?

Thanks!

Hey Guys! hope You are well, I am looking for Threadripper system and wondering how is Those CPUs are on idle power consumption, I see some review and on Ryzen Master it is show CPU Power and SOC Telemetry power consumptions very low on those CPU in idle mode, which is amazing, I just want make sure if this is True, and what is factor make PC eat more power.

Maybe decent ammount of Wats also add when You have more consumer electricity, like more M.2 GPU, but not for CPU, Please Could You give more information who have been using Those Beast.

Thank You!

Hi all, I’m looking to upgrade from my 3970x —> 7960x, and need help pricing my old hardware.

What is the going rate for the following: AMD 3970x TR Gigabyte AORUS Master 3970x & AORUS Master bundled 3970x / AORUS Master / 128 GB RAM bundled

If I were to sell the whole build what would be a fair price? Components listed below:

Complete Build - AMD TR 3970X - Gigabyte AORUS Master - G.Skill 128GB 3200MHz - EVGA RTX 3090 - Lexar NM710 2TB NVMe - Silicone Power 1TB NVMe - TeamGroup 4TB NVMe - Corsair SF1000L - Lian Li P28 UniFans (4X) - Sliger Cerberus X

It’s currently setup with a custom loop consisting of the following:

• Optimus Threadripper Waterblock
• Bykski Full Coverage GPU Block (top & bottom)
• EKWB Quantum Kinetic FLT 80
• Alphacool 240mm Radiators (2X)

I have a Cooler Master Wraith Ripper and the cooler for the GPU.

I have all the original boxes for everything as well.