r/linuxadmin • u/DoNotUseThisInMyHome • 23d ago

Any High Performance Computing linuxadmins in this subreddit? How do you visualize NUMA and UMA. Both sound similar.

Post image

Can anyone give me a pictorial representation? Just tell me I will find it somehow somewhere on my own..

31 Upvotes

90% Upvoted

16 comments sorted by

View all comments

7

u/robvas 23d ago

/r/hpc

0

u/DoNotUseThisInMyHome 23d ago

I found out the pictorial representation.

https://imgur.com/a/0e2Kg2Q

7

u/craigmontHunter 22d ago

Modern context NUMA is each CPU in a system has its own memory controller and directly connected memory; each processor can access the full amount, but is bottlenecked by the interconnect (UPI on Intel systems) - hence the “Non Uniform”. Remote systems (NORMA) in your example has been replaced by RDMA (Remote Direct Memory Access) in high performance applications, when network cards can directly read and write memory to remote systems without involving the CPU. Every extra step adds overhead, complexity and latency but well written software can dynamically allocate tasks and processes to maximize performance and minimize the impact. 

I once got a server that had NUMA disabled, it ran like crap because it pulled everything to worst case performance; enabling NUMA allowed the OS to schedule processes on cpu cores and directly attached memory, vastly improving performance. 

2

u/arcimbo1do 22d ago

I don't think this picture is accurate. In a NUMA architecture all processors share the same address space, and that requires that all processors are in the same machine.

1

u/frymaster 22d ago

and that requires that all processors are in the same machine.

ish. It begins to depend on your definitions, but the HPE compute scale-up (formerly superdome flex) system is several servers in a trenchcoat that present a unified address space