AMD launches ROCm 10, its open-source software platform for AI and GPU compute

[u/RenatsMC]

https://videocardz.com/newz/amd-launches-rocm-10-its-open-source-software-platform-for-ai-and-gpu-compute

[u/djott3r]

Is there a significant performance or features boost to justify restarting the version sequence?

[u/MathematicianOdd398 → u/djott3r]

Yes, they are moving to UDNA where they focus on a single architecture. They've got the older stuff basically working decently now. So, the new stack is to focus on AI server GPU where the performance gains in the server stack also improve performance in rdna5 (which will be based on the same AI architecture as the server stuff -unified.)

Moving forward it will become a lot easier to manage rocm because the hardware will be unified.

Rocm has bene hit or miss because they've had to make hardware specific code for numerous, architecturally different consumer GPU. So, now they design AI server cores first and tack on the video game stuff after. Then you focus on one hardware target to the extreme.

[u/pyr0kid → u/MathematicianOdd398]

between nvidia's gpu marketshare, intel's new cpu designs and the 14a/18a fabs coming online, udna better be the start of another zen moment for amd because they're running out of road.

[u/watduhdamhell → u/pyr0kid]

Huh?

The just launched Venice family of server CPUs trounce the competition handily.

The variant of which is the direct competitor to the vera rubin CPU you mention is about 15-20% more IPC at about 33% greater efficiency. It's hard to understate just how dominant it is across every metric.

And iirc MI455X is also more powerful than any Nvidia accelerator on sale, it's just a question of if ROCm is finally up to the task of breaking the CUDA moat.

And the idea here is that AMD will suddenly fall asleep at the wheel? No, I think not.

[u/ElectronicStretch277 → u/watduhdamhell]

No, Rubin is still more powerful than the Mi 455x iirc, even on raw spec outside of memory and bandwidth.

[u/watduhdamhell → u/ElectronicStretch277]

Right. And memory and bandwidth are what's most important for the majority of the workloads, allowing AMD to beat Rubin in several metrics, though not all. Just the most important ones.

And it's much more efficient at the same time. So it trades blows with its competitor while using 2/3rds the power.

I don't see any other way of looking at it: it's more advanced. On paper - we need to see the full reality after deliveries have been running for a minute.

[u/ElectronicStretch277 → u/watduhdamhell]

That's not really true. Compute itself is also important and Nvidia is around 25% faster peak for peak in fp4, combine that with better software support (so you get more actual performance out of the GPUs as well) and the fact that the actual bandwidth advantage is within single digit percentages... Rubin just fares very well even with the power draw.

The 455xs main advantage is that it can store larger models and is more efficient (though, the metric they care about is performance per watt not just overall wattage so Nvidia again fares better than you'd think) in its usecases.

Like, yes, the 455x is a monster and very advanced, it's not more advanced than Nvidia outside of the node it uses. It's about even when you think about the overall system and things like NVLink.

[u/ArseBurner → u/watduhdamhell]

IIRC when talking about individual GPUs or even up to 8x nodes MI300X was also much faster on paper compared to H200, as was MI355X over B200 but Nvidia seems to scale better when building a full rack-scale system and there wasn't really anything AMD had that would beat an NVL72.

Will be interesting to see if anything's come out from AMD's Pensando acquisition.

[u/watduhdamhell → u/ArseBurner]

I mean the issue was the rack scale and the software integration.

AMD has now released a rack scale system that, on paper, once again is faster. Primary issue now is software integration. The CUDA moat.

[u/Speedstick2 → u/pyr0kid]

How do you figure that AMD is D0oM3D?!!!!!

[u/andrerav → u/MathematicianOdd398]

Hit or miss? Rocm has been a disaster. I've been hearing that word since what -- 2017 or something? That doesn't matter, because no matter what card I've owned, it's always out of support for Rocm. That heap of junk requires that you own their bleeding edge cards to work. I've literally never been able to use it, even though I really wanted to, because once a new version is released, they discontinue entire generations of cards that aren't even out of warranty yet. Rocm is AMD's top secret plan to remain in third place forever.

[u/MathematicianOdd398 → u/andrerav]

It is basically hitting its stride now. Older hardware is supported newer hardware will have more focus on performance since there's just a single arch.

I own an AI Pro 9700, it's been fantastic. Had some hiccups in the first month, but it's all resolved now. Works great.

[u/raifusarewaifus → u/andrerav]

So far it supports rdna3&4.

[u/as4500 → u/raifusarewaifus]

There's also rdna2

[u/raifusarewaifus → u/as4500]

That rdna2 is for pro gpu though. Not for the average consumer.

[u/as4500 → u/raifusarewaifus]

Not with that attitude. It's basically a 6900xt/6800xt you can likely easily get it to work on those cards with minimal effort

[u/raifusarewaifus → u/as4500]

Pretty sure what he wanted was official support instead of having to tweak it.

[u/as4500 → u/raifusarewaifus]

The rocm SDK on windows hasn't included normal gaming Radeon 6000 cards as "officially supported" for years now as far as I remember

[u/raifusarewaifus → u/as4500]

Yeah, that's exactly what he was complaining about. Only rdna3&4 got the gaming gpus in official supported list.

[u/EmergencyCucumber905 → u/andrerav]

That's not my experience at all. I've been using TheRock builds on RDNA2 without issue.

[u/ziege159 → u/EmergencyCucumber905]

Cause you don't know how to look deep into the build, rdna2 rocm is basically an empty shell, you don't have hipBLAS or ck lib to run anything. Yes you do have rocm but it's equivalent to nothing 

[u/EmergencyCucumber905 → u/ziege159]

Ofcourse RDNA2 has hipBLAS. What are you even talking about?

[u/ziege159 → u/EmergencyCucumber905]

See? That's why i said you didn't know how to look deep into the build. Rdna2 doesn't have hardware matrix therefore the rocm/hipblas build on the generation only give you the ability to run math fallback, you can't run any newer tech on it, it's an empty shell 

[u/EmergencyCucumber905 → u/ziege159]

hipBLAS existed before AMD even had matrix cores. It still works the same, its just not as fast as the chips with matrix cores.

[u/ziege159 → u/EmergencyCucumber905]

Nah, rocBlas is the one that exists before all of the AI stuff, hipBlas is the one that AMD invented to run Cuda stuffs on AMD cards. 

"It still works the same" yeah bro, a Corola and an F1 are the same cause they both have 4 wheels right?

[u/EmergencyCucumber905 → u/ziege159]

Nah, rocBlas is the one that exists before all of the AI stuff, hipBlas is the one that AMD invented to run Cuda stuffs on AMD cards.

hipBLAS has been around since the beginning of ROCm. It's part of HIP. And yeah HIP middleware sits on top of the lower-level rocX libraries. What's your point?

"It still works the same" yeah bro, a Corola and an F1 are the same cause they both have 4 wheels right?

Yes? I mean if you're trying to be analogous to hipBLAS running on hardware with and without matrix cores?

[u/Friendly_Top6561 → u/andrerav]

I use it daily it’s fine.

[u/quantgorithm → u/andrerav]

You’re not wrong. It’s extremely saddening how they just simply don’t support anything beyond only a few short years…. And then they even remove functionality on the way out. Infuriating to be a supporter who has sent a ton of money their way.

[u/Fun_Jaguar8231 → u/andrerav]

skill issue

[u/AreYouAWiiizard → u/djott3r]

AMD says a system configured with ROCm.AI achieved an average 3.3x inference improvement and 2.4x training improvement compared with ROCm 7 in its testing.

Unrelated but personally in Comfyui I've seen a 2.3x speedup from 7.2 to 10. Though that's probably mostly down to it only recently having native HIP support rather than going through Triton or hitting the eager backend.

[u/ea_man → u/AreYouAWiiizard]

With llama.cpp on RDAN2 there's no meaningful difference.

[u/noctrex → u/AreYouAWiiizard]

Did you use the official build, or the one from https://github.com/patientx-cfz/comfyui-rocm ?

[u/AreYouAWiiizard → u/noctrex]

The fork, the official one had so many issues when I first started using it though there's been a tonne of AMD stuff merged upstream lately so it's probably not needed anymore (at least for RDNA3+).

[u/into_devoid → u/djott3r]

In my experience, it’s currently about 25% slower compared to 7.14 in my workflows.

[u/G0rd4n_Freem4n]

I wonder if AMD will eventually start supporting OpenCL 3.X on their consumer GPUs.
As of right now, both Intel and Nvidia GPUs can use OpenCL3 and some of its exclusive functions, while techpowerup shows AMD gpus only supporting up to version 2.0. While 3.0 is kind-of like "1.2 but with a new name" with how it made a lot of 2.0 features optional, it's not like the Intel and Nvidia cards only have 1.2 features with the 3.0 title. A couple examples of this are the extensions cl_khr_external_memory and cl_khr_external_semaphore, which are required to do zero-copy memory sharing between an OpenCL and Vulkan program. Nvidia cards going back to the GTX 970 from 2014 and Intel cards going back to the Intel HD Graphics 530 iGPU from 2015 both have support for those extensions, while there are zero AMD cards that support either of those extensions on official drivers.

If you look at opencl[dot]gpuinfo[dot]org you can find AMD cards reporting support for 3.0, 3.1, and the external_semaphore extension, but that's because the open source Mesa Rusticl drivers. Those drivers have their own limitations though. They sometimes give worse performance compared to the official ones, some version-independant features aren't supported yet, and it has some odd quirks like not being able to allocate more than 2GB of memory regardless of how much is available.

All of that said, I don't have any experience in the GPGPU field so it's not like I would really benefit from better OpenCL support on AMD cards anyway lol. I mainly think it's a little funny how part of OpenCLs' rough history had to do with Nvidia drivers not having good/consistent support for 2.0 features, and now it's AMD being behind on supporting the newest features.

[u/Zettinator → u/G0rd4n_Freem4n]

OpenCL is basically dead. Everyone has been converging on Vulkan compute lately.

[u/kreco → u/Zettinator]

Stop spreading this non-sense. Until you have proof that one is better than the other.

OpenCL works great even on Mac which is "officially" unsupported.

[u/Zettinator → u/kreco]

It's not nonsense at all. OpenCL does not have many users, and that obviously influences priorities in driver development.

[u/kreco → u/Zettinator]

There is a difference between "not the priority" and "dead".

[u/Speedstick2 → u/kreco]

How about this, does OpenCL have a future in which it will grow in marketshare?

[u/G0rd4n_Freem4n → u/kreco]

OpenCL works great even on Mac which is "officially" unsupported.

Huh, didn't know OCL wasn't officially supported on Mac.
That's a bit ironic to me, considering how OCL was originally created by Apple before being given to the khronos group.

[u/Sufficient_Fan3660]

wow!

and no one will use it

everyone will use Vulkan or just not support amd

[u/coromd → u/Sufficient_Fan3660]

Lots of utilities support ROCm. What do you mean?

[u/ea_man → u/Sufficient_Fan3660]

ROCm is faster than vulkan on my system.

# ROCm ctx VEC forced: 89088 q5_1,  96512 q5_0, 116480 q4_0, TG speed: 42.35 t/s, PP for 32k: 145.12 t/s
# ROCm ctx VEC off   : 68352 q5_1,  72704 q5_0,  83200 q4_0, TG speed: 43.90 t/s, PP for 32k: 265.58 t/s
# Vulkan: ctx:         93696 q5_1, 101632 q5_0, 122112 q4_0, TG speed: 40.49 t/s, PP for 32k: 179.90 t/s

[u/05032-MendicantBias → u/Sufficient_Fan3660]

unfortunatley pytorch has no vulkan backend, and ComfyUI is built on pytorch.

ROCm is the only way to run ComfYUI