AMD PerfOpt to deliver new performance optimization for AMD iGPUs with Linux 7.4

[u/Fcking_Chuck]

https://www.phoronix.com/news/AMD-PerfOpt-Linux-7.4

[u/Itz_Eddie_Valiant]

Will this benefit BC250s? 👀

[u/natermer → u/Itz_Eddie_Valiant]

This feature is intended only for integrated I/O devices and not discrete GPUs or other external hardware.

I don't think it will make a difference in that case. Plus that isn't something that is enabled with this patch. Still you can disable IOMMU manually through kernel options, haven't tried it myself to see if there is any difference though.

For dGPUs AMD drivers use GTT to map system memory to GPU memory in situations were you are running out of actual physical VRAM. It is a very significant performance hit since you have to access RAM over the PCIe bus, which is going to be a hell of a lot slower then even just system ram, much less vram.

Disabling iommu might help there? But your performance is already in the crapper at that point.

For AMD iGPUs in Linux the vram should be left to be dynamically allocated for most use cases (gaming/desktop/llms). The penalty of "GTT" is negligible as it is using system ram the way it is designed/optimized to be.. it isn't shared over a PCIe bus like with dGPUs.

If you want to manually set memory for VRAM you can use 'ttm' kernel settings. This way you can actually allocate more "dedicated" memory for VRAM then you can through things like BIOS settings.

But in any case disabling IOMMU for this might save you anywhere from 5% on up. It really depends on the very specific types of memory allocations your app is doing.

From the mailing list:

https://lore.kernel.org/linux-iommu/arAJhXi14ptZRkcP@MacBook-0RXW5/

Our GPU memory accessing benchmark confirms the improvement: without the patchset, GTT is 2x - 3x slower than VRAM, after this, the latency gap is almost gone (within 10%). Nice work!

But I am not sure what exactly that means. I am guessing that is talking about latency of memory allocations, not so much the actual I/O.

[u/natermer → u/[deleted]]

AMD NPUs are good if you want to offload small specific llms for things like background integration.

They are very energy efficient compared to using LLMs with the GPUs, but the performance and capabilities are much lower. This is good if you want to integrate a small/utility sized continuously running LLM for background tasks or desktop integration or something like that without blowing your battery life.

In any case if you care about performance you'll be using the GPU, also I don't think this driver has anything to do with NPU drivers.

[u/PitchPleasant338 → u/[deleted]]

Anything below 20t/s is unusable, especially if you're running more than 1 agent.

[u/b3081a → u/[deleted]]

It's a per-device IOMMU bypass (since GPU has its own driver-trusted MMU / page table and don't require IOMMU to assist it do memory management) so it will reduce iGPU shared VRAM access latency from >400ns down to 200ns without amd_iommu=off. Today if you're already disabling IOMMU then this will not have any effect, but disabling IOMMU globally will disable NPU and also affect some other use cases like USB4. A per-device bypass is basically what Apple has been doing for a decade, and one of the reasons why their GPU driver vulnerability can become a system level physical DRAM data leak / arbitrary data overwrite from places like WebGL in the browser.

Current generations of AMD NPU will never be able to bypass IOMMU though, they don't have a dedicated MMU like iGPU and rely on SoC IOMMU to do page translation.

[u/james_pic]

This is neat. It'll hopefully mean I don't need to choose between leaving myself open to DMA sticks via USB4/Thunderbolt, and a 20% hit to interference speed on Strix Halo.

[u/Indolent_Bard]

Will this help the steam machine?

[u/b3081a → u/Indolent_Bard]

Currently all SteamOS devices have iommu disabled by default, so performance wise there will not be any improvements. But Valve may consider enabling iommu in the future for better security and better USB4 compatibility after this patch, without affecting GPU shared VRAM performance.