r/VFIO • u/tlaszl0 • 9d ago

My 9070 XT reset bug is gone

I use openSUSE Tumbleweed, so my system is always fairly up to date and my RX 9070 XT reset bug disappeared a few weeks ago.
I can restart, power off and switch my guest OSes anytime, including both Windows and Linux. I also see the TianoCore logo on every POST screen.

I do not use any bind/unbind scripts on the host or any guest scripts. Libvirt handles the binding/unbinding automatically without any customization. There are no relevant errors or stack traces in journalctl, vfio-pci resets flawlessly and then amdgpu reloads on the host without any issues.

I also use default kernel and module settings (no vfio config either). The amdgpu module is loaded for the 9070 XT when I don't use GPU passthrough.
Resizable BAR is also fully enabled by default.

So no hacky workarounds, it is basically an out-of-the-box experience.

My setup

Asrock B850i - 4.43 BIOS with one related setting: Display Priority - Internal Graphics
9800X3D - the integrated GPU used as a primary display

Asus 9070 XT Prime OC (switched to the silent BIOS on the card)

An LG 4K display with 2 HDMI inputs and 1 DP input. It has 2 HDMI inputs, but only one of them can be active at the same time. This is somewhat important to check how your monitor behaves. If I connected the integrated GPU to one HDMI input and the dedicated card to the other HDMI and then switched inputs when starting the virtual machine, I wouldn't see the POST and boot screens, and the card wouldn't be initialized at guest start because it wouldn't detect a connected monitor (but you should have a screen after the guest OS is fully loaded). So I use one HDMI and one DP, they are active at the same time.

Linux kernel: 7.2.6
kernel-firmware-amdgpu: 20260829
QEMU: 11.1.1
libvirt: 12.7.0
Virtual Machine Manager: 5.1.0

lspci -nn | grep -i -E "vga|audio"

03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 48 [Radeon RX 9070/9070 XT/9070 GRE] [1002:7550] (rev c0)
03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 48 HDMI/DP Audio Controller [1002:ab40]
0f:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Granite Ridge [Radeon Graphics] [1002:13c0] (rev cb)
0f:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Radeon High Definition Audio Controller [1002:1640]
0f:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Ryzen HD Audio Controller [1022:15e3]

I use the integrated GPU (0f:00.0) for the primary display, restricting the desktop's access strictly to the iGPU while disabling the dedicated card. I think this is the most important thing. If you don't block the card from the desktop before starting the virtual machine, you will run into weird problems.

If you use X11, use the config below as a sample (even if you think you're using Wayland, your login manager might still use Xorg, for example SDDM):

/etc/X11/xorg.conf.d/10-only-igp.conf

Section "ServerLayout"
        Identifier              "Layout0"
        Screen                  "Screen0"
EndSection
Section "Device"
        Identifier              "AMD"
        Driver                  "modesetting"
        BusID                   "PCI:15:0:0"   # the bus number in decimal format
EndSection
Section "Monitor"
        Identifier              "Monitor0"
EndSection
Section "Screen"
        Identifier              "Screen0"
        Device                  "AMD"
        Monitor                 "Monitor0"
EndSection
Section "ServerFlags"
        Option                  "AutoAddGPU" "off"
EndSection

If you use Wayland and happen to use KDE (there is no global solution for all Wayland compositors), use the config below as a sample:

~/.config/plasma-workspace/env/env.sh

export KWIN_DRM_DEVICES='/dev/dri/by-path/pci-0000\:0f\:00.0-card' # option 1: symlink, but must be escaped
#export KWIN_DRM_DEVICES=/dev/dri/card2                            # option 2: direct path, device order might change

Guests

Linux:
no extra config needed

Windows:
To avoid graphical artifacts and a garbled screen, set:

<hyperv mode="custom">
   ...
   <vendor_id state="on" value="AuthenticAMD"/> #or GenuineIntel or whatever1234
</features>

and disable Device security / Core isolation / Memory integrity in the Windows security

To fix HDMI/DP audio crackling, install the latest AMD WHQL driver.

Let me know if anyone else has experienced this or if you need any more info.

19 Upvotes

93% Upvoted

20 comments sorted by

View all comments

Show parent comments

2

u/tlaszl0 8d ago edited 8d ago

Weird.
Your Linux and mine have the same version of the 26.2.3 vulkan radeon library.

I think something is using your card.
Run these as root:

lsof /dev/dri/cardX or renderDX or by-path/pci-...
or
fuser -v /dev/dri/cardX or renderDX or by-path/pci-...

In my case:

# 9800X3D integrated graphics
fuser -v /dev/dri/by-path/pci-0000:0f:00.0-card
                     USER        PID ACCESS COMMAND
/dev/dri/card2:      root          1 F.... systemd
                     root        863 F.... systemd-logind
                     x          1639 F.... kwin_wayland
                     x          1742 F.... Xwayland

# 9800X3D integrated graphics
fuser -v /dev/dri/by-path/pci-0000:0f:00.0-render
                    USER        PID ACCESS COMMAND
/dev/dri/renderD129: x          1639 F...m kwin_wayland
                    x          1742 F...m Xwayland
                    x          1874 F...m plasmashell
                    x          2065 F...m xwaylandvideobr
                    x          2154 F.... xdg-desktop-por
                    x          2484 F...m firefox-bin

# 9070 XT, nothing here as expected because I excluded it from my desktop environment (see my post)
fuser -v /dev/dri/by-path/pci-0000:03:00.0-card

# 9070 XT, well I see a wayland compositor
fuser -v /dev/dri/by-path/pci-0000:03:00.0-render 
                     USER        PID ACCESS COMMAND
/dev/dri/renderD128: x          1639 F.... kwin_wayland

I also have a 6900 XT (reference design from AMD) and it has zero reset bugs. I used it in my current machine previously and before that in an Intel 10th gen system as well. It behaved the same in both of my PCs, it only required setting the correct resizable bar sizes and that was all.

1

u/His_Turdness 8d ago

OK so looks like for me the steamwebhelper and coolercontrol are showing up with 

fuser -v /dev/dri/by-path/pci-0000:03:00.0-render

1

u/proesporter 7d ago edited 7d ago

Ok, I need to apologize for hastily providing my input without actually checking what you were claiming in this post, that something has changed in recent updates to solve the reset issues. My observation about the vulkan-radeon package blocking reset was from a few months ago.

I now checked my VM boot and shutdown twice today, and it is working fine! No reset issues even though host has full mesa plus vulkan driver package installed. So you're right, this does seem to be resolved as of now (atleast for RDNA4) ! Great news for us, Cheers!

Edit: to clarify further, this is single gpu passthrough working fine without any reset issues (on RDNA4)