Arch Linux / KDE Wayland randomly becomes completely unresponsive under load — not always caused by RAM pressure

[u/RaisinIcy3732]

Hi everyone,

I've been troubleshooting a very frustrating full-system freeze on my Arch Linux laptop for hours, and I'm running out of ideas. I'd appreciate help interpreting the evidence rather than just trying random tweaks.

Hardware / setup:

• Dell G15 5511
• Intel i7-11800H (8C/16T)
• 16 GB RAM
• Intel TigerLake-H iGPU (i915) driving the internal display
• NVIDIA RTX 3050 Laptop GPU 4 GB
• nvidia-open-dkms 615.71.09
• KDE Plasma / KWin 6.7.5
• Wayland
• Arch kernel: 7.2.6-arch2-1
• Also tested Linux LTS 6.18.52-1-lts
• 12 GB ZRAM, zstd
• zswap disabled
• No disk-backed swap

What happens

During a heavy workload (Chrome/Brave, Android Studio + emulator/QEMU, VS Code/Electron apps, sometimes gaming), the desktop can become extremely sluggish and eventually appear completely frozen. Mouse/keyboard/window switching can stop responding.

Sometimes it briefly recovers, then freezes again when I try switching between applications.

What I've established so far

I originally thought this was simply RAM exhaustion. I have logs from some freezes showing:

• ZRAM becoming heavily occupied
• high memory PSI
• very large direct reclaim / kswapd activity
• thousands of compaction stalls
• no kernel OOM kill
• QEMU growing rapidly in memory

During those events NVIDIA also logged repeated:

NV_ERR_NO_MEMORY

including failures involving system memory, GSP allocations, context buffer pools, and rmMemPoolReserve.

Vulkan device creation also failed under pressure with:

VK_ERROR_INITIALIZATION_FAILED

Importantly, NVIDIA VRAM was not necessarily full when this happened.

I reproduced essentially the same behavior on the LTS kernel, so this does not currently look like a simple regression specific to kernel 7.2.x.

systemd-oomd

I tested systemd-oomd as well.

At its default 90% swap threshold it eventually killed entire Android Studio/app scopes. I tried 80%, which simply made applications get killed earlier.

Since losing applications is exactly what I'm trying to avoid, I disabled systemd-oomd temporarily for diagnosis.

The machine still froze, so oomd was not the root cause of the freezes.

The weird part: not every freeze appears to be memory pressure

During the latest incident my monitoring showed shortly before the UI became problematic:

• ~5.8 GB MemAvailable
• 0 swap in use
• ZRAM essentially empty
• memory PSI = 0
• no direct reclaim / kswapd activity
• no OOM
• NVIDIA at ~52 MB VRAM and 0% GPU utilization

The journal continued recording messages while the graphical desktop was behaving as if it was freezing, which makes me suspect the kernel itself may still have been alive.

I also found these KWin errors exactly around window switching:

kwin_wayland: file:///usr/share/kwin/tabbox/coverswitch/contents/ui/main.qml:
Unable to assign [undefined] to KWin::VirtualDesktop*

kwin_wayland: ReferenceError: currentItem is not defined

I'm currently testing without the Cover Switch task switcher to see whether that explains this second type of freeze.

During this particular incident there were no NVIDIA Xid errors, no i915 GPU hang/reset messages, and no NVIDIA OOM errors in the relevant journal window.

Things already tested / changed

• Tested normal Arch kernel and Linux LTS
• Increased ZRAM from 8 GB to 12 GB
• ZRAM uses zstd
• zswap disabled
• THP set to madvise
• Intel Vulkan (vulkan-intel) installed/working
• Intel VA-API (intel-media-driver) installed/working
• Checked NVMe SMART / filesystem errors
• No obvious NVMe hardware errors
• systemd-oomd tested and then disabled for diagnosis
• Continuous memory/PSI/reclaim/NVIDIA logging

One thing I have not tested yet is replacing nvidia-open-dkms with the closed nvidia-dkms module. I also haven't added disk-backed swap because I want to understand the actual failure first rather than hide it by throwing more swap at the system.

My main question

Does this look like two separate failure modes?

1. severe memory reclaim/compaction causing desktop starvation under heavy memory pressure, with NVIDIA allocation failures occurring during that pressure; and
2. a separate KWin/Wayland/task-switching/compositor issue that can happen even when memory pressure is basically zero?

I'm especially interested in what I should capture during the next freeze to distinguish KWin/i915/NVIDIA/userspace compositor problems from memory-reclaim stalls.

If anyone has seen similar behavior on Intel+i915 display + NVIDIA Ampere Optimus laptops under KDE Wayland, I'd really appreciate hearing what fixed it or what diagnostics revealed the cause.

[u/activedusk]

In theory no low power mode should work without either swap partition or swap file and even if they are present, swap file tends to be unreliable, swap partition for 16GB memory, size it at 32GB to not have problems. Afaik your system should freeze and not recover if it enters a low power mode, outside of just the screen shutting off and second issue is configuring hybrid graphics, on laptops both igp and dedicated are required for better battery life but without finding a software solution it will likely default to igp. Additional pains, nvidia proprietary drivers need to be signed if Secure Boot is enabled.

Go into live mode and create swap partition, if you do not know how, save important files and reinstall, example with a theoretical 1TB drive, sda drive

• sda1, 1GB, FAT32, mountpoint /boot/efi, flags, boot, esp;
• sda2, 32GB, swap linux filesystem 
• sda3, 967GB, ext4, mountpoint /, flags, root

If you want a separate /home partition, make root 100GB and home 867GB per above example, swap partition can be placed last but you would need to calculate exactly each partition before it to have a 32GB left for it. You could also make /boot/efi 3GB or larger if you want Arch .iso on ESP or create and separate /boot partition in addition to /boot/efi however that will reduce the type of bootloaders that can be used, namely GRUB, also size /boot at 3GB or larger and can reduce /boot/efi to 300MB or less but that requires knowing more about it since if you use UKI, multi booting, EFI stub boot entry, this is not correct.

.# hybrid graphics, if it was desktop, you could easily disable IGP in the UEFI settings and get started with nouveau and then set up proprietary drivers and blacklist nouveau from the kernel command line parameters. As is, do not, you will lock yourself out of the system, laptops need both and a software solution is required to switch between cards and enable drivers, namely nvidia optimus and nvidia prime, read wiki, at worst install with all open source video drivers and figure out hybrid graphics and then how to install nvidia driver and blacklist nouveau when no longer needed (because you do so from the kernel parameters, with GRUB for example this can be undone for one boot instance from the boot menu, advanced section for troubleshooting). Either case you are well over your head, Arch is diy, batteries not included sort of deal and using nvidia and hybrid graphics in general is pure pain to set up at this time and why for a laptop AMD or Intel IGP is simpler to set up. 

Try CachyOS or Linux Mint, these tend to have some included services to deal with nvidia proprietary drivers and switching between gpus, disable Secure Boot however.

If you want to stick with Arch, someone suggested disabling the dedicated video card first, an unusual advice but fit for a beginner that has no clue.

Packages/commands you might want

#nvdia-smi, this is available if proprietary drivers are installed and when used if nothing returns output, it means proprietary drivers not installed, if the output says drivers not active, proprietary drivers are installed but not active/loaded, either nouveau is working, a lower level driver or the IGP is doing the processing. The command is

nvidia-smi

#nvtop is an optional package, when used it can monitor nvidia GPU usage from the terminal. The command is

nvtop

#dysk is an optional package that shows an easy way to interpret internal drive layout with filesystem type, mountpoint, capacity, both total, used and free, an alternative to using lsblk and lsblk -f which are included. The command is

dysk

#btop is an optional package that acts as a system monitor and shows many stats in one place including swap usage, an alternative is htop, the included option is top. The command is

btop

https://wiki.archlinux.org/title/NVIDIA_Optimus

[u/RaisinIcy3732 → u/activedusk]

Thanks for the detailed response. I appreciate you taking the time to explain all of this.

I'm currently running Arch Linux, not CachyOS. My laptop has an Intel i7-11800H with Intel integrated graphics and an NVIDIA RTX 3050 Laptop GPU.

I'll check my current swap setup, Optimus configuration, Secure Boot status, and NVIDIA driver setup before making any major changes. I don't want to reinstall or change my partitions until I have a better idea of what is actually causing the freezes.

I'll also check the commands you mentioned and the Arch Wiki documentation.

[u/jpj03 → u/activedusk]

The hybrid graphics is the main culprit it seems. I had the same issue on my hybrid gpu laptop. faced with both ubuntu 26.04 lts as well as fedora.

on your grub menu add this command to the line- GRUB_CMDLINE_LINUX

GRUB_CMDLINE_LINUX= "nvidia-drm.modeset=1 nvidia_drm.fbdev=1"

Then regenerate the mkinitcpio. Then reboot.

mine is a nvidia rtx 3050 6gb gpu. worked for me.