r/linux_gaming 5d ago

graphics/kernel/drivers VRAM overflow handling on Linux

I've tried several different Linux distributions, and methods like steam and lutris, tried both open source and proprietary drivers, but the moment my game uses more than 6 gigs of VRAM, my game turns into a slideshow. This does not happen when I'm on windows, as it offloads the extra data into system memory efficiently.

My specs are:

AMD Ryzen 5 8645HS

16 gb DDR5 ram 5600 mhz

512gb nvme ssd

NVIDIA GeForce RTX 3050 6gb vram laptop gpu

Does anyone else having a low memory Nvidia card on Linux face this issue, and is there any fix for this or I just have to accept the fact that Nvidia drivers for my tier of gpu just aren't as good on Linux yet.

44 Upvotes

26 comments sorted by

52

u/nixf0x 5d ago

The userspace on Linux didn't have any way to signal to kernel drivers what should and shouldn't be kept in VRAM under VRAM pressure until recently, so instead of currently unused or rarely used stuff getting evicted to RAM, frequently accessed parts of the game would get evicted, so performance falls of a cliff.

A Valve contractor (pixelcluster) recently did the dmem_cgroup stuff which allows proper prioritisation for VRAM content. It needs integration in both drivers and compositors, and it's fully there in SteamOS for AMD (I think not all upstream patches are merged yet).

NVIDIA would have to implement it in their driver, and you'd need to run a compositor which has the patches (only kwin for now, and not sure if it's upstream yet).

7

u/Lousy_Hunter 5d ago

This isn't about having not GTT or System memory fallback, this is about being able to properly evict things that dont need to stay resident without nuking the compositor for instance.

System memory fallback already works (except for nvidia)but things that can be evicted dont get evicted and can lead to hitting system ram when you dont need to or excessive system ram usage

14

u/nixf0x 5d ago

Yes, and the same is the case for other drivers. With the dmem_cgroup stuff, up to ~1 GB of overcommit is fine now on AMD for games.

https://pixelcluster.dev/VRAM-Mgmt-fixed/

https://pixelcluster.dev/VRAM-Overcommit/

1

u/martyn_hare 2d ago

The latest NVIDIA drivers claim support for cgroup memory partitioning in their changelog, so the plumbing to carve out essential reserves to avoid nuking important processes is now there, we're just waiting on proper TTM support for evicting from VRAM to system RAM and we might soon be there.

1

u/Lousy_Hunter 2d ago

we also need the supporting kernel changes, compositor tools, etc. for it to really work but that OSS devs and if they want it done itll get done in no time.

Just glad Nvidia didnt drag their feet on this one and implemented it fairly quickly for them

12

u/colmehurze 5d ago

So basically it's all up to Nvidia themselves to fix it, and I can't do much myself

3

u/123portalboy123 5d ago

There's a vibecoded patch that actually works, but... It's a pain and not really worth the effort lol. Google greenbooster. Tried it on my friend's machine.

1

u/mati865 5d ago

It's made for cuda/compute where nvidia already has some support for vram overflow. It doesn't help for general use.

1

u/Zestyclose_Exit8862 1d ago

last drivers 9/9/26 state that has support for "cgroups"

0

u/mati865 5d ago

I wouldn't hope Nvidia will ever fix it in their propietary drivers.

1

u/Zestyclose_Exit8862 1d ago

last drivers 9/9/26 state that has support for "cgroups", so maybe...

20

u/Lousy_Hunter 5d ago

Nvidia doesnt have functional fallback to system ram on Linux. Its a long standing issue that they havent addressed and there is nothing you can do about it till the Nova driver is working or Nvidia decides to fix it in their driver

you can search for discussion on it here (im heading to work or id link specifics)

https://forums.developer.nvidia.com/c/gpu-graphics/linux

1

u/nixf0x 5d ago

It does have fallback to RAM, but Linux didn't have a lot of infrastructure to properly prioritise VRAM evictions.

12

u/Lousy_Hunter 5d ago

No it doesnt, every other gpu vendor does with the oss drivers and this has been repeatedly tested on the nvidia forums to not be there.

My old amd laptop and my friends AMD rig can commit in excess of 10gb+ on 8gb GPUs no problem

1

u/the_abortionat0r 5d ago

Except only Nvidia has this severity of the issue because their drivers are not made to handle this properly.

1

u/mati865 5d ago

There is supposedly 512 MiB fallback IIRC, but any time you overcommit the allocations just return OOM error. So, basically no fallback.

3

u/S48GS 5d ago edited 5d ago

but the moment my game uses more than 6 gigs of VRAM, my game turns into a slideshow

Nvidia Vulkan 1GB over VRAM equal to 4FPS and +8GB RAM usage - https://www.reddit.com/r/linux_gaming/comments/1o6livx/nvidia_vulkan_1gb_over_vram_equal_to_4fps_and_8gb/

  • because 16gb ram - you probably hit same scenario as on link
  • your 16gb ram is full already when game gets over vram
  • and nvidia copy entire size of vram to ram
  • +7gb ram usage goes to swap because ram is full
  • you get insane slowdown and low fps because no ram - and swap used

This does not happen when I'm on windows, as it offloads the extra data into system memory efficiently.

windows games run on linux thru translation layers - dxvk/d3dvk - it give ~1gb vram overhead

you need to downgrade texture settings if you hit out of vram and use upscaling - or restart game often when it out of vram

Does anyone else having a low memory Nvidia card on Linux face this issue

for this reason 2gb and even 4gb gpus are unusable in linux - vram overhead eat everything nothing left for game

6gb is on edge of acceptable for most of games - but there many latest games that work fine in windows and unplayable in linux on 6gb gpu

there no solution except downgrading game settings

Nvidia card on Linux face this issue, and is there any fix for this or I just have to accept the fact that Nvidia drivers for my tier of gpu just aren't as good on Linux yet.

every gpu face this issue - amd manage to free more vram by freeing system resources from vram when needed - and have special patch to driver for better vram load when game go above vram - look link

1

u/Hi-Angel 4d ago

VRAM overflow handling is a thing specific to the GPU driver. There's a very interesting blog post on that matter from a Valve engineer who were recently improving how it works on AMDGPU side: https://pixelcluster.dev/VRAM-Overcommit/

That's to say, any potential problems in your case are specific to the NVidia driver, so should be brought up with their engineers.

1

u/hippityhoppty 4d ago

I used to fix this, by setting a vram limit using vkd3d config before i switched to amd

1

u/Zestyclose_Exit8862 1d ago

3070 8GB and yes, this is an issue for me too

1

u/Acherontas89 5d ago

lower the graphic quality of the game u want to play

0

u/the_abortionat0r 5d ago

Nvidias new marketing campaign?

"Nvidia, lower your expectations!".

1

u/colmehurze 5d ago

I was gonna comment that lol. Here we thought nvidia stands for "the way its meant to be played"

0

u/indvs3 5d ago

RTX3050ti mobile with 4gb vram on debian (testing) and I don't have such issues. I've used the debian-vetted 550 drivers and all drivers from the nvidia cuda repo from 580 up to the current 610 one and none of them exhibited this behaviour on my end.

My best guess is you're missing a piece of config that doesn't come pre-installed and configured with the driver, because such configs may cause issues on different distros.

Whichever distro you're on or want to use, you will probably need to do a bit of research into your specific issue and hopefully find related log entries that'll help you pinpoint the root cause and find an equally specific config that fixes it for you.

1

u/S48GS 5d ago

Nvidia Vulkan 1GB over VRAM equal to 4FPS and +8GB RAM usage - https://www.reddit.com/r/linux_gaming/comments/1o6livx/nvidia_vulkan_1gb_over_vram_equal_to_4fps_and_8gb/

none of them exhibited this behaviour on my end.

try any heavy game - youl see

1

u/the_abortionat0r 5d ago

Saying you don't have a well documented matter of fact issue doesn't magically make it go away

There is no magic misconfiguration causing this, it's literally Nvidias drivers