Summary
A Valve engineer added kernel and userspace tweaks to stop VRAM evictions on 8GB GPUs, improving game performance.
The fix uses cgroups/DMEM to prioritize active apps so games don’t get swapped to slower RAM.
Two user packages, dmemcg-booster and plasma-foreground-booster, protect the foreground game’s VRAM from eviction.
One of the coolest things about Linux distros is that, if someone has a problem with the OS, they can whip up a solution themselves and have it added to Linux proper. For example, there is someone who has made it her life’s mission to add every rhythm game controller possible to Linux’s kernel. Nobody else was doing it, so she did it instead.
Now, a Valve engineer noticed that Linux wasn’t properly managing the memory on GPUs with 8GB VRAM or less while rendering a game. So, not only did she publish a change in the Linux kernel itself, but she also introduced new packages to put her work to good use.

Related
4 things you should do to make the ultimate Linux gaming PC
Get your Linux PC ready for those long gaming sessions.
A Valve Linux graphics driver team member helps 8GB GPUs run games better
She wrote about everything she did, too

Credit:Â ROG Global
As reported by Phoronix, this amazing feat was performed by one of Valve’s Linux graphics engineers, Natalie Vock. Like all good engineers, Vock had a problem with how Linux handled VRAM, so she made a solution and implemented it. And like all great engineers, she wrote everything down so we could see what she did.
As she mentions on her Pixelcluster’s GPU Blog, if Linux spots that the GPU’s VRAM is full (likely because you have a few background apps open alongside a game), the OS will begin evicting VRAM data onto regular RAM. Moving the memory prevents the app from crashing, but unfortunately, RAM is slower than VRAM, meaning whatever gets shunted out suffers a performance loss.
The problem is, when Linux had to perform these evictions, it couldn’t identify which parts of the VRAM was handling the game (which you really don’t want evicted, else you have performance woes) and which parts were handling the random Chrome window you’ve had open for three hours (which you’d much prefer was evicted while you play Cyberpunk 2077). So, Vock got to work fixing that.
The solution she settled on used cgroups, a Linux kernel feature that systemd uses to put every running app into its own little unit. Doing this allows the kernel to see which app deserves priority and which can be evicted. To help Linux manage all of this, Vock used the DMEM group controller within Linux, which she wrote alongside Maarten Lankhorst of Intel and Maxime Ripard of Red Hat.
With the DMEM group controller, Linux can now identify which bits of memory are most important. Now, instead of evicting blocks of memory without a care for what they’re actually doing, Linux can instead ensure that all non-vital apps are evicted before it begins touching the most important one: the one the user is using in that moment.
Her changes include some tweaks in the Linux kernel, plus two new packages in the userspace: dmemcg-booster and plasma-foreground-booster. The former package uses cgroup management to prevent games from being shunted out of VRAM, while the latter tells the PC to always focus on the current active window. The end result means far better game performance loss protection on Linux, especially on 8GB and lower VRAM.

Related
Wine has been translating Windows games to Linux since 1993, but Proton is what made it effortless
Wine is the foundation that makes gaming on Linux possible.