I’m curious about the relationship between physical RAM and /swap.
How does memory management decide when to use /swap?
I came to this while creating an iso image from Gnome Disks of a failing external drive ( to another external drive). The /swap usage seems much higher than I’ve seen it before. Is this because I’m running a disk intensive process?
Here’s a SC from System Monitor:
That’s actually an incredibly complex topic. The simple answer is ‘it depends’.
In general, swap is used when there isn’t enough memory to back the pages actually used. The kernel doesn’t limit the pages requested, but only actually allocates a page when its first written. So, when a page is allocated, and then written, if there isn’t a ready to go page, the kernel scrounges one up.
The first complicating feature is disk cache. When a file is read or written, a copy of its contents lingers in memory, in case it’s needed again soon. Normally, disk cache is lower precedence than program memory, so if the kernel needs to scrounge a page, it’ll dump disk cache. Setting vm.swappiness to 100 makes it consider disk cache and program pages on equal footing. It may discard a page of disk cache, if there’s one that hasn’t been used in a while. If there’s a program page that’s older, it’ll write that to swap instead.
The next complicating feature is zram/zswap/zcache. These are 3 similar mechanisms (zcache is going away) to let the kernel keep some compressed pages around, getting a 2-3x space gain in exchange for extra cpu time (of them, zram is both the most flexible and the most efficient, but the setup is the most complex). Most computers don’t have this enabled by default.
The last complicating feature is cgroups. You can put artificial limits on memory use by programs (and you want to do so if you run a swap file or especially zram). If you do this, you can put both a soft and a hard cap. If a program group exceeds its soft cap, the kernel will start swapping some of the pages out, but not in a huge hurry. If it hits the hard cap, it’ll be treated like it would if the whole system ran out, including tripping the OOMK if needed.
You’ve not given enough data to answer why swap is higher, but probably due to your vm.swappiness value combined with the large amount of disk cache that’s likely to be in use from the disk copy. If you want to avoid it, you could use cgroups to limit the amount of disk cache the copy program uses (since it won’t be looking at the data again once written, it doesn’t need much cache).
i 
lperkins2.
Thanks for this. It makes sense then that the Disk gui would limit, since I would I probably FREAK OUT if copying files between drives in the background made the system stagger.
Really concise and thorough, thanks.
so what happens if there is no /swap and there is just a huge ram disk ?
For clarity, it’s not /swap, it’s just swap. It’s not mounted at any place on the filesystem.
That said, if there is no swap, the kernel still allows overcommit (programs can request huge amounts of memory). If a program tries to write to an over committed page, the kernel dumps disk cache. Depending on the system, when it’s critically low on available pages, it notifies interested applications through /dev/mem_notify or some similar feature (/dev/mem_notify is currently unsupported on kernels newer than 2.6.28). This lets programs clean up program-level caches to free some memory. If the system doesn’t have a userspace mem_notify system, or that doesn’t free up enough memory, it next fires the out of memory (OOM) killer, which picks the ‘best’ program and sends it a SIGTERM. Best is determined by shortest running, biggest memory in use, no/high value written to /proc/self/oom_score_adj. Basically, it looks for what program is likely to be causing the OOM condition, and kills that.
Although that varies with distro and version. More recent versions of Ubuntu have abandoned the swap partition and it is just a file that is located in the root directory.
Except it isn’t. I mean, /swap (or /swap.something) may well be a swap file, but it isn’t your swap, it isn’t mounted on the filesystem. It’s just a normal file, which is used as a backing store for swap. You can actually have multiple backing stores for swap, and the kernel will load balance between them. You can sort of think of all the different swap devices, partitions, files, et cetera as a single raid0. It is correct to point to one of the devices and say ‘that contains swap’, but just as /dev/sdxN is not your /, /swap.img is not your swap.
That aside, swapfiles come and go as the recommended backing system. Unless you do something smart with cgroups, I generally recommend only using a small zram, and then letting the OOMK do its job, Linux’s performance degrades so terribly when swapping that it’s generally better to just kill the offending program.
sounds like a matrix reference. thank you for your input !
This thread is missing an important point. The hibernation feature (suspend-to-disk) writes out the contents of RAM to the swap partition before turning off the machine. Therefore, your swap partition should be at least as big as your RAM size. Without swap, hibernation doesn’t work.
i don’t use hibernation on desktop but for anybody who does or who would like to do so it’s a very good observation !
Good point about hibernation, I haven’t had a computer that wasn’t always on in long enough I didn’t think of it. That said, if you want hibernation without the poor behaviour that comes with swap, you can set up a swap file that gets activated during the hibernation process, and deactivated on resume.
It should be sufficient to max out swappiness to avoid it being used for active paging. And cold pages - who cares of them. The only downside is if you do start paging for whatever reason - the system is lost, oom killer won’t kick in till swap is exhausted and that’s by far too late
… is a bit unreliable. I stopped using it - and it seemed like each major upgrade was removing hibernation as a configured option by default, like ‘they’ didn’t want you to use it. Your mileage may vary e.g. may depend on distro and version. I’m using Ubuntu.
So by now, on several computers, I have lots of RAM and no swap at all.
I wish there was a way with cgroups to limit swap use. You can soft or hard limit active pages, which lets you force a low priority memory hungry application to swap, without degrading overall system performance, but you cannot have swap and force a cgroup to OOM kill something rather than touching swap.
i’ve a question about swap … newer versions of ubuntu auto create a swap file under / but debian doesn’t (not even on version 10). how do you prevent the swap file from beeing auto enabled upon each power-on and how do you get rid of it completely ?
i prefer debian since it has a more “thorough” GUI installer for desktops and gives a lot more control on configuring stuff but takes a little bit longer. Calamares is good imo …
Not running ubuntu for a long time already but am pretty sure it will be a systemd service unit.
And unless they are not quite sane they’d bind it to swap.target.
So either you can find out which service unit does that and disable the unit, or you may disable entire target.
You can run sudo systemct list-dependencies and see who runs what and disable the unit accordingly.
actually scratch that - Debian does auto-configure a swap PARTITION automatically (aprox the same size of the total installed RAM) if you choose to tell the installer to do it for you but it does have the option of deleting the partition after and using the free-space. i guess it does that because on a laptop hibernation IS usefull.
Outline of what I would try (all commands as root)
swapoff -v /swapfile
mv /swapfile /swapfile.not
vi /etc/fstab (and comment out the line for /swapfile)
reboot
then, if you specifically want to, rm /swapfile.not
If you don’t need the disk space, I wouldn’t necessarily delete the file since, up until that point, the instructions are easily reversible.
If you do delete the swapfile and later want to get a swapfile back, here’s a random web page that tells you what steps to carry out.
thank you ! i assume these CLI instructions will work with any gnu/linux distribution right ? Debian, RHEL, SEL, Arch, Gentoo, the big players right ? as long as there is vi or vim installed if not another CLI text editor will do …
Your assumption, not mine. Honestly I don’t know. You asked about Ubuntu and they will work with Ubuntu.
There are at least two things to check on a different distro:
swapoff and swapon exist as commands? test with man or which
man swapon will probably answer that)