We’d like to welcome Pureboot users to give our Release-19 preview-1 beta/test images a try and provide feedback here.
This preview release features a completely revamped Pureboot UI / menu system, with faster navigation, better guidance when errors occur, better consistency in background color indication, as more.
It also features a new root filesystem verification option, which extends the checking already done on /boot to root (/) as well. The feature is completely optional, and can be used manually or configured to run automatically on each boot.
We’d love to get feedback on these new features/updates, as well as any other ways we can improve Pureboot for the next release.
This is a new feature that I added recently and really welcome feedback on. I figured I’d give some background on this feature and my thought process behind it in this thread so people understand where I’m coming from and why I made the design decisions I did with it. I’ve been thinking a long time about how to extend tamper detection from /boot into the main file system. I wanted to give people the ability to test their system for tampering from within some sort of trusted environment, and there were a few approaches we could take to do it.
There are certainly more advanced tools out there like dm_verity but that requires significantly more effort on the user side, major changes within the OS, and just in general adds more complexity. I’m not saying it’s a bad approach, just that it’s one that would take more time to implement, as well as only be available for people running PureOS (since that would be where we would start development, it would be up to other distros to mimic our approach).
Scan from initrd
My initial plans were to extend the chain of trust over to the initrd file and add root hash verification occur there after the kernel takes over and initrd unlocks the root file system, but immediately before pivot_root changes over to it.
There are some advantages to the initrd approach, but the primary disadvantage is that the reliance on initrd makes it distribution-specific. This is something I ran into when developing the Librem Key LUKS unlock feature for PureBoot. This means we’d end up developing a script that would work only with PureOS and hope that we might upstream it some day at least into Debian, but since other distributions handle initrd updates different from the Debian ecosystem, people would have to adapt our effort to all the different initrd update approaches out there.
Scan from Live Disk
Another idea I had was to create a limited, trusted live USB disk that contained the appropriate scripts and then a user could boot into that and perform their checks from that trusted environment. Of course that means either maintaining a separate ISO just for that process or otherwise incorporating it into our PureOS ISO, all just to run commands we already had in PureBoot–a trusted environment. So I decided against that approach.
Scan from PureBoot
An alternative to the initrd and live disk approaches is to handle everything from PureBoot. With that approach, root hashes are initially generated from within PureBoot and stored in /boot alongside (but in a separate file to) the /boot hashes. I like this approach because PureBoot is already a limited and trusted environment, so the underlying system is limited in its ability to interfere with scanning run from here.
By default we don’t automatically scan files on the root disk, because depending on the disk it might take a minute or more. Instead it’s something a user can choose to do at any time from the PureBoot menu whenever they perform system updates, or whenever the hardware was outside of their custody.
The normal, manual workflow would look something like this:
User builds initial hashes of root file system within PureBoot
User boots into system and uses as normal
When there are updates to apply, user reboots and checks signatures from PureBoot to detect any changes that aren’t related to software updates
User then applies updates
User then reboots and immediately does another root hash check to see which changes software updates made, and then sign those changes
In the case of PureOS, since we default to using Packagekit for software updates, the system will automatically reboot to a more limited environment to apply updates so this workflow is a bit simpler to manage.
Automatic Scanning at Boot
We also added the option (off by default) to automatically scan root at each boot. We turn it off by default due to the dramatic increase in boot time, but if you turn it on, then at each boot, after PureBoot verifies the files in /boot haven’t been tampered with, it moves on to prompt you to unlock your root disk and then performs scanning there. It will alert you to any changes and allow you to re-sign them (in the case they are the result of software updates) or otherwise you can abort the boot process and investigate.
I remember a while back there was a discussion about PureBoot only checking changed files, but not noticing new files. Is this still the case?
I’m asking because on / there are possibly quite a few ways to attack a system by merely adding a file.
Yes, I took the same approach with this that Heads already takes with /boot. Because this is using sha256sum against a signature file, just like with /boot, it only checks whether existing files have been tampered with (so will help with things like rootkits that replace system binaries with trojaned versions), so you will have to factor that into your security models.
This doesn’t check all files in /, because certain directories change frequently, like /home and /var. Instead it defaults to checking /bin, /boot (which should be empty since PureBoot systems require a separate /boot partition, but it’s here just in case), /lib /sbin, and /usr.
But you can edit the list of directories it scans to add or remove what it checks. For instance if you want to set this up to run at each boot, but don’t like how long it takes, you could remove /usr entirely (or replace it with /usr/bin, /usr/lib and /usr/sbin). That would of course reduce protection from tampering but it would also dramatically increase scanning time.
I believe you already made this point earlier in the thread. As I said, the goal with this feature at the moment is to extend tamper detection on existing files past /boot and into specific root directories, which would catch a rootkit that replaces system binaries with trojaned versions. The goal isn’t to stop all possible attacks on the system and I don’t think anyone is claiming that it does that.
If you want to detect other types of tampering on the file system, like attacks that add new files to the file system, you’d want to use a different tool for that job. This tool also doesn’t (by default) address attacks within /home or /var. There are plenty of places that malware can persist on a system including in unused blocks on disk.
Would it be a nice feature to detect the addition of files within the scanned directories? Sure, there are many additional things we could do. Does this protect people from certain classes of attacks better than without it? Yes.
I tend to approach security like golf, in the sense that I aim to get closer to the hole (a secure system) with each stroke. Then at my next stroke I can start from the new spot, possibly with a different tool that’s better suited for getting closer to the hole from that point. The closer I get the better, but if a stroke gets closer but doesn’t put the ball in the hole I don’t throw that attempt away and start from the same initial spot. Some security practitioners only swing for holes in one and spend most of their time no closer to the hole.
Well, no, for /boot it was only a theoretical concern, I don’t know a way how to exploit it there.
That is where I don’t get your reasoning. An attacker writes a file, for it to be automatically executed.
Whether that file replaces an existing file, or is new, but tricks an existing file into loading it, is a very academic distinction.
I understand there are other kinds of attacks that need other tools. But to me, that’s still the same attack.
Isn’t the fix for this (notice files that don’t have a checksum recorded) equally trivial than the workaround for the attacker (don’t overwrite files)?
@Kyle_Rankin i see one issue with your approach to tampering detection:
thanks to systemd there is no longer /bin /sbin /lib /libx32 /lib32 /lib64
all those are symlinks to /usr/*
times where system was cut into 2 pieces (boot/startup critical + core system tools in /, rest of userspace in /usr) ended…
what is the sense of tampering detection on / if i have whole system encrypted
and more rethorical question:
why we keep /boot uncencrypted?
technically ore wise would be keep entire disk encrypted, and use TPM LUKS Unseal.
/usr is one of the directories we include by default in the list of directories we scan, however by including bin, lib, and others we make sure we don’t miss out on systems that may not have that symlink in place.
The thing about encryption at rest is that the system is often not at rest. The goal here is to detect rootkits that might have replaced system binaries with tampered versions (classic example would be /bin/bash) while the system was running, extending the tamper detection we already perform that would catch rootkits that tamper with the kernel or initrd (or grub config). We would also detect tampering that occurred by an attacker who had access to the system while at rest, who was able to crack the disk passphrase.
PureBoot and Heads needs a scratch space it can read, so it can store the HOTP incrementing counter, so it can parse and display the GRUB boot options, and it also needs to be able to test files in /boot for tampering before executing them. At the moment the simplest approach is to leave /boot unencrypted, relying on the fact that we can detect attempts to tamper with files stored there (and that directory doesn’t contain any secrets).
One downside to encrypting /boot with respect to PureBoot, is that it would require us to prompt the user for a secret before we have authenticated ourselves to the user (specifically to their Librem Key), so we could have access to that HOTP counter. If we could somehow store that counter elsewhere, I suppose it might be OK to prompt them for a secret, once they can trust the PureBoot firmware isn’t tampered with. Unfortunately the TPM only supports a single auto-incrementing counter at a time and that is already being used for the rollback counter in Heads. We do prompt the user for their / LUKS secret to test / for tampering, but only after we have proven the firmware is trustworthy.
I’ll look into the implications of adding this feature. Hopefully after we have traversed the file system once to do the sha256sum check, the directory contents should be cached in RAM and a second traversal with the find command would be much faster.
I turned my forum entry above into a full blog post, but instead of splitting the discussion I figured it makes sense to continue it all in this thread.
