Heads: Adding files to /boot not detected?

I just enabled the heads / LibremKey integration on a Librem13v4. Really great work :slight_smile: !!

After setting it up I looked at the checksums and the pgp signature in /boot and found two old files I had saved there during writing the instructions for re-encryption of the disk.

I deleted these temporary files and didn’t think much more about it.

During the next reboot heads told me my device had been tampered with - no wonder: the two deleted files had ended up in the list of hashes on /boot.

To demonstrate the tamper dectection I generated some file in /boot and restarted.

No warning. Shouldn’t the addition of files to boot be detected the same way removing files from boot is detected?

1 Like

Arguably yes. …

Thinking about it I couldn’t really come up with an actual threat that would be the result of writing to /boot without the owner being aware of it (as in tamper detection).

I had some ideas, that seem to be quiet a bit far fetched:

  • Some malware/attacker could write information gained while the system is running to /boot to make it available unencrypted on the device. Still the attacker would have to solve the problem of getting to the unencrypted data, but this would open the possiblility to just copy it from that disk when the attacker gains physical access.
  • Often there are mechanisems in operating systems that are unknown to users. I’m not aware of anything that could actually be triggered by adding files to boot, but as I wrote - what do I know and what mechanisms will be added in future if they do not exist today.

For me by the definition of ‘tamper detection’ I’d expect to be warned if files would be added or even if data on the partition which does not belong to a file would have been altered.

I’ll try the following: I’ll fill up the free space on /boot with one file containing random data and test

  • wheter the full /boot has negative effects on the boot process
  • how long it’ll take to check this extra data while booting

Maybe it’s a simple way to improve the tamper detection and extend it to nearly all unencrypted space on the device.

Heads does two different types of tamper detection at two different phases of boot.

  1. It detects whether the firmware has been modified at the moment you boot your computer, by sending measurements of the firmware that’s executing over to the TPM. If this succeeds, your Librem Key flashes green and you know that you can trust the firmware (and importantly, the GPG keyring inside the firmware).
  2. When you tell Heads to boot your OS, before it kexecs your particular kernel, it first scans /boot for any changes to relevant files there. It compares those measurements to ones it has stored in one of the config files labeled kexec* for changes. The goal here is to detect rootkits that have modified your kernel, initrd, or grub.cfg files.

Once Heads is confident that the kernel, initrd, and grub.cfg are trusted it hands over control and trust to the kernel by kexecing into it. Otherwise it will flash an error when you attempt to boot into the OS.

1 Like

The main tampering it is concerned about are changes to trusted files that can impact boot, which is why it would in fact get triggered if you added a kernel and changed grub.cfg. My initial reponse to your question is that triggering on adding new files probably wouldn’t be necessary as long as triggering on the list of trusted files is in place. I’d need to think through any legitimate attack scenarios though to see if I can think of a situation where adding a file to /boot after the existing files have been signed, could affect the security of the kernel.

It’s an interesting problem. While it seems rather esoteric, I think the exfiltration case should not be dismissed lightly, even if that would require root access.
But also the infiltration case. @Kyle_Rankin Maybe we can’t think of a real existing threat, but that could change unexpectedly with an innocent update, where some software changes its behavior if a certain file exists. Even if it’s just a forgotten development aid.

While for infiltration it might be sufficient to check the file count, I guess it would be safest to check the partition as a whole.
Which also makes me wonder if all of the flash is checked or only the part occupied by the coreboot image.

It’s more just a case of understanding what threats Heads is attempting to protect you from. It’s important to limit the scope of security tools and to fully understand what it is supposed to protect you from and what’s out of scope.

In the case of Heads, the goal it to protect against the following threats:

  • An attacker modifying the firmware
  • An attacker modifying the kernel, initrd, and grub.cfg you boot into

The goal is to start the chain of trust from a trustworthy place, to protect the very first code the system executes from tampering, so that you can trust it up to the point that it loads the kernel. At that point the kernel can take over and handle any subsequent verification checks you may want to perform (such as on the underlying system files etc.).

2 Likes

Thanks for clarification on this. I read through - I guess - most of the documents of Purism about integrating heads. What got stuck in my head has been until you explained: Tamper detection - bios - firmware - /boot.

Either I didn’t read carefully enough or I had been misleaded to have the expectation of /boot being validated in a more complete way. Partly the reason probably is that I deleted temporary files in boot and had been warned by boot about my own tampering (removing files). This probably helped a lot to lead to the expectation that adding files would be detected also.

Well, don’t know - maybe someone else could give her/his opinion whether this should be more emphasized in the describing documents.

I thought also some more about a possible attack vector that could be a reason to change the behavior of heads and startet thinking along this track without having read into details or even code, yet:

Heads probably is somehow quiet condensend because its binary has to fit into a limited space. For that and any other possible reason the code to access the filesystem is probably not the same as in the kernel driver. Even if it is the same at any point of time the update of those codes for accessing the extX-filesystem is asynchronous and so the code base could drift apart.

If this is true it can be assumed that the checks and available feature for the extX filesystem differ or might differ at some time in future.

If so my actual attack vector would be trying to trick the kernel and heads to have a different view on the same binary data on the boot partition by - e.g. checking on the handling of inodes, sysmlinks, hardlinks, mishandling of unexpected data in the partition.

Don’t know if there is the slightest chance this could work, but - and this is my point - I don’t know neither whether it is impossible. Can somebody tell me without a lot of research that this approach is impossible?

If so: Good :), but I guess it is not that easy. If not: I’d again and still have the expectation that heads tampering detection would protect me against such an attack that is based on altering my unencrypted /boot.

I tried this. The time to sha256sum the missing ~900MB on the command line on my notebook is a little less than 5 seconds. Not that big difference.

Tampering my file full of zeros filling up /boot is detected by heads.

At least like this it is more difficult to hide an amount of data on my /boot for whatever reason. I would be warned.

Still this is not a protection to possible security issues regarding the implementation of ext in heads…

Neither am I but my concern was the proliferation of logic like the following.

if file exists then
    use it

or

use the first file that exists from among this list of files
    (could be different directories to look in or different files to look for,
     or both)

So the mere creation of a new file could potentially alter the behaviour.

I think this is what you are saying and it applies both to current functionality and to future functionality.

Maybe each relevant directory should be ‘hashed’ by hashing the list of contained files in some canonical form.

Yes, that’s what I wrote earlier.

In my later post I speculated about an attack vector that might exist because the filesystem drivers used by heads, kernel, grub, etc. might differ at some point of time in a way that you could trick heads into validating the wrong data on disk, make it unexpected silently fail or make the kernel read wrong data.

If there is no way to definitly say this could’nt happen, I’d like to wait 5 seconds more (I tested sha256sum times on my whole /boot) to validate the whole partition in binary.

This might sound easy. And yes, the code for reading a filesystem could be dropped more or less (depending whether e.g. mount counters or similiar metadata has to be ignored that might change during normal operation without explicitly writing to a file on /boot) and the whole process would become more independent of the used filesystem.

But heads needs a place to store the checksums and there it might get somehow complicated again. My first thought is that the filesystem used on /boot could leave some space at the end of the partition to save the hash.

Probably there are better ways to do it and certainly this is not as easy as it might sound.

There is a package in the debian repo called chkboot which sounds like it may do what you want.
The github repo for it can be found at https://github.com/philippmeisberger/chkboot

1 Like

This is probably the bottom line as far as further verification BUT I am not sure that those checks normally get done in a comprehensive fashion. How much in the wider system is digitally signed in a secure verifiable way? From what I’ve seen, not much.

The following page discusses some of the issues, albeit a bit old.

So perhaps the boot path is protected only as far as it is practical to do so.

There are limits to what can be achieved with a straight hash. Digital signature systems generally do more than that. Alternatively, you need a safe place to store the hash, as you wrote.

I read into the description. The scripts can’t and do not check the code before it is executed. It is probably a good way to protect against automated attacks as long as they do not include modifications to those scripts.

Heads checks on much of the boot code before it gets executed: It uses tpm to check the bios and thereby its own code and it then checks /boot before any code from there is executed.

Furthermore it checks like the actual heads files on /boot and not the partition as a whole.

If someone is using heads I can’t see how using chkboot would further improve security, but I might have missed some case…

(about sha256sum)

sha256 can’t help much on its own - that’s true. From my posting it didn’t become obvious, that I would expect heads to check the sha256 for the whole boot partition against a signature the same way it does it checks the signature against the list of hashes in /boot, now.

The signature would have to be stored alongside with the hash of the boot partition (and maybe further hashes for the partition table and whatelse should be protected against tampering). The signature should be checked against the LibremKey - as it is already done.

Better not to try!

Beware - since linux-image-4.19.0-5-amd64 4.19.37-5+deb10u2 that I installed today, filling up /boot breaks the boot process.

Easy way out is to select the rescue shell in heads menu, delete the data that filled up the disk, reboot, sign the changed /boot and restart.

Anyway: It is a bad idea to not delete any disk fillup file on /boot before an update, because a new generated initramfs might not fit into /boot.

1 Like

@Kyle_Rankin Just became aware of some of those:

Another attack vector discussed here had been the interpretation of the file system. Actual now is the CVE-2019-5094 that might have security implications to heads (depending on what ext-fs-code is used in heads-

I can only repeat that in my opinion the idea of a) interpreting the file system to make checksums of existing files and b) ignoring the addition of new files is not sufficient to reach the goals heads is aimed at (at least as I understand them).

Both - CVE and automatic configuration in raspbian - show in my opinion that this cannot be achieved the way it works now.

2 Likes

That’s always possible e.g. if general purpose code gets pulled in - but a read-only file system has no need to look at the quota file.

Still, that’s just a bug, right? It gets fixed and it’s not a problem any more - whereas the other specific issues that you mention are more endemic (and much more what worried me).

True - and that could be a really big can of worms e.g. if two pieces of code interpret the file system in different ways (not necessarily due to any bugs in that code).