Qubes Boot fails on almost new Librem 14

I have a new Librem 14. I used it for a few days and I updated with the update tools offed within Qubes. When rebooting there is a signature verification error:

+++ Found verified exec boot params
gpg: verify signatures failed: unknown system error
Invalid signature on exec boot params
!!! Failed default boot
New value of PCR[4]: (…)
!!! Starting recovery shell

how to proceed?

I would think is normal. You have to re-sign things after a relevant software update.

Updates that affect /boot do require re-signing, but it shouldn’t drop you directly to a recovery shell, it should indicate that the /boot signatures aren’t valid and ask you to re-sign if you are sure the change was intentional.

Is the RTC time still correct? The date/time appears in the main menu, or date in the recovery shell will show it.

Is it possible /boot was corrupted? You might want to fsck it from a live system. (If you need detailed instructions, let me know.)

In the meantime if you need to use the system and are sure it hasn’t been tampered, you should be able to interrupt the automatic boot and use “Boot options” > “Ignore tampering and force a boot (unsafe)”.

If one of those suggestions fixes it, please let me know so we can improve the diagnostics, otherwise we can start troubleshooting the signature verification to see what precisely is failing.

The time seemed correct. I had changed the timezone but I did not write a new time into the hardware clock.
Should I proceed to resign my root?
I do not need the machine immediately, so i would prefer to solve this properly.
Best regards,

I’m not sure if re-signing will fix it, it’s hard to say from gpg what the error is. Most of the reasons I can think of for it to fail with an “unknown system error” would be reading the signature file (kexec.sig), which re-signing would try to replace. But it could be failing to read it due to filesystem corruption, a hardware error, etc.

You could try re-signing to see what happens, but there is a small chance this could make corruption worse, etc. Or you could check dmesg | tail -30 in the recovery shell after the error, the kernel might tell us more about what triggered it. A photo of the screen is OK if it’s readable.

Here is a precise description:
0) Clock shows correct UTC time (checked with accuracy of a second)

  1. GUI comes up, TOTP: 763711 | HOTP: Invalid code
    Librem key blinks regularly red, clock seems to correct
  2. I do Refresh TOTP/HOTP, screen says briefly “Operation Success”
  3. Power Off
  4. Unplug Librem key
  5. Plug Librem Key
  6. Power on, same behaviour
  7. Exit to recovery shell
  8. screen says “New Value of PCR[4]: …”
    I could extract the dmesg output but it seems I’m not allowed to attach a text file. I copy the last lines below.
    “default boot” will drop me into that gpg error. this does not add anything to /proc/kmsg or dimes

There was certainly no TPM tampering.

Actually I was not dropped directly into the recovery shell, at least this time. I’m sorry I gave a misleading initial description. I am dropped there when I attempt a “default boot”
I did not do anything important on the computer, so a full factory reset would not hurt.

From here, I would proceed to reset the TPM and resign /boot using the gui-init options. Do you recommend that, assuming that complete data loss would not hurt?

[ 1.463303] input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/LNXVIDEO:00/input/input4
[ 1.467634] brd: module loaded
[ 1.470956] loop: module loaded
[ 1.470979] Loading iSCSI transport class v2.0-870.
[ 1.471415] iscsi: registered transport (tcp)
[ 1.472249] nvme nvme0: pci function 0000:03:00.0
[ 1.472423] i8042: PNP: PS/2 Controller [PNP0303:PS2K,PNP0f13:PS2M] at 0x60,0x64 irq 1,12
[ 1.487272] nvme nvme0: Shutdown timeout set to 10 seconds
[ 1.508056] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 1.508064] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 1.508125] rtc_cmos 00:03: RTC can wake from S4
[ 1.510634] rtc_cmos 00:03: registered as rtc0
[ 1.510811] rtc_cmos 00:03: setting system clock to 2023-01-17T21:13:43 UTC (1673990023)
[ 1.510831] rtc_cmos 00:03: alarms up to one month, y3k, 242 bytes nvram, hpet irqs
[ 1.511021] device-mapper: ioctl: 4.43.0-ioctl (2020-10-01) initialised: dm-devel@redhat.com
[ 1.511216] intel_pstate: Intel P-state driver initializing
[ 1.512287] nvme nvme0: 12/0/0 default/read/poll queues
[ 1.513131] intel_pstate: HWP enabled
[ 1.513236] NET: Registered protocol family 17
[ 1.514908] IPI shorthand broadcast: enabled
[ 1.514913] AVX2 version of gcm_enc/dec engaged.
[ 1.514915] AES CTR mode by8 optimization enabled
[ 1.515431] nvme0n1: p1 p2
[ 1.517811] sched_clock: Marking stable (1508127610, 9668699)->(1519271261, -1474952)
[ 1.536617] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input5
[ 1.727812] fbcon: i915drmfb (fb0) is primary device
[ 1.741127] Console: switching to colour frame buffer device 240x67
[ 1.767843] i915 0000:00:02.0: [drm] fb0: i915drmfb frame buffer device
[ 1.768423] Freeing unused kernel image (initmem) memory: 868K
[ 1.768429] Write protecting the kernel read-only data: 14336k
[ 1.769679] Freeing unused kernel image (text/rodata gap) memory: 2044K
[ 1.770503] Freeing unused kernel image (rodata/data gap) memory: 1896K
[ 1.770506] Run /init as init process
[ 1.770509] with arguments:
[ 1.770511] /init
[ 1.770513] with environment:
[ 1.770514] HOME=/
[ 1.770516] TERM=linux
[ 1.770518] intel_iommu=igfx_off
[ 1.771946] [U] hello world
[ 1.777368] [U] mount: mounting devpts on /dev/pts failed: No such device
[ 1.938237] tsc: Refined TSC clocksource calibration: 1608.011 MHz
[ 1.938241] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x172db496b93, max_idle_ns: 440795228640 ns
[ 1.938327] clocksource: Switched to clocksource tsc
[ 2.091921] ehci_hcd: USB 2.0 ‘Enhanced’ Host Controller (EHCI) Driver
[ 2.103998] ehci-pci: EHCI PCI platform driver
[ 2.136750] xhci_hcd 0000:00:14.0: xHCI Host Controller
[ 2.136762] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 1
[ 2.137950] xhci_hcd 0000:00:14.0: hcc params 0x200077c1 hci version 0x110 quirks 0x0000000000009810
[ 2.137959] xhci_hcd 0000:00:14.0: cache line size of 64 is not supported
[ 2.138521] hub 1-0:1.0: USB hub found
[ 2.138546] hub 1-0:1.0: 12 ports detected
[ 2.138971] xhci_hcd 0000:00:14.0: xHCI Host Controller
[ 2.138978] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 2
[ 2.138984] xhci_hcd 0000:00:14.0: Host supports USB 3.1 Enhanced SuperSpeed
[ 2.139166] hub 2-0:1.0: USB hub found
[ 2.139183] hub 2-0:1.0: 6 ports detected
[ 2.139343] usb: port power management may be unreliable
[ 2.474194] usb 1-1: new full-speed USB device number 2 using xhci_hcd
[ 2.750550] usb 2-6: new SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 2.898216] usb 1-2: new high-speed USB device number 3 using xhci_hcd
[ 3.182204] usb 1-3: new full-speed USB device number 4 using xhci_hcd
[ 4.148520] EXT4-fs (nvme0n1p1): mounted filesystem with ordered data mode. Opts: (null)
[ 4.202451] random: tpm: uninitialized urandom read (20 bytes read)
[ 4.579677] random: shred: uninitialized urandom read (312 bytes read)
[ 4.579726] random: shred: uninitialized urandom read (312 bytes read)
[ 5.014932] EXT4-fs (nvme0n1p1): re-mounted. Opts: (null)
[ 5.037336] EXT4-fs (nvme0n1p1): re-mounted. Opts: (null)
[ 39.284163] random: crng init done
[ 39.284165] random: 7 urandom warning(s) missed due to ratelimiting

Thanks for all the detailed info @ullrich, yes I would go ahead and reset / re-sign. Based on the dmesg output it doesn’t look like filesystem corruption or a hardware issue, so you should be fine to reset / re-sign.

Thank you! I have reset the TPM and re-signed /boot and now I‘m back to normal.

I got confused because because I did not expect this. In my case it would have been helpful if this procedure, which seems normal and neccessary to me now, had been announced more clearly.

1 Like