Problems starting, systemd-unit not finishing

After my last update on 30th of September I started having problems starting PureOS. Startup seems to be indefinitely waiting for a job to finish.

[  *** ] A start job is running for /dev/mapper/luks-UUID (9min 47s / no limit)

I found that it is possible to enable a systemd debug shell by using the kernel parameter systemd.debug-shell and looked into the problem.

A bit difficult to use the debug shell since systemd writes its annoying playful screaming colourful messages to console and disturbs every second the screen output with its updates.

What I achieved is that the system continues to start once I just issued

swapon -a

Using ps and systemctl status /dev/mapper/luks-UUID.swap I didn’t find any output that would help to explain the issue.

Furthermore in one of ten boots the startup works just like expected.

I had the same issue a few times already. Rebooting the system once more so far “solved” it each time, but a more permanent solution would be great.

Does the journal show anything interesting related to swap? How about /var/log/syslog?

While testing stuff around systemd without rebooting my notebook all the time inside a kvm running Debian Buster I found that there is a time limit of 1 min 30 sec for swap in Debian Buster for unencrypted swap.

It does not solve the problem itself, but this way the system keeps on booting after waiting for the timeout.

I wonder if the timeout ‘no limit’ is a result of using encrypted swap or a patch by Purism?

No, not really. The journal after unsuccessful tries didn’t contain anything and neither when I just typed swapon -a in tty9 (debug shell of systemd) to continue booting.

I set systemd log_level to ‘debug’ and read through /var/log/syslog without really finding differences between successful and unsuccessful boots.

What I learned is that systemd generates unit files from /etc/crypttab and /etc/fstab, but didn’t find information, yet, if this might lead to a race condition between all those swap units running at startup.

While reading I found how to set in /etc/fstab a timeout for the unit and make it fail without the consequence of the boot process being broken. One just has to modify the options in the swap line to defaults,noatime,nofail,x-systemd.device-timeout=20. Timeout in this case is 20 seconds.

I found that my system now is booting every time - sometimes with swap activated and sometimes without swap.

Edit:

Now I can get messages using systemctl status on the unit:

● dev-mapper-luks\x2dabcdef00\x2dcd0d\x2d4d38\x2d9bee\x2d3c99e922066d.swap - /dev/mapper/luks-abcdef00-cd0d-4d38-9bee-3c99e922066d
   Loaded: loaded (/etc/fstab; generated)
   Active: inactive (dead)
     What: /dev/mapper/luks-abcdef00-cd0d-4d38-9bee-3c99e922066d
     Docs: man:fstab(5)
           man:systemd-fstab-generator(8)

Okt 02 00:01:11 hostname systemd[1]: Dependency failed for /dev/mapper/luks-abcdef00-cd0d-4d38-9bee-3c99e922066d.
Okt 02 00:01:11 hostname systemd[1]: dev-mapper-luks\x2dabcdef00\x2dcd0d\x2d4d38\x2d9bee\x2d3c99e922066d.swap: Job dev-mapper-luks\x2dabcdef00\x2dcd0d\x2d4d38\x2d9bee\x2d3c99e922066d.swap/start failed with result 'dependency'.
Okt 02 00:01:22 hostname systemd[1]: Dependency failed for /dev/mapper/luks-abcdef00-cd0d-4d38-9bee-3c99e922066d.
Okt 02 00:01:22 hostname systemd[1]: dev-mapper-luks\x2dabcdef00\x2dcd0d\x2d4d38\x2d9bee\x2d3c99e922066d.swap: Job dev-mapper-luks\x2dabcdef00\x2dcd0d\x2d4d38\x2d9bee\x2d3c99e922066d.swap/start failed with result 'dependency'.
Okt 02 00:01:32 hostname systemd[1]: Dependency failed for /dev/mapper/luks-abcdef00-cd0d-4d38-9bee-3c99e922066d.
Okt 02 00:01:32 hostname systemd[1]: dev-mapper-luks\x2dabcdef00\x2dcd0d\x2d4d38\x2d9bee\x2d3c99e922066d.swap: Job dev-mapper-luks\x2dabcdef00\x2dcd0d\x2d4d38\x2d9bee\x2d3c99e922066d.swap/start failed with result 'dependency'.
Okt 02 00:01:43 hostname systemd[1]: Dependency failed for /dev/mapper/luks-abcdef00-cd0d-4d38-9bee-3c99e922066d.
Okt 02 00:01:43 hostname systemd[1]: dev-mapper-luks\x2dabcdef00\x2dcd0d\x2d4d38\x2d9bee\x2d3c99e922066d.swap: Job dev-mapper-luks\x2dabcdef00\x2dcd0d\x2d4d38\x2d9bee\x2d3c99e922066d.swap/start failed with result 'dependency'.
Okt 02 00:01:53 hostname systemd[1]: Dependency failed for /dev/mapper/luks-abcdef00-cd0d-4d38-9bee-3c99e922066d.
Okt 02 00:01:53 hostname systemd[1]: dev-mapper-luks\x2dabcdef00\x2dcd0d\x2d4d38\x2d9bee\x2d3c99e922066d.swap: Job dev-mapper-luks\x2dabcdef00\x2dcd0d\x2d4d38\x2d9bee\x2d3c99e922066d.swap/start failed with result 'dependency'.

If systemctl status is showing info, you can get the same info, plus a bit more with journalctl -xe and pass the service unit itself to the journal; journalctl _SYSTEMD_UNIT=avahi-daemon.service

Thanks a lot! After the next boot that ends up without swap I’ll try and get back.