Anyone experience an issue with the latest update?

sdb1 is the /boot partition so should not be changed unless you are going to be re-imaging the entire partition.

So the entry for the /boot partition in fstab should also not be changed.

2 Likes

Copy that. I can change that part of the file back. Thanks for the quick reply. Guess it wasn’t as obvious to me afterall!

And we come to the moment of truth…

Files are there, time to unmount all the phone stuff and see what happens!

Closer. At least one problem is fixed. I could now type in the crypto password. Then the Librem 5 screen came up, and it said cryptsetup: crypt_root: set up successfully

That is further than it was getting, but then it hangs there and eventually goes dark. Looks like I had more go corrupt on that update than just one or both of those files. Hmm…

Can you post the contents of the /etc/cryttab and /etc/fstab files? Just to be sure those are correct.

The crypt_root successful message means that it has got as far as unlocking the LUKS container but if the UUIDs in the crypttab or fstab are not correct then it will just sit on a blank/black screen.

Also, I’m guessing from the out you have posted that you downloaded build 10988?

Sure. Crypttab:
crypt_root UUID=b67e4249-2c7c-4e8d-999d-f3c418cd2c9b none luks,discard,keyscript=/usr/share/initramfs-tools/scripts/osk-sdl-keyscript,initramfs

fstab:
proc /proc proc defaults 0 0
UUID=30f84051-7df0-4986-9c56-4401529be1ec / ext4 errors=remount-ro 0 1
UUID=283a4739-85fc-44b7-8ee0-e0e4c328ab42 /boot ext2 errors=remount-ro 0 2

And yes, I did use build 10988, since I figured it would make for a safer follow along.

You have the wrong UUIDs in these files, the /etc/crypttab file should have the following contents…

crypt_root UUID=040fa73a-aa27-458e-a8cc-7fba8ced031c none luks,discard,keyscript=/usr/share/initramfs-tools/scripts/osk-sdl-keyscript,initramfs

/etc/fstab should have the following contents…

proc /proc proc defaults 0 0
UUID=f8d74bc4-a994-4f5c-86b0-34f4929e4d2f / ext4 errors=remount-ro 0 1
UUID=283a4739-85fc-44b7-8ee0-e0e4c328ab42 /boot ext2 errors=remount-ro 0 2

Sorry. That was stupid. I showed the backup file contents that I made in case I screwed everything up. What they really contain:
Crypttab:
crypt_root UUID=040fa73a-aa27-458e-a8cc-7fba8ced031c none luks,discard,keyscript=/usr/share/initramfs-tools/scripts/osk-sdl-keyscript,initramfs

fstab:
proc /proc proc defaults 0 0
UUID=f8d74bc4-a994-4f5c-86b0-34f4929e4d2f / ext4 errors=remount-ro 0 1
UUID=283a4739-85fc-44b7-8ee0-e0e4c328ab42 /boot ext2 errors=remount-ro 0 2

I know… hard to help the helpless!

These files look better and correct.

You may be correct that there is some corruption on the root filesystem also, it might be worth running an fsck on the root filesystem. I am also unsure what other files, if any, from the /boot drive would be needed to fully boot? The original initrd.img and vmlinuz files appeared OK and to be the correct size according to ls so I’m thinking there could be other needed files in the /boot drive that are showing to be good (according to ls) but may be corrupt.

For running fsck, you’ll need to reopen the LUKS container (assuming it comes back up as sdb when reconnected via Jumpdrive)…

cryptsetup open /dev/sdb2 l5disk

Then run fsck

fsck -t ext4 /dev/mapper/l5disk

If it doesn’t produce any output, try…

echo $?

To get the exit status/code

2 Likes

Oddly enough when I used the tune2fs (at least I think it was when I ran that), it told me that I had mounted that disk too long (it preferred a fresh mount… don’t we all?), and it wanted me to run fsck. It was probably because I had mounted it when my computer auto-asked me. Then dismounted. Then mounted again.

When I ran fsck, it gave me two things it thought it had fixed. I forget exactly what they were, but they didn’t sound too critical. Something like it was using up too much space? I don’t remember. I looked, but couldn’t find the exact message again. —Free blocks count wrong? I think that’s what it was.

I’ll try to go through the process again tomorrow or something, and maybe copy over a couple other files too, just in case. I should be plenty faster at it by know.

Thank you immensely for all of your help. I’m pretty sure if my problem were only those two files, it would have been solved, but it must just be running deeper.

It is quite often the case that getting over one issue unmasks another.

When you get back to it, check if there is a “lost+found” directory in your root filesystem and what, if anything, it contains.

On the plus side, you have made some progress, as you can now type the crypto passphrase on screen it rules out a hardware failure. And, as you can open the LUKS container and mount the root filesystem on your desktop machine it’s not completely disastrous. My mind is still leaning a little more towards the possibility that there are other files on the /boot drive that are probably not right more than anything else.

I’ll be interested to hear how you get on.

3 Likes

Woke up a 5 am, so I went back through the steps. Ran fsck again and it again found Free blocks and Free inodes counted wrong. But supposedly fixed it. This time I just copied the System.map… and boot.scr file and got back out. Unmounted, booted up, and it worked!

Thank you, thank you, thank you, Loki.

My phone works, and it was oddly entertaining learning experience!

4 Likes

Good job, that’s great news and good to hear.

I would be a little concerned about the inode count discrepancy and is something that probably should be investigated a little further.

It also wouldn’t hurt, particularly as you want to avoid starting fresh, to take an image of the phone’s full disk while it’s in a known good state.

I second this.

I’m just booting Jumpdrive and using dd ... | gzip ... on the host computer for this. I haven’t been rigorous in finding the fastest options - but gzip was to a remote file on the local network anyway and the host computer is not superfast and it was still fast enough for my purposes (under 10 minutes).

I took a day off, since I was happy just to use my phone again, but now I kind of want to investigate a little more. I noticed something weird. Before the issue, my phone had been giving me the “Low Disk Space on ‘Filesystem root’” error, so I tried to do a pretty good cleanup. I followed the instructions on how to purge old kernels and such, and thought all went fine, but I still got that message (albeit with a bit more space). So when I updated, and I experienced failure similar to others’ failures caused by running out of space, I assumed that was it. However:

Which I believe is probably mostly the case. Ok, well Loki was able to help me get my phone back up and running. Awesome. A big part of it was replacing the two same files that fixed the original problem - but it just had to be done in a much more complicated manner.

I did notice though that I am still getting the error message: “Low Disk Space on ‘Filesystem root’” and it thinks I only have 125.4 MB of space. However, df claims I have 341,248. I find this odd. Perhaps the fact that df thinks I have enough space, but the message system thinks I don’t have enough space, is related to my previous issue that cause failure after the update. i.e. the fix didn’t work on my phone because of this anomaly. It did manifest the same way, and if the way it checks for space is df or similar, maybe it just didn’t realize I didn’t have enough space. Y’all have thoughts on this? Including @dos?

So now I’m a bit apprehensive to apply the update that just came out, since I have conflicting stories about how much room I have. So I’ll be seeing if I can fix that next.

Are you talking about the root file system (which is /) or the /boot file system. In the root file system you should have some Gigabyte free disk space:

purism@pureos:~$ df -kh /
Filesystem                                              Size  Used Avail Use% Mounted on
/dev/disk/by-uuid/30f84051-7df0-4986-9c56-4401529be1ec   29G   11G   17G  38% /
purism@pureos:~$ df -kh /boot
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p1  451M   94M  334M  22% /boot
3 Likes

Yeah, good comment. I read and wrote root. That’s what the error said. In my mind I thought boot, because of course I “knew” that boot was the smaller one that could fill up and give me issues. So I never checked the large partition. Like you said, there should be gigs of free space.

So I ran df again and looked back where I probably should’ve noticed before. /dev/dm-0 (my root) had just a bit over 120MB of room. So the phone was not lying to me. But how was that possible. Total space 3,783,616 total, with most of it used. Didn’t make sense to me, since like you said, it was supposed to be 30G or so (and why I never looked at it to begin with).

So I checked out the partitions…

And there it is. Someone short-changed me on my partitions when the phone was setup. So as @guru noted this IS where I was running out of room (but didn’t think to check, because I expected plenty). And it’s why when I tried to make more on the boot partition it didn’t help.

I’d guess that running out of room on the root partition is what caused the initial fail after update? And why the initial fix for that problem didn’t catch it - if the fix just verifies the /boot partition for sufficient space, but space is needed on the /root as well?

But hey - now it seems I just have to expand a partition to prevent the issue from recurring. And to get some more storage space. Thanks all. Let my egregious oversight serve as a warning to all.

1 Like

If you installed the LUKS variant of byzantium from scratch then I believe that is known, intentional and annoying behaviour. You are expected to expand the partition immediately after the install.

1 Like

So, until this last journey into mounting a non-native image to copy over files, the only changes I’ve made to the primary software package are the updates - once manually, but the rest through the Purism GUI. And I haven’t done any manual re-partitioning. But seeing the LUKS partition sure does make it seem like that’s what is on my device. But I’m supposing the LUKS variant doesn’t go in on one of the updates, so it would have to have been a “factory” install choice.

1 Like

IMO, this is most likely related to the fact that you powered on, for the very first time, your Librem 5 with the almost empty BPP-L503 battery and your Librem 5 turned off without allowing to PureOS to auto-expand its related partitions? After recharging your phone battery you continued using it without “by itself” expanded partitions, without taking care about data space available to/after your specific case “factory” PureOS installation.

1 Like

This is misleading information. Users are expected and encouraged to check if auto-expanding of two PureOS partitions happened (if ever such non-expanding of partitions thing happens, with initially turning on Librem 5 under its original power supply connected, for example). In other words, regularly and usually there is no need to expand partitions by any new Librem 5 phone user.

1 Like

That would make sense then. Any reason not to use the native “Disks” GUI preloaded on the LIbrem 5?

1 Like