Bootloader failure - Insufficient battery level to power on

Hardware Librem 5
1

I.
I’ve been dealing with an error, described in the Librem 5 docs, where the phone won’t boot when the battery level is insufficient.

I’m guessing here, but I think the cause is that my L5 may have been running when I thought I had completely powered it off. [And I had just fully charged it prior to that.] This is not the first time that has happened. I’ve never seen the flashing green LED before, but I recently updated the battery firmware from an incorrect version, so maybe that’s why I now get the warning LED.

I followed the recovery instructions in the docs, but still couldn’t boot, even after many hours of trickle charging. I eventually triedjumpdrive to see if I could mount the phone’s internal storage on my laptop, and succeeded… but only for a few seconds before it went blank.

I then reconnected the phone to an electric outlet and got it to start charging normally again.

II.
I want to prevent any near-total battery drain from happening in the future, so here’s what I’ve done… and @dos or anyone, please chime in if you think this is a bad idea:

First, I made a backup of /etc/UPower/UPower.conf, then made a couple of adjustments to the default settings:

purism@pureos:~$ sudo nano /etc/UPower/UPower.conf

Default version:

# Only the system vendor should modify this file, ordinary users
# should not have to change anything.

[UPower]

# Enable the Watts Up Pro device.
#
# The Watts Up Pro contains a generic FTDI USB device without a specific
# vendor and product ID. When we probe for WUP devices, we can cause
# the user to get a perplexing "Device or resource busy" error when
# attempting to use their non-WUP device.
#
# The generic FTDI device is known to also be used on:
#
# - Sparkfun FT232 breakout board
# - Parallax Propeller
#
# default=false
EnableWattsUpPro=false

# Don't poll the kernel for battery level changes.
#
# Some hardware will send us battery level changes through
# events, rather than us having to poll for it. This option
# allows disabling polling for hardware that sends out events.
#
# default=false
NoPollBatteries=false

# Do we ignore the lid state
#
# Some laptops are broken. The lid state is either inverted, or stuck
# on or off. We can't do much to fix these problems, but this is a way
# for users to make the laptop panel vanish, a state that might be used
# by a couple of user-space daemons. On Linux systems, see also
# logind.conf(5).
#
# default=false
IgnoreLid=false

# Policy for warnings and action based on battery levels
#
# Whether battery percentage based policy should be used. The default
# is to use the time left, change to true to use the percentage, which
# should work around broken firmwares. It is also more reliable than
# the time left (frantically saving all your files is going to use more
# battery than letting it rest for example).
# default=true
UsePercentageForPolicy=true

# When UsePercentageForPolicy is true, the levels at which UPower will
# consider the battery low, critical, or take action for the critical
# battery level.
#
# This will also be used for batteries which don't have time information
# such as that of peripherals.
#
# If any value is invalid, or not in descending order, the defaults
# will be used.
#
# Defaults:
# PercentageLow=20
# PercentageCritical=5
# PercentageAction=2
PercentageLow=20
PercentageCritical=5
PercentageAction=2

# When UsePercentageForPolicy is false, the time remaining in seconds at
# which UPower will consider the battery low, critical, or take action for
# the critical battery level.
#
# If any value is invalid, or not in descending order, the defaults
# will be used.
#
# Defaults:
# TimeLow=1200
# TimeCritical=300
# TimeAction=120
TimeLow=1200
TimeCritical=300
TimeAction=120

# The action to take when "TimeAction" or "PercentageAction" above has been
# reached for the batteries (UPS or laptop batteries) supplying the computer
#
# Possible values are:
# PowerOff
# Hibernate
# HybridSleep
#
# If HybridSleep isn't available, Hibernate will be used
# If Hibernate isn't available, PowerOff will be used
CriticalPowerAction=HybridSleep

Then I made the following changes (in bold):

PercentageLow=20
PercentageCritical=15
PercentageAction=13

and:

CriticalPowerAction=PowerOff

===
Hopefully that will preserve at least enough charge for booting, by shutting down before the battery drains to a critically low level.

1 Like
2

The flashing green LED is done by u-boot whenever the battery level is below 3%. It can be forced to boot anyway by simply pressing the power button again while it’s flashing, or by simply plugging a charger in.

1 Like
3

Good to know about the second press. I’ll do that if it ever happens again.

Plugging it in after the flashing LED didn’t produce any visible sign that it was charging, even after hours and hours. It wasn’t warm to touch, either.

4

It’s still a mystery how my battery level got so low while theoretically powered off, and just 1 to 2 days after a full charge.

5

It is not better like “blinking” than “flashing”? while flash is incorrectly used for flashing to nand-memory despite it is for ram. imho. I just saying.

6

Hah hah. Fair point. English is a sucky language.

The official documentation says “flashing” (about the LEDs) so let’s stick with “flashing” and accept the inherent ambiguity.

1 Like
7

Seems a reasonable change.

It would be interesting to know what state the phone was in.

Do you (still) have the audio charge reminder running? If so, at what percentage level?

8

English just CRASH from Spanish.

9

Yes, at 40% still, and it’s functional.

But in the present situation, to all appearances, the phone was off and dark, so I hear no auditory warning. And I think I would hear it, because I’m near the phone most of the day and into the night.

I’m inclined to think the phone is sometimes going into some sort of in-between state in which some process is still running even though it appears to be powered down. I do often try to verify the “off” state by v.e.r.y briefly pressing the power button to make sure the screen doesn’t come on. Maybe I should avoid doing that.

P.S. And actually, I wonder if the adjustment I made above will work if the phone “thinks” it’s already off. :rofl:

1 Like
10

Those terms are pretty much interchangeable in English, at least when referring to lights. But yes, “flashing” has several different meanings when talking about computing.

1 Like
11

No “running processes” there, but there are some “in-between states” to consider indeed.

Kernels before 6.12.75 could occasionally hang on resume from system suspend when attempting to reset shdci after a failed tuning process. The phone would appear off - not reacting to any inputs, other than perhaps red LED when plugged in - but still be in fact on until you long-pressed the power button to actually turn it off.

Are you sure the phone was actually off, as in - you did order it to shut itself down? Or have you just grabbed the phone, checked if it’s on and assumed it was off after no reaction to a power button press?

Another thing is that there’s a difference between system “halt” and “power off”. When you halt the phone, it appears off - the kernel is stopped, the display backlight is off, but the SoC is still powered on. Many PCs either power themselves off when halting, or make it obvious that it’s still on when merely halted (by keeping the logs displayed on the screen, for example; or continuing to produce fan noise, shining power LED etc.), so some people may be used to turning the system off by typing “halt”. On PureOS, we configure systemd to always force power off even when the user requested a halt, but other distros may not configure it this way.

And the last thing is that there’s a difference between powering the system off and having it unplugged from power. When the SoC is off, but its circuits remain powered, it will still consume some power and so will all the resistors and other components on the board that remain on powered paths. To minimize this power drain, we are using the “shipping mode” feature of the charging controller to effectively cut the battery power out from the rest of the system, keeping only the controller itself, battery gauge chip and RTC on. Shipping mode is enabled in PureOS a few seconds after orderly shutdown, there are however some cases when it isn’t being engaged and the phone will continue to drain some non-insignificant power while “off”:

  • forcing a shutdown by long-pressing the power button
  • keeping USB-C plugged in during proper shutdown and unplugging it afterwards without turning the phone back on
  • having the phone be turned off by osk-sdl (either by a short power button press or input inactivity on LUKS passphrase prompt)

The last two are going to be fixed in PureOS soon-ish, but the first one is trickier.

However, I wouldn’t expect the battery to be drained so fast to be empty after two days if it was off without shipping mode engaged. I’d rather expect something like a week or two.

2 Likes
12

Thanks for those particulars!

I always power off from the menu, and I never leave it connected to anything, unless I’m charging it, after which I disconnect it.

I’ve only ever used the button long-press to power off when there was some sort of application hang and the phone appeared to be frozen when I wanted to shut it down. That hasn’t happened in a long time, though. When it did happen, I always powered on again, then shut down via the menu.

13

I might have done this sometimes. (But unplugging within a couple of seconds after shutdown.)

14

It checks the state right before the actual shutdown, so that’s the moment that matters.

1 Like
15

Thanks. I’ll be careful with that now.

16

The problem with shipping mode is that it cuts the battery from the power path, and this includes charging. So if we just enabled shipping mode on shutdown all the time, it would prevent it from charging while off, but if we don’t do it at that point, then there’s nobody to enable it once the cable is unplugged as the system is already off.

I have recently made a PoC to have the phone pretend to go off by suspending during shutdown when plugged in and only actually powering off once it’s unplugged, resulting in shipping mode engaging in all (non-crashing) cases, but it still needs some polishing: WIP: Ship mode 2.0 (!392) · Merge requests · Librem5 / librem5-base · GitLab

2 Likes
17

Very occasionally, and not so much recently, I’ve seen desktop Linux fail to shut down, where quite a lot of things do terminate (so, for example, cron jobs might no longer run and maybe networking is shut down so it won’t be pingable on the network) but something won’t shut down properly so the shutdown process never completes. (On desktop you can sometimes press Esc and see a text screen that will show you where it’s up to and maybe give clues about what is hung.)

I would avoid doing that until well after a normal successful shutdown should have completed.

Definitely a legitimate question. If the implementation is in the phone itself (which I would guess to be the case) and the implementation uses a service (untested assumption) and the service has already been terminated by the shutdown but then shutdown hangs … you may well be right to wonder. I think it would be a case of gathering evidence on that one.

Is there any definitive sign that “critical power level shutdown” has occurred? and if so when? e.g. something in the system journal? I guess you could test that by setting those three values to much higher e.g. 50% for Action and letting it shut down due to “low” power.

1 Like
18

Result:

I’m not sure if my solution works or not.

I allowed the battery to drain down to the mid-20% range, then I powered it off from the shudown menu and let it sit for a few days. Yesterday, as a test, I did a brief tap of the power button to verify the power-off state, just to add my established habit to the test.

Today I powered it on again, expecting to see either a charge level close to the mid-20% where I had left it, or approximately 13%, according to my configuration change.

Instead, the charge was down to 7%, with a notification warning that battery was “empty,” at which point I plugged it in and started recharging.

So, I conclude that either my config change didn’t help, because the phone really was off, or the age of my battery (over 5 years old) is having an effect on the residual charge, accelerating a sudden final drain anyway, or some combination of factors, or something else entirely.

19

Indeed. But it was never clear what the result was going to be because it was never clear what the state of the operating system was after you “powered off”.

At least you can easily obtain a new battery from Purism (if that’s what the problem is).

Here’s a further test that you may or may not want to do …

after powering it off at “mid-20% range”, remove the battery.

A couple of scenarios:

  • if the phone has not correctly powered off and the disks have not been unmounted then there could be observable symptoms when you next power on (unclean umount / fsck)
  • if the phone has not correctly powered off but the disks have been unmounted then it will of course crash the operating system but there may be no observable symptoms of that
  • if the phone has correctly powered off then the phone should be fine with the removal of the battery

Then you reinsert the battery before powering on the next day. (This is not convenient as an ongoing workflow. It is just for investigation.) What percentage does the battery show after powering on? Is it the same kind of result that you are reporting in your post or different?

In conjunction with that, if you have a multimeter, you could measure the battery voltage immediately after removal and immediately before re-insertion.

1 Like