Customers who have chosen to receive Dogwood should have their Dogwood shipments go out today. But wait, you might say, didn’t you post that Dogwood was already “finalizing testing before shipping to those who are part of this batch” a few weeks ago?
It’s true, at the time of that post we had started our final burn-in testing and started contacting Dogwood backers to confirm their current shipping address and also to give them an opportunity to switch to Evergreen if they wanted. It took a week or more of back-and-forth emails until we started getting responses settled.
In parallel with contacting customers we prepared each Dogwood device and performed a 96 hour burn-in test to identify any problematic hardware before we would ship it out. During this burn in we started to discover that devices were hard shutting down out of the blue somewhere around the 5-6 hour mark on average (although some lasted longer than a day before this happened). We then started an incredibly extensive set of testing across a fleet of Dogwood devices to troubleshoot the issue.
This has proven to be a really tricky issue to troubleshoot, first because of the long debug cycle (to test whether a fix works, you have to demonstrate a device can run for multiple days w/o a fault). Over the past few weeks we’ve gone down a number of false paths in the troubleshooting process and each time we thought we had tracked down a cause, further tests disproved it.
Up to this point we are still researching whether this is an actual hardware fault and if so, if it is something that can be corrected/mitigated in software, of if it’s a software configuration issue, such as if it’s being triggered by incorrect voltage or clock settings that need to be changed for the new Dogwood hardware. As of the time of this post, while we have tightened the debug cycle in terms of triggering the bug, and conversely have found settings and tweaks that make the hardware more stable, we haven’t yet tracked down the root cause or fix.
As I mentioned, in parallel we had been getting responses back from customers who still wanted Dogwood. Since we have not yet tracked down the cause of this hardware issue, we didn’t want Dogwood customers to be surprised by the flaw. We went through that list and contacted them again to inform them of this bug and advised them that:
If you are a developer wanting the Librem 5 to hack on, test, write software, submit an App to PureOS, these issues are likely not going to impact you too significantly.
If you are desiring Dogwood to be mass-production quality like Evergreen and be your daily driver without hardware issues, this will impact you significantly.
Please let us know which you prefer:
A. Ship Dogwood with known issues now!
B. Move your order to the front-of the line for Evergreen.
So after another round of back-and-forth emails last week into the weekend, we now have a list of customers who still want Dogwood and are shipping those out today.
As someone who does use a Dogwood device as a daily driver, for me personally I tend to mostly hit this flaw when I wake up in the morning and pick up the device to listen to podcasts. Sometimes the shutdown has happened overnight and sometimes not. For the most part in my case, at least, I tend to use my own phone throughout the day without hitting the flaw. So at least in my case, it’s been annoying but not a showstopper.
Obviously we are working incredibly hard to get to the bottom of the issue and this issue will not be present in Evergreen.
But the fact that it is perhaps (to my mind) a hardware thing, doesn’t mean it isn’t fixable with software. Maybe something is not running within the specification because of some initial driver or firmware.