Librem One Seems to Be Down

Neither I nor my spouse can receive or send Today. Last message received at 9:35EST, second time this week. New sending error however, haven’t seen this one before:

Sending of the message failed.
An error occurred while sending mail. The mail server responded: tracy@librem.one: Temporary lookup failure. Please verify that your email address is correct in your account settings and try again.

It would be nice if https://status.librem.one/ had some automated checks, and didn’t look like a static page last manually updated in 2020. It would help narrow down if this was likely about issue with the account or with the service.

2 Likes

It looks to be partially down for me as well:

𐄂 Librem Chat (…well, I’ve been using Element) has not connected to any rooms for me for ~3 days
𐄂 I receive a server error (500) when trying to recover a Librem One password.
✓ I have been able to receive email

I wonder if anyone has contacted support. On average I see very little forum activity over the weekends, and even less from Purism employees over weekends.

I know this started as a status check, but a more official response might be more beneficial even if it’s an account issue and not a system issue.

I sent them an email af few hours back, muaa, ha ha ha! (I used gmail of course.)

1 Like

I cannot login to my Social/Mastodon. But where I already is logged in on another device, so Social itself seems to work.
I am offline on Chat/Matrix.
I know (from Twitter) Purism team is aware of issues.

@Kyle_Rankin can we get this kind of status update on the status page please? Not getting those updates there really devalues the page

Linking to the Twitter thread. Thanks for pinging them on Twitter, Henrik!

I just tried to send an email, and it errored out. So yes, it seems like a couple of their services (Chat, Mail) are down.

Librem Social (Mastodon) is working for me.

1 Like

This puts a wrench in services that use 2 factor authentication to let you in and your librem one email is the only method listed. Will have to change all my accounts to something else more reliable once this is up. Lucky this happened on a weekend else I’d cancel this service entirely and I’d have to print new business cards.

(And I woke up at 3AM EDT to post this because I couldn’t sleep.)

Ten minutes after the above last response the backlog of email started rolling in.

Librem Chat seems to be back up as well!

Good luck to Purism on getting Librem Chat ready for version upgrades :slight_smile:

My understanding is that the downtime we had was limited to the Matrix server. We have been trying for months to upgrade, and ultimately had to bring on a DBA to troubleshoot why database migrations during upgrades were taking days and never completing. In any case, as part of this troubleshooting I believe we ran a few database maintenance steps that locked the Matrix database, taking that service down. This would be unrelated and not connected to email or other services though, and Social was unaffected as well, so if customers had issues with email that would have been something separate and unrelated as far as I know.

In any case, my hope is that soon we will be in a position where we can (finally) upgrade Matrix. As anyone who has managed Matrix knows, the protocol frequently creates backwards-incompatible updates that will break either older servers or older clients. Managing a Matrix server in production means trying to maintain a balance between supporting older clients from users who may not be ready to upgrade, and updating servers so that backwards-incompatible changes don’t isolate you from the rest of the Matrix network.

My hope is that Matrix will stabilize soon (I mean Matrix in general, not just our instance) so that future upgrades don’t break backwards compatibility.

5 Likes

How do I convert my purism account?
When I go to the link (Click here to switch to Librem One for free) nothing happens.

Contact support.

2 Likes

Might I suggest a move to XMPP? The kitchen sink already exists and is rock solid stable.

Matrix just feels like an exercise in bloat.

1 Like

Otherwise the Matrix may have to summon Neo.