[Cross-]Compiling for the Librem5?

Has someone successfully cross-compiled something for the Librem5? Often when I try to compile something on the phone itself it either takes all day at best, or crashes due to insufficient memory. Let’s suppose this is a CMake/C++ application. The relevant documentation is intimidating to say the least, especially as I have zero knowledge about CMake. It mentions preparing a toolchain file. Is this a once-only task which I can then use for all my cross-compilation tasks? (If so, could someone more experienced share it?) If I need binaries like libraries or something during compilation, might it be easier to apt-get them on the phone and transfer them to the computer (compared to trying to obtain them directly on the computer)?

Option 2: as I guess compilation fails due to insufficient memory, could I make a large swap file and compile overnight on my phone? Would that work?

Option 3: @dcz mentioned using qemu instead of bothering with cross-compilation. Could/should I run the Librem 5 image for that? And then I can just set it up to have a ton of my computer’s memory?

Option 4: My Raspberry Pi 4 has a bit more memory than my Librem 5. Would it work if I compile on the Pi and copy the resulting files to my phone? Both are aarch64. But I don’t know if library versions are close enough.

1 Like

One of the things that I compiled on my Librem 5 was the Magic Set Editor 2, from here:

It looks about like this when not docked:

I also compiled the NetSurf browser to run on my Librem 5, although sometimes I have been forgetting to use it and was instead using Epiphany or Firefox because of the addicting power of senselessly running nonfree javascript to communicate with the nonfree world.

So, regardless of that, I figured I would bring this up because my experience seems to have been quite different from yours. When I wanted to build this application on the Librem 5, basically I downloaded the CMake project, then followed the instructions for building it. And this application is a very old C++ application with 17 years of commits in the repo, and because of its age and because I had never used wxwidgets before, I was stuck for a little while on compiling until I went and actually read through their C++ code and realized that they were using a raw C programming varargs concept in some way that modern g++ was rejecting. But I changed this to a variadic template and all the problems went away and the program compiled and just started working. I would assume that I would have had the same issue on any standard modern GNU/Linux computer that had a newer version of WxWidgets that no longer supported their old WxString varargs printf or whatever it was.

So, it’s not as if we don’t have to get our hands dirty to accomplish our goals sometimes, but I don’t think I encountered the kind of issues that you are describing… really at all. Essentially, I downloaded the C++ / CMake project with a basic git clone <whatever> from inside a folder like /home/purism/repos/<nameofproject> that I had created, and then I ran the basic cmake commands to compile that project. Are you trying to compile something immensely complex that requires an extraordinary amount of RAM just to compile? In my head, the Magic Set Editor 2 is a fairly complex program because of how it supports 20 years of different cards styles and settings and special font formatting for card cost symbols, and special autocomplete for the syntactical descriptions of powers and abilities. But I guess at the end of the day, it is a harmony of graphics and text like a word processor in a way. And, I did compile it while my device was docked so that I could treat it like a Linux PC rather than a handset.

But I don’t really recall it taking all that long to compile this. Maybe it took some time, but it felt reasonable to me at the time. Is your problem that building applications on the Librem 5 is taking too long? Or is it physically not working at all due to some error?

When I use cmake, per the instructions on the Magic Set Editor 2 example (which matches almost every other cmake project I ever used), I basically just cmake build -DCMAKE_BUILD_TYPE=Release or something similar to it, and by invoking cmake on command line then the project setup is created, and then afterwards we can invoke the project build such as by using make or ninja or whatever cmake was configured for. But in the case of the Magic Set Editor 2 they save us time by suggesting to have cmake pick for us by just using cmake --build as the second command to type that actually compiles the application.

So, that has all worked quite well for me.

Option 2: This might be physically possible but for these few projects that I compiled myself, I never felt the slightest need to make a swap file like that because 3 GB of memory seemed like it was usually enough to compile most things.

Option 3: I tried using qemu for the Librem 5 img a week or two ago for something I was doing and I kept doing it wrong. There must be a tutorial somewhere, and I didn’t look in the right place. So maybe someone else can help. But I managed to get a crappy ubuntu QEMU setup that was specifically running ubuntu aarch64, and it was miserable. The aarch64 emulation on a big PC with 64 GB of RAM and a Ryzen 7 2700x processor and an nVidia 2080 Ti 11GB card was abysmally slow, even when I told the aarch64 qemu configuration to use 8 cores and tons of RAM. Just getting to the point of booting took hours.

Option 4: Seems very project specific. Maybe you could try it and tell us what kind of problems you encounter? I tried dual-booting my Librem 5 into a second install of PureOS that was using landing instead of byzantium repos, and when I tried to compile the latest axolotl binaries in rust and then copy them back to the byzantium version of the Librem 5, it was a terrible idea and exploded everything. A lot of the shared libraries simply did not match across versions, so running a binary with at on of dependencies simply did not work. As you kind of suggested, after apt-get installing the necessary binaries in the landing version of PureOS mobile, I tried to physically manually copy them from /lib/aarch64-linux-gnu/<thing>.so from one to the other and what I ended up doing was corrupting the basic functionalities of my PureOS installation so that even web browsing did not work, and then I reinstalled PureOS byzantium elsewhere and copied again back to having the byzantium versions and got it working again but it was all very crazy. I think it’s cool that free software allows us to make mistakes like that, but also I think that doing that is a mistake.

I have setup chroot to build project to opt and Debian packages for Librem 5 years ago. I have that chroot even exported as NFS which would allow to boot real Librem 5 over NFS from the development environment. I have not done that yet with Librem 5 but it is standard setup which I use for many other GNU/Linux based system during development and which we use even for hundreds of student PCs at the university with centrally server diskless environment.

Some rough notes for the setup for simple development (done as root in my case, can be done in LXC or other jail environment too)

Preparation of chroot location

mkdir -p /srv/nfs/pureos-arm64/usr/bin
mkdir -p /srv/nfs/pureos-arm64/usr/sbin

# it is critical to ensure that setup of daemons in chroot
# system does not manipulate with state, networking of the main system
# change my_system to host system hostname
# the second  $(hostname) in the script should be replaced
# your actual system hostname
echo '#!/bin/sh\n[ "$(hostname)" = "'"$(hostname)"'" ] && exit 101\nexit 0' /srv/nfs/pureos-arm64/usr/sbin/policy-rc.d

Then I setup QEMU to run ARM binaries on x86 or other non Aarch64 system in the chroot. I install qemu-user or qemu-user-static, qemu-utils and qemu-user-binfmt. Then find package qemu-user-static and copy ‘/usr/bin/qemu-aarch64-static’ into ‘/srv/nfs/pureos-arm64/usr/bin’ then make symlink

(cd  /srv/nfs/pureos-arm64/usr/bin ; ln -s qemu-aarch64-static qemu-aarch64)

In a newer qemu-user-binfmt the QEMU user space handlers are served indirectly through ‘/usr/libexec/qemu-binfmt/’, so copy content of that directory into ‘/srv/nfs/pureos-arm64/usr/libexec/qemu-binfmt’. These steps may not be necessary on some setup, because binfmt handlers even in chroot finds correct one in the host system. But I have not investigated that part and use what I use for 20+ years already.

The you can debootstrap Pure OS, you need to add pureos-archive-keyring.gpg or some other details, and other details, if you have problem give me a note and I try to help

usr/sbin/debootstrap \
  --keyring=/srv/nfs/pureos-archive-keyring.gpg \
  --arch=arm64 \
  --include=debian-keyring,mc,libc6-dev,libstdc++6,busybox,aptitude,etckeeper \
  byzantium /srv/nfs/pureos-arm64 https://repo.pureos.net/pureos /usr/share/debootstrap/scripts/byzantium

You can get into troubles with some versions of Debian that they are too picky about checking if the ARM64 bit binaries are runable on the host system. They should be, because you have installed binfmt, but I have seen that check was incorrect and fails which prevents debootstrap to run in the mode when the second phase is run as native utilities calls and execution in populated /srv/nfs/pureos-arm64. I have use some trick including replacing test executable formats scripts by true in such cases. I hope, that it is solved in actual systems.

Then I run simply

chroot /srv/nfs/pureos-arm64/

So this is my setup for running full target system software and distribution managements tools (apt-get, etc.) on my development systems. There are many other ways, but I am not big fan of docker and other complex ones. I use LXC, so this is another way.

For simple CMake, autotools, meson etc based projects I often setup standard cross GCC and setup project to point into right tools and libraries etc. Some examples there GitHub - ppisa/cross-utils: Some set of scripts, configurations and tool which I use for cross-compilations. . This is much faster, because x86_64 build of the target architecture GCC is used. The the debootstrap environment obtained by above steps would help as source of the target system root, sysroot so you build against actual GLibc and rest of libraries found on real target.

3 Likes

Yes, this will work.

This will work too. And is the best option. The Raspberry pi4 is much faster, has more memory, and is less likely to overheat. You do not have to worry about libraries as the kernel runs stand-alone.

1 Like

This is cool. Unbelievable that there is a phone like the Librem 5 that can actually do this at all!

3 Likes

I would suggest making sure your rpi runs a Debian release similar to what’s on the L5 to ensure better chances of successfully running what you compile.

That means the newest Rpi OS just release this month is too new (based on Bookworm, which would be the equivalent of PureOS crimson, so you would need to use the previous release.

1 Like

You do not have to worry about libraries as the kernel runs stand-alone.

Yes, tdlib-purple. I wouldn’t think a headless messaging library would fit that qualifier but it consumes all my phone’s memory and then I have to long press the power button to regain access. Last night I tried on my Raspberry 4, and this morning the Raspberry was unresponsive, I had to cut power to make it reboot.

I set the -j1 option to use only one thread.

I guess I’ll go for option 2 and create a huge swap file. Good thing it’s winter over here, I’ll be able to heat my house all day during compilation :smiley:

3 Likes