Performance: Compare Librem 5 to Ubuntu Touch devices

This app lets you compare the performance of different Ubuntu Touch devices by using sysbench: https://open-store.io/app/ut-sysbench-qt-gui
Should some of you have a Librem 5, it would be nice, if you could install sysbench (apt install sysbench) on your device and execute the following commands and share screenshots of the results:

sysbench --test=cpu --num-threads=2 --cpu-max-prime=20000 run
sysbench --test=cpu --num-threads=8 --cpu-max-prime=20000 run
sysbench --test=memory --num-threads=4 run
sysbench --test=mutex --num-threads=64 run

I would like to add the Librem 5 in this and other comparison charts and I am also interested in how the unusual hardware of the Librem 5 performs:

2 Likes

I got:

purism@evergreen:~$ sysbench --test=cpu --num-threads=2 --cpu-max-prime=20000 run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 2
Initializing random number generator from current time


Prime numbers limit: 20000

Initializing worker threads...

Threads started!

CPU speed:
    events per second:   595.77

General statistics:
    total time:                          10.0015s
    total number of events:              5962

Latency (ms):
         min:                                    3.35
         avg:                                    3.35
         max:                                    3.42
         95th percentile:                        3.36
         sum:                                19997.85

Threads fairness:
    events (avg/stddev):           2981.0000/0.00
    execution time (avg/stddev):   9.9989/0.00

purism@evergreen:~$ sysbench --test=cpu --num-threads=8 --cpu-max-prime=20000 run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 8
Initializing random number generator from current time


Prime numbers limit: 20000

Initializing worker threads...

Threads started!

CPU speed:
    events per second:  1190.77

General statistics:
    total time:                          10.0042s
    total number of events:              11919

Latency (ms):
         min:                                    3.35
         avg:                                    6.71
         max:                                   24.63
         95th percentile:                       15.27
         sum:                                79976.39

Threads fairness:
    events (avg/stddev):           1489.8750/13.51
    execution time (avg/stddev):   9.9970/0.01

purism@evergreen:~$ sysbench --test=memory --num-threads=4 run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 4
Initializing random number generator from current time


Running memory speed test with the following options:
  block size: 1KiB
  total size: 102400MiB
  operation: write
  scope: global

Initializing worker threads...

Threads started!

Total operations: 56466248 (5643286.07 per second)

55142.82 MiB transferred (5511.02 MiB/sec)


General statistics:
    total time:                          10.0002s
    total number of events:              56466248

Latency (ms):
         min:                                    0.00
         avg:                                    0.00
         max:                                    2.00
         95th percentile:                        0.00
         sum:                                20776.56

Threads fairness:
    events (avg/stddev):           14116562.0000/10471.11
    execution time (avg/stddev):   5.1941/0.00

purism@evergreen:~$ sysbench --test=mutex --num-threads=64 run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 64
Initializing random number generator from current time


Initializing worker threads...

Threads started!


General statistics:
    total time:                          16.1842s
    total number of events:              64

Latency (ms):
         min:                                13175.41
         avg:                                15885.53
         max:                                16133.69
         95th percentile:                    16224.31
         sum:                              1016674.22

Threads fairness:
    events (avg/stddev):           1.0000/0.00
    execution time (avg/stddev):   15.8855/0.38
6 Likes

Are you posting the results somewhere?

Thank you really much, do the results come from the Evergreen model?

Yes, but there shouldn’t be any meaningful difference between batches.

2 Likes

I want to include the results in the next version of the app (https://open-store.io/app/ut-sysbench-qt-gui). It will look something like this for the test with two threads:

Comparison CPU Benchmark (2 threads):

Cosmo Commun. | 1652 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
M10 Tab + 2.gen | 1235 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Volla Phone | 1091 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Xperia X | 1035 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Pixel 3a | 918 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Librem 5 PureOS | 596 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Nexus 5 | 58 events/second
...β–ˆ
Nexus 4 | 57 events/second
...β–ˆ
Meizu MX4 | 52 events/second
...β–ˆ
bq E5 | 46 events/second
...β–ˆ

The results of the performed tests have been
copied to your clipboard πŸ“‹
(you can paste it into text fields
of other apps or websites).

If you want to, I could also post the charts of the results of other the three other tests here as soon as I implemented it for the app (this can take some time). But you could also take a look at this posted screen shots, if you already want to have the data for other devices: https://forums.ubports.com/topic/6750/ut-sysbench-qt-gui-compare-cpu-performance-of-different-devices

3 Likes
Comparison CPU Benchmark (2 threads):

Xiaomi Mi A2 | 1717 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Cosmo Commun. | 1652 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
M10 Tab + 2.gen | 1235 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Volla Phone | 1091 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Xperia X | 1035 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Pixel 3a | 918 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Pinephone | 620 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Librem 5 (PureOS) | 596 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Nexus 5 | 58 events/second
...β–ˆ
Nexus 4 | 57 events/second
...β–ˆ
Meizu MX4 | 52 events/second
...β–ˆ
bq E5 | 46 events/second
...β–ˆ


Comparison CPU Benchmark (8 threads):

Cosmo Commun. | 5287 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Xiaomi Mi A2 | 4915 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
M10 Tab + 2.gen | 4005 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Pixel 3a | 3973 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Volla Phone | 3788 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Xperia X | 2467 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Librem 5 (PureOS) | 1191 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Pinephone | 1190 events/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Meizu MX4 | 146 events/second
...β–ˆ
Nexus 4 | 113 events/second
...β–ˆ
Nexus 5 | 96 events/second
...β–ˆ
bq E5 | 91 events/second
...β–ˆ


Comparison Memory Benchmark (no armhf devices):

Xiaomi Mi A2 | 5849 MiB/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Librem 5 (PureOS) | 5511 MiB/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Pixel 3a | 2560 MiB/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
M10 Tab + 2.gen | 2443 MiB/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Cosmo Commun. | 2300 MiB/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Volla Phone | 2250 MiB/second
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Pinephone | 815 MiB/second
...β–ˆ
Xperia X | 795 MiB/second
...β–ˆ


Comparison Mutex Benchmark (less time is better):

Librem 5 (PureOS) | 16.1842 seconds
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Meizu MX4 | 3.8107 seconds
...β–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
Nexus 5 | 2.7758 seconds
...β–ˆβ–ˆβ–ˆ
Nexus 4 | 2.4609 seconds
...β–ˆβ–ˆβ–ˆ
bq E5 | 2.4057 seconds
...β–ˆβ–ˆβ–ˆ
Volla Phone | 1.8590 seconds
...β–ˆβ–ˆ
M10 Tab + 2.gen | 1.8361 seconds
...β–ˆβ–ˆ
Xperia X | 1.6159 seconds
...β–ˆβ–ˆ
Cosmo Commun. | 1.2297 seconds
...β–ˆ
Xiaomi Mi A2 | 1.1919 seconds
...β–ˆ
Pinephone | 1.0368 seconds
...β–ˆ
Pixel 3a | 0.9193 seconds
...β–ˆ
1 Like

Why is Mutex Benchmark so bad on L5? And whar are the consequences?

Really odd that the PinePhone is scoring better the Librem 5 in the Sysbench’s two thread CPU benchmark. Both @dos and I found that the L5 had better Sysbench CPU benchmarks than the PP. I suspect that something is slowing down your system, because the i.MX 8M Quad should be faster than the A64. See:


It would be interesting to run sysbench from the command line and see if you get the same results. Also which distro and kernel are you using on the L5?

By running
sysbench --test=cpu --num-threads=2 --cpu-max-prime=20000 run

I got the same (595 events/s) on my L5 evergreen, kernel5.16.0-1librem5, sysbench 1.0.20

@nico202, Can you rerun it this way to compare with what I got in my tests?
sysbench –test=cpu –threads=4 run

In my test, the L5 scored 33% better than the PinePhone, which is what I would expect, considering the differences in clock speed (1.5GHz vs 1.152GHz) and node size (28nm vs 40nm)

I’ve just ran it on L5 and PP and got 595.50 and 454.00 respectively, which sounds reasonable. Both results obtained using sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3).

1 Like

That’s a 31% difference, which is close to what I was seeing with 4 threads, so I’m guessing that something was slowing down the L5’s CPU performance in @johndoe’s tests. I wonder if running a Qt interface inside the GTK/GNOME+Phosh environment has any impact on the performance.

In johndoe’s tests, the problem is not about the L5 being slow, but the PP being fast. Maybe it’s a PinePhone Pro?