Lib[rem5|camera] to use everywhere

galilley · April 1, 2025, 8:48pm

No time to explain, just a new thread for discussion this post.

Few hours later… Sorry for early posting. Of course, I will explain everything in details. Let’s start from a short background.

As far as I know we have only one application (millipixels) that can take a photo in Librem5 and no other application can give an access to the camera (like decoder or firefox). The reason is that it uses libcamera api rather that standard v4l2. As a result, we have to use it directly and work with raw images (that millipixels does) or by some standartized wrapper, that all other application do. Pipewire is a such wrapper now, and it could be configured to work with video-only streams (to not break calls). So, the pipeline should look something like this: cam (front or back) → libcamera → pipewire → your app. Sounds quite simple but it does not work. At least it does not work in Byzantium and Crimson so far. I’m not sure about ‘Mobian’ and ‘pmOS’… But here, on Byzantium backports it takes much more time than I expected. Starting point is here.

Linux kernel config must be patched to enable dmabuf heaps.
libcamera must be patched to enable software postprocessing and built with gnutls to avoid isolation.
wireplumber must be configured properly.

Finally, I was managed to put the whole puzzle together!
That is about this thread. Next I will present some details and put links here.

Content of this thread

Checklist to share L5 cameras between apps
Additional configs (may be included in libcamera package in the future)

JR-Fi · April 1, 2025, 9:03pm

Contain your excitement Your topic and posts do not explain anything (the other one is way off topic too in that other thread). How about you edit the first post to tell everyone what it’s about, how its done and link to relevant sources…?

galilley · April 1, 2025, 8:45pm

Just a ball. The most important thing is how this photo has been taken - by standard gnome Camera application through libcamera managed by pipewire! It means that the camera just became available almost everywhere! In all preinstalled/flatpaked/selfmade application! Just everywhere. So, lets start a new stage in Librem5 usability improvements

spacemanspiffy · April 1, 2025, 9:18pm

Yeah ok so this is a pretty big deal

ASwyD2 · April 2, 2025, 1:17am

I must have less packages installed than you because it is not working for me and I have your backports installed

galilley · April 2, 2025, 2:26am

I will push all necessary packages in a day

galilley · April 2, 2025, 3:47am

Sorry, you are right. I edited the first post to make it much more clear

galilley · April 3, 2025, 3:06am

Checklist to share L5 cameras between apps:

Note: I believe that you use the latest version of Byzantium or ~~Crimson~~ (not built yet) backports.

0. Required packages should be installed

$ sudo apt install pipewire-libcamera libcamera-tools gstreamer1.0-tools

1. Cameras should be visible by the lib:

$ cam -l
[0:21:55.572391385] [3744]  INFO Camera camera_manager.cpp:327 libcamera v0.4.0
...
Available cameras:
1: Internal front camera (/base/soc@0/bus@30800000/i2c@30a40000/camera@20)
2: Internal back camera (/base/soc@0/bus@30800000/i2c@30a50000/camera@2d)

2. The library should be able to capture an image:

$ cam -c1 -C1
...
cam0: Capture 1 frames
1429.065381 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 7970688

3. At this stage you should be able to grab video stream e.g. using `gstreamer`:

$ gst-launch-1.0 libcamerasrc camera-name="/base/soc@0/bus@30800000/i2c@30a40000/camera@20" saturation=5 ! video/x-raw,format=RGB,width=720, height=576,framerate=10/1 ! videoconvert ! queue ! fpsdisplaysink text-overlay=false

4. Wireplumber should see your cameras:

(check README, section No Sound first)

$ wpctl satus
PipeWire 'pipewire-0' [1.4.1, purism@pureos, cookie:3974875000]
 └─ Clients:
        33. WirePlumber (video-only)            [1.4.1, purism@pureos, pid:1235]
        41. WirePlumber (video-only) [export]   [1.4.1, purism@pureos, pid:1235]
        56. xdg-desktop-portal                  [1.4.1, purism@pureos, pid:1500]
        57. xdg-desktop-portal-wlr              [1.4.1, purism@pureos, pid:2056]
        58. chatty                              [1.4.1, purism@pureos, pid:1878]
        59. wpctl 

Video
 ├─ Devices:
 │      42. imx-capture                         [v4l2]
 │      43. imx-capture                         [v4l2]
 │      44. nxp,imx8mq-vpu-g1-dec               [v4l2]
 │      45. nxp,imx8mq-vpu-g2-dec               [v4l2]
 │      46. hi846                               [libcamera]
 │      47. s5k3l6xx                            [libcamera]
 │  
 └─ Sources:
    *   52. Internal front camera
        54. Internal back camera

5. Grabbing used pipewire source also should work

path option is important, it must be equal your source id

$ gst-launch-1.0 pipewiresrc path=52 ! videoconvert ! video/x-raw,format=RGB,framerate=10/1 ! fpsdisplaysink text-overlay=false

6. Enjoy your cameras!

In fact - not. Just because a lot of options have not been realized yet (like autofocus) inside libcamera or realized not optimal.

galilley · April 5, 2025, 9:18am

Some additional config files that seem to be important:

add rules for video group to access dma heaps

$ cat /usr/lib/udev/rules.d/90-libcamera.rules 
SUBSYSTEM=="dma_heap",GROUP="video",MODE="0660"

0010-ipa-simple-Add-tuning-file-for-hi846.patch

$ cat /usr/share/libcamera/ipa/simple/hi846.yaml 
# SPDX-License-Identifier: CC0-1.0
%YAML 1.1
---
version: 1
algorithms:
  - BlackLevel:
      blackLevel: 4096
  - Awb:
  - Lut:
  - Agc:
...

0009-ipa-simple-Add-tuning-file-for-s5k3l6xx.patch

$ cat /usr/share/libcamera/ipa/simple/s5k3l6xx.yaml 
# SPDX-License-Identifier: CC0-1.0
%YAML 1.1
---
version: 1
algorithms:
  - BlackLevel:
      blackLevel: 4096
  - Awb:
  - Lut:
  - Agc:
...

JR-Fi · April 5, 2025, 9:52am

The interesting testing question is: can this setup be used to stream usable video in any two-way communication app, or even video conferencing. Or video voice mail.

galilley · April 5, 2025, 12:02pm

Looks like this is only possible way to make video communication apps work, the first step to do that in librem5.

Ick · April 5, 2025, 12:45pm

I already did video communication in 2023. Plug in a desktop webcam into USB and there you go. For sure not that handy (carrying another device), but works exactly as on desktop.

epinez · April 19, 2025, 4:09pm

Thank’s for this post

I’ll try that on Mobian once I’ll have the time.
Would love to have this supported upstream some day.