Lib[rem5|camera] to use everywhere

[galilley]

No time to explain, just a new thread for discussion this post.

Few hours later… Sorry for early posting. Of course, I will explain everything in details. Let’s start from a short background.

As far as I know we have only one application (millipixels) that can take a photo in Librem5 and no other application can give an access to the camera (like decoder or firefox). The reason is that it uses libcamera api rather that standard v4l2. As a result, we have to use it directly and work with raw images (that millipixels does) or by some standartized wrapper, that all other application do. Pipewire is a such wrapper now, and it could be configured to work with video-only streams (to not break calls). So, the pipeline should look something like this: cam (front or back) → libcamera → pipewire → your app. Sounds quite simple but it does not work. At least it does not work in Byzantium and Crimson so far. I’m not sure about ‘Mobian’ and ‘pmOS’… But here, on Byzantium backports it takes much more time than I expected. Starting point is here.

1. Linux kernel config must be patched to enable dmabuf heaps.
2. libcamera must be patched to enable software postprocessing and built with gnutls to avoid isolation.
3. wireplumber must be configured properly.

Finally, I was managed to put the whole puzzle together!
That is about this thread. Next I will present some details and put links here.

Content of this thread

1. Checklist to share L5 cameras between apps
2. Additional configs (may be included in libcamera package in the future)

[JR-Fi]

Contain your excitement Your topic and posts do not explain anything (the other one is way off topic too in that other thread). How about you edit the first post to tell everyone what it’s about, how its done and link to relevant sources…?

[galilley]

Just a ball. The most important thing is how this photo has been taken - by standard gnome Camera application through libcamera managed by pipewire! It means that the camera just became available almost everywhere! In all preinstalled/flatpaked/selfmade application! Just everywhere. So, lets start a new stage in Librem5 usability improvements

[spacemanspiffy]

Yeah ok so this is a pretty big deal

[ASwyD2]

@galilley

by standard gnome Camera application through libcamera managed by pipewire! It means that the camera just became available almost everywhere! In all preinstalled/flatpaked/selfmade application!

I must have less packages installed than you because it is not working for me and I have your backports installed

[galilley]

I will push all necessary packages in a day

[galilley]

Sorry, you are right. I edited the first post to make it much more clear

[galilley]

Checklist to share L5 cameras between apps:

Note: I believe that you use the latest version of Byzantium or Crimson backports.

0. Required packages should be installed

$ sudo apt install pipewire-libcamera libcamera-tools gstreamer1.0-pipewire gstreamer1.0-tools

1. Cameras should be visible by the lib:

$ cam -l
[0:21:55.572391385] [3744]  INFO Camera camera_manager.cpp:327 libcamera v0.4.0
...
Available cameras:
1: Internal front camera (/base/soc@0/bus@30800000/i2c@30a40000/camera@20)
2: Internal back camera (/base/soc@0/bus@30800000/i2c@30a50000/camera@2d)

2. The library should be able to capture an image:

$ cam -c1 -C1
...
cam0: Capture 1 frames
1429.065381 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 7970688

3. At this stage you should be able to grab video stream e.g. using gstreamer:

$ gst-launch-1.0 libcamerasrc camera-name="/base/soc@0/bus@30800000/i2c@30a40000/camera@20" saturation=5 ! video/x-raw,format=RGB,width=720, height=576,framerate=10/1 ! videoconvert ! queue ! fpsdisplaysink text-overlay=false

4. Wireplumber should see your cameras:

(check README, section No Sound first)

$ wpctl satus
PipeWire 'pipewire-0' [1.4.1, purism@pureos, cookie:3974875000]
 └─ Clients:
        33. WirePlumber (video-only)            [1.4.1, purism@pureos, pid:1235]
        41. WirePlumber (video-only) [export]   [1.4.1, purism@pureos, pid:1235]
        56. xdg-desktop-portal                  [1.4.1, purism@pureos, pid:1500]
        57. xdg-desktop-portal-wlr              [1.4.1, purism@pureos, pid:2056]
        58. chatty                              [1.4.1, purism@pureos, pid:1878]
        59. wpctl 

Video
 ├─ Devices:
 │      42. imx-capture                         [v4l2]
 │      43. imx-capture                         [v4l2]
 │      44. nxp,imx8mq-vpu-g1-dec               [v4l2]
 │      45. nxp,imx8mq-vpu-g2-dec               [v4l2]
 │      46. hi846                               [libcamera]
 │      47. s5k3l6xx                            [libcamera]
 │  
 └─ Sources:
    *   52. Internal front camera
        54. Internal back camera

5. Grabbing used pipewire source also should work

path option is important, it must be equal your source id

$ gst-launch-1.0 pipewiresrc path=52 ! videoconvert ! video/x-raw,format=RGB,framerate=10/1 ! fpsdisplaysink text-overlay=false

6. Enjoy your cameras!

In fact - not. Just because a lot of options have not been realized yet (like autofocus) inside libcamera or realized not optimal.

[galilley]

Some additional config files that seem to be important:

add rules for video group to access dma heaps

$ cat /usr/lib/udev/rules.d/90-libcamera.rules 
SUBSYSTEM=="dma_heap",GROUP="video",MODE="0660"

0010-ipa-simple-Add-tuning-file-for-hi846.patch

$ cat /usr/share/libcamera/ipa/simple/hi846.yaml 
# SPDX-License-Identifier: CC0-1.0
%YAML 1.1
---
version: 1
algorithms:
  - BlackLevel:
      blackLevel: 4096
  - Awb:
  - Lut:
  - Agc:
...

0009-ipa-simple-Add-tuning-file-for-s5k3l6xx.patch

$ cat /usr/share/libcamera/ipa/simple/s5k3l6xx.yaml 
# SPDX-License-Identifier: CC0-1.0
%YAML 1.1
---
version: 1
algorithms:
  - BlackLevel:
      blackLevel: 4096
  - Awb:
  - Lut:
  - Agc:
...

[JR-Fi]

The interesting testing question is: can this setup be used to stream usable video in any two-way communication app, or even video conferencing. Or video voice mail.

[galilley]

Looks like this is only possible way to make video communication apps work, the first step to do that in librem5.

[Ick]

I already did video communication in 2023. Plug in a desktop webcam into USB and there you go. For sure not that handy (carrying another device), but works exactly as on desktop.

[epinez]

Thank’s for this post

I’ll try that on Mobian once I’ll have the time.
Would love to have this supported upstream some day.

[JR-Fi]

Did you get it to work? I’ve still had millipixels which dies of seg fault every time I try to take pic.

Mobian structure seem different, there’s no “90-libcamera.rules” file (there is 70-camera.rules that has v4l stuff… - replace those, I guess?). Haven’t gone deeper yet.

[galilley]

Yes, it works both in Crimson and Byzantium backports, but too much problems should be solved… E.g native gnome-decoder shows the picture upside down, but flatpak works fine.

The first thing is DMABUF option in the kernel
The second one - patched libcamera no newer than 0.4.

[JR-Fi]

FYI: Release libcamera v0.5.1 · libcamera-org/libcamera · GitHub new version and changes

[galilley]

When I tried v0.5.0 it did not work with our camera at all, looks like they mark it as obsoleted… I’m not sure, more detailed investigation required to find the exact moment when it became broken.

[JR-Fi]

I don’t suppose you’ve come across a way to take a picture using command line…?

[galilley]

What exactly did you mean?.. The simplest way to take a picture using command line is:

cam -c1 -C1

Of course, you should specify a file name, but I newer look inside such file… Another way is the gstreamer pipelines, but it looks much more complicated. Regarding to libcamera v0.5.0 I did not see my camera using cam -l command, so, I can’t select the right device to take a picture.

[JR-Fi]

Just troubleshooting a problem with Mobian and wanted to see if non-GUI method would help. It did not.