Another way could be porting libwebrtc to work with openssl 3.0 or another gpl compatible tls library (it currently uses google’s fork of old openssl licensed code which is incompatible with gpl and hence epiphany cannot use this).
Or is there some work going on to get firefox/chromium work on Librem 5?
There is also a jitsi meet electron app. Does electron apps work yet? The version in flathub is not compatible (could not install).
I have tested current Librem 5 with kernel 5.15.0-1-librem5 + firefox-esr + firefox-esr-mobile-config with our BigBlueButton system installation which we use for education at our university. It seems to work for videoconferencing and presentations. Little annoying is that top screen indication of microphone use open over whole screen in mobile mode, but it can be easily closed from Posh. The GUI has some usability glitches on small screen but not critical. Incoming videos, presentations, screen sharing and both audio directions works as expected… Problem is sharing Librem 5 camera, it stucks, it seems that kernel support does not advertise or setup modes and power state correctly. But I thnik that when VLC starts to work with Librem 5 provided V4L2 device then even Firefox and BBB should…
No, not really. The L5 cameras aren’t like webcams that already provide YUV or even MJPEG data, they’re raw Bayer sensors where data needs to be processed before constructing a meaningful image. Exposing the modes over V4L2 is not enough, as most camera-using software does not do debayering, 3A and other processing needed for raw sensors.
The plan is to use libcamera as a middleware layer between the kernel and V4L2 applications, so our raw sensors can look like regular webcams to other software. There’s still quite a lot of work to do before that can happen though.
Thanks for clarification. I have only little experience with V4L2 (I have implemented drivers for FLIR infrared cameras for i.mx6 in grey scale formats MEDIA_BUS_FMT_Y12_1X12 for 3.x and 4.x kernels).
I have tried v4l2-ctl -d /dev/video2 --get-fmt-video which seems to report all formats, not only expected [15]: 'GBRG' (8-bit Bayer GBGB/RGRG), but i.MX8m CSI interface can probably support all reported. v4l2-ctl -d /dev/video2 --all lists Pad 0x01000005 : 0: Sink. But my attempts to get list of resolutions even for supported format seems strange v4l2-ctl -d /dev/video2 --list-framesizes 'GRBG' -> Size: Continuous 1x1 - 65535x65535, may it be limit of i.MX8M CSI input. If I try v4l2-ctl -d /dev/video0 --get-subdev-fmt 0, I receive error. I get reasonable output for command retrieving actual setting v4l2-ctl -d /dev/video2 --get-fmt-video. Attempt with simple example how to grab single frame of V4L2 video seems to work and stores 820560 bytes which matches raw format. Same for the command
v4l2-ctl --set-fmt-video=width=1052,height=780,pixelformat=GRBG --stream-mmap --stream-count=1 -d /dev/video2 --stream-to=raw.img
But I miss a way how to enumerate real supported formats on global and or subdev/pad level…
I have not found how to list controls for gain, exposure and balance as well even that I see them in the driver s5k3l6xx.c. They should be listed probably on the pad level but attempt to list formats for given (0, CSI) pad seems to fail.
As for the conversion, I would expect that there is some option in CSI or VPU module to do GRBG to RGB or YUV conversion in kernel directly in DMA streams. But I have only old IMX8M manual which is not clear and actual seems to require agreement with restrictions. Formats conversion in kernel at DMA level would prevent loading CPU so heavily
Even in case that kernel cannot process conversion, then libv4lconvert should be able to adjust and report synthetics BGR24, RGB24, YUV420 and YVU420 formats in user-space. So applications using v4l2 library should work out of the box… But when limited format support is not enumerated and all seems to be available then applications has no chance to find that conversion is necessary.
But I probably miss something…
I am curious how is the support planned/progressing. Is there some better documentation or location where to look on ongoing development, discus it and or try to contribute?
Yeah, you need to use V4L2 media API since the video pipeline consists of many subdevices.
media-ctl -p -d platform:30b80000.csi allows you to see the topology of the device that handles the big camera (use platform:30a90000.csi for the selfie cam).
For the reference, this should let you fully configure the rear cam’s pipeline for full resolution and stream a single raw frame from it:
Format conversion is just part of the issue, you also need to handle focus, exposure, gain and do white balancing (plus bunch of other image improving algorithms to deal with stuff like lens corrections), otherwise all you’re going to see will be a green mess instead of a proper image.
Re-reading your post, seems I have missed the clue of it:
I don’t know how to list that either
v4l2-ctl -d /dev/v4l-subdev2 -l works for me (after checking the subdevice number in media-ctl). There’s no balance control though - raw bayer data is always unbalanced.
Thanks a lot, it seems that media subsystem evolved a lot from time I worked with FLIR.
Anyway, I expect that Firefox and VLC does not go so deep and stays on /dev/video2 level only. So I would expect that it would be preferable if the supported formats are reported directly from the /dev/video2. If they should not be limited for whole device then (hopefully) applications looks at pad level control. I am not sure if VLC can recognize GRBG and convert it seamlessly. I have not dug into its sources but some discussion looks like it should work. I have seen some reports that some old applications work with workarond LD_PRELOAD=/usr/lib/libv4l/v4l1compat.so. So I would expect that if it is feasible to report right formats through /dev/videoX then it could help at least in some situations, i.e. FFmpeg should work out of the box with GRBG format…
I found working with subdevices directly impractical, all of them in the chain needs to be reconfigured before it works. I cheat a litthe in Millipixels where I use what’s basically libcamera example code to do the config before using the subdevice.
For reference, I have found libcam.cpp. Is this actual place? Is it done through subdev commands on /dev/video2 or through media or by other IOCTL on /dev/v4l-subdev5 level?
What are the plans? I would expect that possibility to open /dev/video2 and use it from VLC, Firefox etc. at least for basic setup for reported resolutons would worth. Exposure adjustment and other things can be done from other application. I used v4l2-ctl to set camera and then stream VLC from our systems in lab for the forced remote education last year. There seems to be problem with automatic exposure in this approach. As I remeber from some other SoCs camera interfaces, these have some registers for statics computation during pixels reception which allows to tune gain and exposure without need to process whole image in kernel or user-space. It seems that there is no other way than to do such processing on i.MX8 is software.
Or the PipeWire would solve stream preprocessing in the future? I have experience with OBS (it allows camera parameters tuning) and passing the video stream to Firefox through virtual camera. That was for parallel recording and forwarding lectures to BigBlueButton connected students. I would prefer if such standard methods work on Librem 5 for at least experiments too. For regular use, it would require probably much power and data copying that it is not feasible for mobile SoC when not processed in some dedicated HW. Passing data through V4L2 mmap interface directly to FFmpeg, VLC and or Firefox seems to me as better option. But I am not sure about GRBG to RGV/YUV conversion in Firefox… It should work in FFmpeg at least for recording and possible white balance adjustmen afterwards.
This used to work with the legacy drivers, but the new interface got a whole lot more capable, and as a consequence, all subdevices must be configured. The plan is to have libcamera handle this for the higher layers instead, including stream preprocessing. That would create a new standard place for camera handling.
It can’t be done from other application unless it’s somewhat receiving video stream as well (or unless you want to be adjusting it manually, which isn’t practical). Really, it’s not feasible to expect applications like VLC or Firefox to work sensibly with raw streams at all, you need some middleware layer in-between (which is what we want to use libcamera for).
Someone on mastodon suggested Angelfish browser. It is available on flathub but the browser don’t start. Anyone tried it? Running from command line don’t throw any errors, it just don’t start.
By default the navigation bar makes the jitsi bottom panel cut, this can be worked around by hiding the navigation bar by swiping from right edge and choosing the ‘hide navigation bar option’. So we have a browser that is well maintained and can be used for audio meetings with jitsi meet or BBB.
