Librem 5 support H.265 on NXP i.MX8M

Collabora fundation is the creator of the Libre Hantro-vpu Driver including G1.h.264,G2.h.265 to VeriSilicon VPUs found in SOCs Rockchip, i.MX8, Microchip. I guess that final kernel 5.11 or 5.12 will came with hantro-G2 merged then Purism will can enable HEVC decoding on L5 on pureos byzantium.
I checked the hantro driver on my Evergreen then it seems that purism is not yet enabled this cool driver on stable kernel.

@carlosgonz enabling the driver in the kernel is not the issue, see https://source.puri.sm/Librem5/linux-next/-/merge_requests/129 - weā€™re currenty sorting out the gstreamer side to make use of it (due to kernel API changes when things got moved out of staging)

6 Likes

Thank you @guido.gunther
Great that Purism is already tracking the Hantro driver to L5. :smiley:

2 Likes

Hereā€™s h264 via gstreamer on the L5 https://social.librem.one/@agx/105799071793016596 (working but not yet packaged for the phone, will likely be byzantium only due to the amount of gstreamer changes), h265 didnā€™t land in gstreamer yet.

6 Likes

Thank you @guido.gunther you are doing really really great work on pushing H.264/5 fast to Librem 5.
Will Totem(gnome video) play H.265 by using FFmpeg on Byzantium?
Will Epiphany play H.265 on some time on Byzantium by upgrading Gstreamer?

1 Like

Eventually likely yes but that needs e.g. more work in totem (which we also need for h264).

2 Likes

I just report that Totem is played very well a video in H.264 on amber and L5.
Librem 5 is awesome on every updates.
W a i t i n g for Byzantium.
Thank you Purism HKS.

4 Likes

looking forward to using Celluloid < MPV GTK GUI front-end on the L5. on the desktop you can basically drag and drop snooptube AV-content over to MPV and it-just-works ā„¢

@guido.gunther @dos @dcz I can not wait much, to watching videos on H.264 or H.265 on full screen on my L5 Evergreen, but I want to keep my L5 cooler, smooth motion, and low energy while i watching the videos so please, please please Purism turn ON the VPUs on Byzantium. :face_with_monocle:

This is maybe a minor thing, but the only thing that taints the kernel on the L5 now is that the hantro_vpu driver is in staging. Does anyone know when it could be accepted to the main portion of the kernel? Would be really cool to have a completely untainted kernel running on the L5 :slight_smile:

1 Like

Hantro g1 and g2 it already on L5 kernel, just that at the moment Gstreamer only can handle g1, so now you can watch videos H.264 by using Hantro_vpu driver and ā€œClapperā€ is a GNOME media player.

Iā€™m getting some pretty bad artifacts when playing back a H.265 (HEVC) sample video with Clapper. I had to enable ā€œUse playbin3ā€ under ā€œTweaksā€ to at least get some video output (albeit corrupted) instead of a black screen.
Since PureOS doesnā€™t ship with a recent GStreamer version that supports the v4l2slh265dec plugin, I tried to run GStreamer within the Clapper flatpak with results in the same behavior:

flatpak run --command=sh com.github.rafostar.Clapper
$ gst-launch-1.0 filesrc location=Videos/h265.mp4 ! qtdemux ! v4l2slh265dec ! glupload ! glimagesink

I get the same behavior when running the same gst-launch-1.0 command outside the flatpak after flashing postmarketOS edge, which provides a more recent version of GStreamer.

H.264 media is decoded correctly without issues though :slight_smile:

Could anyone confirm if this is also an issue youā€™re seeing?

1 Like

I guess it is a known issue in Hantro to L5 VPU(Thanks Collabora), i not tested h.265 but i tested VP9 and bad artifacts it there too.
As far i know this bug is not for PPP VPU.

@dos 0xd05

I did a new test today for h.265 under matroska and clapper and it played extremely good motion at 60 fps 720p. not more bad artifacsā€¦ let is make more debugging

Sadly i do not have a good camera, like capable to recording at 60fps to capture the extremely motion of Librem 5.
Thanks to 0xd05 for the 63 or 65 hz for L5 it feelingā€¦

Very interesting! I would like to reproduce this on my device to investigate further.

Could you provide a link to a H.265 file that played back well on your device, so I can try to reproduce it? Additionally, what kernel version are you on? You can find out by executing uname -a.
Are you also running Clapper 0.5.2 installed via Flatpak?

I would love to be able to play H.265 media without issues on my Librem 5, but I havenā€™t been successful so far.

https://www.libde265.org/hevc-bitstreams/bbb-1280x720-cfg06.mkv

LinuxLibre 6.2

Yes

Purism team.

1 Like

Thanks for providing a link! This media file played back on my device without issues with v4l2slh265dec. Here is the video metadata for future reference as shown by mediainfo:

MediaInfo working sample
Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 10 min 34 s
Width                                    : 1 280 pixels
Height                                   : 720 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 60.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Language                                 : English
Default                                  : Yes
Forced                                   : No

However, the following media file does not render correctly with visible artifacting near the bottom of the video:
https://www.libde265.org/hevc-bitstreams/spreed-1080p.mkv

MediaInfo broken sample 1
Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 1 min 0 s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (29970/1000) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Language                                 : English
Default                                  : Yes
Forced                                   : No

Another file that has artifacts is the following: https://test-videos.co.uk/vids/bigbuckbunny/mp4/h265/720/Big_Buck_Bunny_720_10s_1MB.mp4

MediaInfo broken sample 2
Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main@L3.1@Main
Codec ID                                 : hev1
Codec ID/Info                            : High Efficiency Video Coding
Duration                                 : 10 s 0 ms
Bit rate                                 : 842 kb/s
Width                                    : 1 280 pixels
Height                                   : 720 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 30.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.030
Stream size                              : 1.00 MiB (99%)
Writing library                          : x265 2.6+49-7219376de42a:[Windows][GCC 7.3.0][64 bit] 8bit+10bit
Encoding settings                        : cpuid=1050111 / frame-threads=3 / numa-pools=8 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=1280x720 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=25 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=25 / lookahead-slices=4 / scenecut=40 / radl=0 / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=3 / subme=3 / merange=57 / temporal-mvp / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=1.00 / no-rd-refine/ no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=abr / bitrate=872 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=2 / transfer=2 / colormatrix=2 / chromaloc=0 / display-window=0 / max-cll=0,0 / min-luma=0 / max-luma=255 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr / no-hdr-opt / no-dhdr10-opt / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-mv-type=0 / copy-pic=1
Codec configuration box                  : hvcC

Do you observe the same artifacts for the broken samples?

1 Like

I also noticed something interesting when playing back the sample file you provided with Clapper. This also happens with the broken samples I mentioned in my previous post. If I seek to any other time in the video (by moving the slider to another position with my finger), the phone completely freezes and requires a long press on the power button to make it reboot. I also tried to print kernel output with sudo dmesg -w using the serial port over USB /dev/ttyACM0 but it completely freezes as well without printing anything just before the issue.

Does this also happen on your device?

Seeking to arbitrary positions for H.264 media works fine though.

1 Like

Yes i got one freeze too, i will investigate this bug.

3 Likes