jeffycn / libv4l-rkmpp Goto Github PK

A rockchip-mpp V4L2 wrapper plugin for chromium V4L2 VDA/VEA

License: GNU Lesser General Public License v2.1

C 98.33% Meson 1.67%

rockchip mpp chromium v4l2 libv4l-plugin vda vea

libv4l-rkmpp's Introduction

libv4l-rkmpp

A V4L2 plugin that wraps rockchip-mpp for the chromium's V4L2 video decoder/VEA (requires custom patches to enable those features).

The original idea comes from v4l-gst.

Dependencies

v4l-utils - with this patch:
0001-libv4l2-Support-mmap-to-libv4l-plugin.patch
rockchip-mpp

Building

   $ meson build
   $ meson compile -C build

Quick Start

Install libv4l-rkmpp.so into /usr/lib/libv4l/plugins/
Create dummy V4L2 device files for chromium V4L2 video decoder/VEA in boot service:

   # echo dec > /dev/video-dec0
   # chmod 666 /dev/video-dec0
   # echo enc > /dev/video-enc0
   # chmod 666 /dev/video-enc0

Configure codec capabilities

The codec capabilities (depends on chip spec) are configurable in device files:

   # cat /dev/video-dec0
   log-fps=1
   log-level=2
   type=dec
   codecs=VP8:VP9:H.264:H.265:AV1
   max-height=1920
   max-width=1080

Run with chromium browser:

   export XDG_RUNTIME_DIR=/run/user/0
   chromium --no-sandbox --gpu-sandbox-start-early --ignore-gpu-blacklist

This plugin is tested with custom chromium on rk3588 EVB.

Limitation

There're a lot of chromium related hacks in it, might not work for other apps.

For proper decoding usage, there's a ffmpeg solution with a few extra buffer copies.

FAQ

MPP reports errors?

Try the newest MPP release branch or develop branch or the commit with the closest commit date.

Also test with the mpi_dec_test to check if the MPP works:

# mpi_dec_test -t 7 -i test-25fps.h264

How to get more verbose logs?

For chromium, use these command line flags to change the log level: --enable-logging --vmodule=/media/gpu=4

For libv4l-rkmpp, set the "LIBV4L_RKMPP_LOG_LEVEL" environment variable to change the log level. And set "LIBV4L_RKMPP_LOG_FPS" to enable logging fps.

For MPP, set the environment variable "mpp_debug", "rkv_h264d_debug", "mpp_dec_debug", "mpi_debug", etc. to change the modules' log levels.

For vpu driver, write verbose log level to "/sys/module/rk_vcodec/parameters/debug".
What about the performance?

The performance should be much the same as other MPP based decoders/encoders (e.g. mpi_dec_test and gstreamer MPP plugin).

And the performance would mostly related to the video's attributes (e.g. resolution and bitrate) and the vpu clock rates.

Maintainers

Jeffy Chen <[email protected]>

libv4l-rkmpp's People

Contributors

Stargazers

Watchers

libv4l-rkmpp's Issues

VLC support

Thanks for this great repo - any chances you could provide some details to validate VLC configuration with your V4L driver? Is that supported?

There are two locations defining decoder size limit: meson_options.txt and libv4l-rkmpp-dec.c. By default we use the values defined in meson.
But that is a global config overwriting all the decoders.
AV1 decoder on rk3588 supports 8K while the h264/h265 decoders support up to 4K. So why do we need global size limit defined in meson?

v4l2-ctl 通过libv4l-rkmpp抓取h264帧

如果我想通过v4l2-ctl命令调佣libv4l-rkmpp直接抓取h264的帧该怎么实现呢？
我尝试了把libv4l-rkmpp编译的libv4l-rkmpp.so放入到/usr/lib/libv4l/plugins/目录，且v4l-utils也打了补丁0001-libv4l2-Support-mmap-to-libv4l-plugin.patch ，通过v4l2-ctl --list-formats-ext -d /dev/video0 列出支持的格式，似乎还是没有显示出libv4l-rkmpp-enc.c里面添加的H.264、VP8等。

Can't play av1 video with big bit rate

I can play this 20MB video.
But when playing this 30MB video, chromium gets stuck and after a long time it falls back to dav1d decoder(software decoder).
This video is played well with ffmpeg and gstreamer.

Dependency chain in README

It would be helpful to track which librockchip-mpp is required and which version of chromium-browser is compatible

Trouble diagnosing RKMPP not loading when moving from chromium 100 - 114

I am have trouble diagnosing issues related to the plugin not starting after moving to v114 of chromium with the patches from meta-rockchip/. I had this successfully working on chromium version 100.

With the log level set to 5 I get this repeated attempts to start the plugin with no specific errors on version 114. There are mentions of invalid ioctls but I also get this error when the plugin inits successfully on version 100.


 Using mplane plugin for capture
 [418.629] [RKMPP] [13606] rkmpp_parse_options(712): parsing option: type=enc
 [418.629] [RKMPP] [13606] rkmpp_parse_options(712): parsing option: codecs=VP8:VP9:H.264:H.265:AV1
 [418.629] [RKMPP] [13606] rkmpp_parse_options(712): parsing option: max-height=1080
 [418.629] [RKMPP] [13606] rkmpp_parse_options(712): parsing option: max-width=1920
 [418.629] [RKMPP] [13606] rkmpp_parse_options(742): ctx(0x200035c300): LEAVE
 [418.629] [RKMPP] [13606] rkmpp_enc_init(1090): ctx(0x200035c300): ENTER
 [418.629] [RKMPP] [13606] rkmpp_enc_init(1127): ctx(0x200035c300): LEAVE
 [418.629] [RKMPP] [13606] plugin_init(837): ctx(0x200035c300): plugin inited
 [418.629] [RKMPP] [13606] plugin_init(839): ctx(0x200035c300): LEAVE
 [418.629] [RKMPP] [13606] plugin_ioctl(902): ctx(0x200035c300): ENTER
 [418.629] [RKMPP] [13606] plugin_ioctl(906): ctx(0x200035c300): VIDIOC_QUERYCAP
 [418.629] [RKMPP] [13606] rkmpp_enc_ioctl(1162): ctx(0x200035c300): ENTER
 [418.629] [RKMPP] [13606] rkmpp_querycap(130): ctx(0x200035c300): ENTER
 [418.629] [RKMPP] [13606] rkmpp_querycap(146): ctx(0x200035c300): LEAVE
 [418.629] [RKMPP] [13606] rkmpp_enc_ioctl(1226): ctx(0x200035c300): LEAVE
 [418.629] [RKMPP] [13606] plugin_ioctl(913): ctx(0x200035c300): VIDIOC_QUERYCAP ret: 0
 [418.629] [RKMPP] [13606] plugin_ioctl(917): ctx(0x200035c300): LEAVE
 [418.629] [RKMPP] [13606] plugin_ioctl(902): ctx(0x200035c300): ENTER
 [418.629] [RKMPP] [13658] encoder_thread_fn(229): ctx(0x200035c300): ENTER
 [418.629] [RKMPP] [13658] encoder_thread_fn(231): ctx(0x200035c300): starting encoder thread
 [418.629] [RKMPP] [13606] plugin_ioctl(906): ctx(0x200035c300): VIDIOC_QUERYCAP
 [418.629] [RKMPP] [13606] rkmpp_enc_ioctl(1162): ctx(0x200035c300): ENTER
 [418.629] [RKMPP] [13606] rkmpp_querycap(130): ctx(0x200035c300): ENTER
 [418.629] [RKMPP] [13606] rkmpp_querycap(146): ctx(0x200035c300): LEAVE
 [418.629] [RKMPP] [13606] rkmpp_enc_ioctl(1226): ctx(0x200035c300): LEAVE
 [418.629] [RKMPP] [13606] plugin_ioctl(913): ctx(0x200035c300): VIDIOC_QUERYCAPret: 0
 [418.629] [RKMPP] [13606] plugin_ioctl(917): ctx(0x200035c300): LEAVE
 [418.629] [RKMPP] [13606] plugin_close(860): ctx(0x200035c300): ENTER
 [418.629] [RKMPP] [13606] plugin_close(862): ctx(0x200035c300): closing plugin
 [418.629] [RKMPP] [13606] rkmpp_enc_deinit(1136): ctx(0x200035c300): ENTER
 [418.629] [RKMPP] [13606] rkmpp_enc_deinit(1153): ctx(0x200035c300): LEAVE
 [418.629] [RKMPP] [13606] plugin_close(891): ctx(0x200035c300): LEAVE
 Failed to query video capabilities: Inappropriate ioctl for device
 Failed to query video capabilities: Inappropriate ioctl for device

This is the log from version 100 with similar initial output but eventually starts the plugin.

 [665.616] [RKMPP] [16597] plugin_close(862): ctx(0x4c003e4000): closing plugin
 [665.616] [RKMPP] [16597] rkmpp_enc_deinit(1136): ctx(0x4c003e4000): ENTER
 [665.617] [RKMPP] [16597] rkmpp_enc_deinit(1153): ctx(0x4c003e4000): LEAVE
 [665.617] [RKMPP] [16597] plugin_close(891): ctx(0x4c003e4000): LEAVE
 Failed to query video capabilities: Inappropriate ioctl for device
 Failed to query video capabilities: Inappropriate ioctl for device
 [666.612] [RKMPP] [15319] rkmpp_parse_options(712): parsing option: type=enc
 [666.612] [RKMPP] [15319] rkmpp_parse_options(712): parsing option: codecs=VP8:VP9:H.264:H.265:AV1
 [666.612] [RKMPP] [15319] rkmpp_parse_options(712): parsing option: max-height=1080
 [666.612] [RKMPP] [15319] rkmpp_parse_options(712): parsing option: max-width=1920
 [666.612] [RKMPP] [15319] rkmpp_parse_options(742): ctx(0x240381dc00): LEAVE
 [666.612] [RKMPP] [15319] rkmpp_enc_init(1090): ctx(0x240381dc00): ENTER
 [666.612] [RKMPP] [15319] rkmpp_enc_init(1127): ctx(0x240381dc00): LEAVE
 [666.612] [RKMPP] [15319] plugin_init(837): ctx(0x240381dc00): plugin inited
 [666.612] [RKMPP] [15319] plugin_init(839): ctx(0x240381dc00): LEAVE
 [666.612] [RKMPP] [15319] plugin_ioctl(902): ctx(0x240381dc00): ENTER
 [666.612] [RKMPP] [15319] plugin_ioctl(906): ctx(0x240381dc00): VIDIOC_QUERYCAP
 [666.612] [RKMPP] [15319] rkmpp_enc_ioctl(1162): ctx(0x240381dc00): ENTER
 [666.612] [RKMPP] [15319] rkmpp_querycap(130): ctx(0x240381dc00): ENTER
 [666.612] [RKMPP] [15319] rkmpp_querycap(146): ctx(0x240381dc00): LEAVE
 [666.612] [RKMPP] [15319] rkmpp_enc_ioctl(1226): ctx(0x240381dc00): LEAVE
 [666.612] [RKMPP] [15319] plugin_ioctl(913): ctx(0x240381dc00): VIDIOC_QUERYCAP  ret: 0
 [666.612] [RKMPP] [15319] plugin_ioctl(917): ctx(0x240381dc00): LEAVE
 [666.612] [RKMPP] [16679] encoder_thread_fn(229): ctx(0x240381dc00): ENTER
 [666.612] [RKMPP] [16679] encoder_thread_fn(231): ctx(0x240381dc00): starting encoder thread
 [666.612] [RKMPP] [15319] plugin_ioctl(902): ctx(0x240381dc00): ENTER
 [666.612] [RKMPP] [15319] plugin_ioctl(906): ctx(0x240381dc00): VIDIOC_ENUM_FMT
 [666.612] [RKMPP] [15319] rkmpp_enc_ioctl(1162): ctx(0x240381dc00): ENTER
 [666.612] [RKMPP] [15319] rkmpp_enum_fmt(156): ctx(0x240381dc00): ENTER
 [666.612] [RKMPP] [15319] rkmpp_enum_fmt(188): ctx(0x240381dc00): LEAVE
 [666.612] [RKMPP] [15319] rkmpp_enc_ioctl(1226): ctx(0x240381dc00): LEAVE
 [666.612] [RKMPP] [15319] plugin_ioctl(913): ctx(0x240381dc00): VIDIOC_ENUM_FMT  ret: 0
 [666.612] [RKMPP] [15319] plugin_ioctl(917): ctx(0x240381dc00): LEAVE
 [666.612] [RKMPP] [15319] plugin_ioctl(902): ctx(0x240381dc00): ENTER
 [666.612] [RKMPP] [15319] plugin_ioctl(906): ctx(0x240381dc00): VIDIOC_ENUM_FMT
 [666.612] [RKMPP] [15319] rkmpp_enc_ioctl(1162): ctx(0x240381dc00): ENTER
 [666.612] [RKMPP] [15319] rkmpp_enum_fmt(156): ctx(0x240381dc00): ENTER
 [666.612] [RKMPP] [15319] rkmpp_enum_fmt(188): ctx(0x240381dc00): LEAVE
 [666.612] [RKMPP] [15319] rkmpp_enc_ioctl(1226): ctx(0x240381dc00): LEAVE
 [666.612] [RKMPP] [15319] plugin_ioctl(913): ctx(0x240381dc00): VIDIOC_ENUM_FMT  ret: 0
 [666.612] [RKMPP] [15319] plugin_ioctl(917): ctx(0x240381dc00): LEAVE
 [666.612] [RKMPP] [15319] plugin_ioctl(902): ctx(0x240381dc00): ENTER
 [666.612] [RKMPP] [15319] plugin_ioctl(906): ctx(0x240381dc00): VIDIOC_ENUM_FMT
 [666.612] [RKMPP] [15319] rkmpp_enc_ioctl(1162): ctx(0x240381dc00): ENTER
 [666.612] [RKMPP] [15319] rkmpp_enum_fmt(156): ctx(0x240381dc00): ENTER
 [666.612] [RKMPP] [15319] rkmpp_enum_fmt(195): compressed format(2) not found
 [666.612] [RKMPP] [15319] rkmpp_enum_fmt(197): errno: 22
 [666.612] [RKMPP] [15319] rkmpp_enc_ioctl(1226): ctx(0x240381dc00): LEAVE
 [666.612] [RKMPP] [15319] plugin_ioctl(913): ctx(0x240381dc00): VIDIOC_ENUM_FMT  ret: -1
 [666.612] [RKMPP] [15319] plugin_ioctl(917): ctx(0x240381dc00): LEAVE
 [666.612] [RKMPP] [15319] plugin_close(860): ctx(0x240381dc00): ENTER
 [666.613] [RKMPP] [15319] plugin_close(862): ctx(0x240381dc00): closing plugin
 [666.613] [RKMPP] [15319] rkmpp_enc_deinit(1136): ctx(0x240381dc00): ENTER
 [666.613] [RKMPP] [15319] rkmpp_enc_deinit(1153): ctx(0x240381dc00): LEAVE
 [666.613] [RKMPP] [15319] plugin_close(891): ctx(0x240381dc00): LEAVE
 Failed to query video capabilities: Inappropriate ioctl for device
 [666.615] [RKMPP] [15319] rkmpp_parse_options(712): parsing option: type=enc
 [666.615] [RKMPP] [15319] rkmpp_parse_options(712): parsing option: codecs=VP8:VP9:H.264:H.265:AV1
 [666.615] [RKMPP] [15319] rkmpp_parse_options(712): parsing option: max-height=1080
 [666.615] [RKMPP] [15319] rkmpp_parse_options(712): parsing option: max-width=1920
 [666.615] [RKMPP] [15319] rkmpp_parse_options(742): ctx(0x240381dc00): LEAVE
 [666.615] [RKMPP] [15319] rkmpp_enc_init(1090): ctx(0x240381dc00): ENTER
 [666.615] [RKMPP] [15319] rkmpp_enc_init(1127): ctx(0x240381dc00): LEAVE
 [666.615] [RKMPP] [15319] plugin_init(837): ctx(0x240381dc00): plugin inited
 [666.615] [RKMPP] [15319] plugin_init(839): ctx(0x240381dc00): LEAVE
 [666.615] [RKMPP] [15319] plugin_ioctl(902): ctx(0x240381dc00): ENTER
 [666.615] [RKMPP] [15319] plugin_ioctl(906): ctx(0x240381dc00): VIDIOC_QUERYCAP
 [666.615] [RKMPP] [15319] rkmpp_enc_ioctl(1162): ctx(0x240381dc00): ENTER
 [666.615] [RKMPP] [15319] rkmpp_querycap(130): ctx(0x240381dc00): ENTER
 [666.615] [RKMPP] [15319] rkmpp_querycap(146): ctx(0x240381dc00): LEAVE
 [666.615] [RKMPP] [15319] rkmpp_enc_ioctl(1226): ctx(0x240381dc00): LEAVE
 [666.615] [RKMPP] [15319] plugin_ioctl(913): ctx(0x240381dc00): VIDIOC_QUERYCAP  ret: 0
 [666.615] [RKMPP] [15319] plugin_ioctl(917): ctx(0x240381dc00): LEAVE
 [666.615] [RKMPP] [15319] plugin_ioctl(902): ctx(0x240381dc00): ENTER
 [666.615] [RKMPP] [15319] plugin_ioctl(906): ctx(0x240381dc00): VIDIOC_ENUM_FRAMESIZES
 [666.615] [RKMPP] [15319] rkmpp_enc_ioctl(1162): ctx(0x240381dc00): ENTER
 [666.615] [RKMPP] [15319] rkmpp_enum_framesizes(205): ctx(0x240381dc00): ENTER
 [666.615] [RKMPP] [15319] rkmpp_enum_framesizes(223): ctx(0x240381dc00): LEAVE
 [666.615] [RKMPP] [15319] rkmpp_enc_ioctl(1226): ctx(0x240381dc00): LEAVE
 [666.615] [RKMPP] [15319] plugin_ioctl(913): ctx(0x240381dc00): VIDIOC_ENUM_FRAMESIZES  ret: 0
 [666.615] [RKMPP] [15319] plugin_ioctl(917): ctx(0x240381dc00): LEAVE
 [666.615] [RKMPP] [15319] plugin_ioctl(902): ctx(0x240381dc00): ENTER
 [666.615] [RKMPP] [15319] plugin_ioctl(906): ctx(0x240381dc00): VIDIOC_TRY_ENCODER_CMD
 [666.615] [RKMPP] [15319] rkmpp_enc_ioctl(1162): ctx(0x240381dc00): ENTER
 [666.615] [RKMPP] [16680] encoder_thread_fn(229): ctx(0x240381dc00): ENTER
 [666.615] [RKMPP] [16680] encoder_thread_fn(231): ctx(0x240381dc00): starting encoder thread
 [666.615] [RKMPP] [15319] rkmpp_enc_ioctl(1221): unsupported ioctl cmd: VIDIOC_TRY_ENCODER_CMD(3223869006)!
 [666.615] [RKMPP] [15319] rkmpp_enc_ioctl(1223): errno: 25
 [666.615] [RKMPP] [15319] plugin_ioctl(913): ctx(0x240381dc00): VIDIOC_TRY_ENCODER_CMD  ret: -1
 [666.615] [RKMPP] [15319] plugin_ioctl(917): ctx(0x240381dc00): LEAVE
 [666.615] [RKMPP] [15319] plugin_ioctl(902): ctx(0x240381dc00): ENTER
 [666.615] [RKMPP] [15319] plugin_ioctl(906): ctx(0x240381dc00): VIDIOC_QUERYCAP
 [666.615] [RKMPP] [15319] rkmpp_enc_ioctl(1162): ctx(0x240381dc00): ENTER
 [666.615] [RKMPP] [15319] rkmpp_querycap(130): ctx(0x240381dc00): ENTER
 [666.615] [RKMPP] [15319] rkmpp_querycap(146): ctx(0x240381dc00): LEAVE
 [666.615] [RKMPP] [15319] rkmpp_enc_ioctl(1226): ctx(0x240381dc00): LEAVE
 [666.615] [RKMPP] [15319] plugin_ioctl(913): ctx(0x240381dc00): VIDIOC_QUERYCAP  ret: 0
 [666.615] [RKMPP] [15319] plugin_ioctl(917): ctx(0x240381dc00): LEAVE
 [666.616] [RKMPP] [16678] plugin_ioctl(902): ctx(0x240381dc00): ENTER
 [666.616] [RKMPP] [16678] plugin_ioctl(906): ctx(0x240381dc00): VIDIOC_REQBUFS
 [666.616] [RKMPP] [16678] rkmpp_enc_ioctl(1162): ctx(0x240381dc00): ENTER
 [666.616] [RKMPP] [16678] rkmpp_reqbufs(373): ctx(0x240381dc00): ENTER
 [666.616] [RKMPP] [16678] rkmpp_get_queue(259): type = 10
 [666.616] [RKMPP] [16678] rkmpp_reqbufs(385): release buffers
 [666.616] [RKMPP] [16678] rkmpp_reqbufs(473): ctx(0x240381dc00): LEAVE
 [666.616] [RKMPP] [16678] rkmpp_enc_ioctl(1226): ctx(0x240381dc00): LEAVE
 [666.616] [RKMPP] [16678] plugin_ioctl(913): ctx(0x240381dc00): VIDIOC_REQBUFS  ret: 0
 [666.616] [RKMPP] [16678] plugin_ioctl(917): ctx(0x240381dc00): LEAVE
 [666.616] [RKMPP] [16678] plugin_ioctl(902): ctx(0x240381dc00): ENTER
 [666.616] [RKMPP] [16678] plugin_ioctl(906): ctx(0x240381dc00): VIDIOC_REQBUFS
 [666.616] [RKMPP] [16678] rkmpp_enc_ioctl(1162): ctx(0x240381dc00): ENTER
 [666.616] [RKMPP] [16678] rkmpp_reqbufs(373): ctx(0x240381dc00): ENTER
 [666.616] [RKMPP] [16678] rkmpp_get_queue(259): type = 9
 [666.616] [RKMPP] [16678] rkmpp_reqbufs(385): release buffers
 [666.616] [RKMPP] [16678] rkmpp_reqbufs(473): ctx(0x240381dc00): LEAVE
 [666.616] [RKMPP] [16678] rkmpp_enc_ioctl(1226): ctx(0x240381dc00): LEAVE
 [666.616] [RKMPP] [16678] plugin_ioctl(913): ctx(0x240381dc00): VIDIOC_REQBUFS  ret: 0
 [666.616] [RKMPP] [16678] plugin_ioctl(917): ctx(0x240381dc00): LEAVE
 [666.616] [RKMPP] [16678] plugin_ioctl(902): ctx(0x240381dc00): ENTER
 [666.616] [RKMPP] [16678] plugin_ioctl(906): ctx(0x240381dc00): VIDIOC_S_FMT
 [666.616] [RKMPP] [16678] rkmpp_enc_ioctl(1162): ctx(0x240381dc00): ENTER
 [666.616] [RKMPP] [16678] rkmpp_s_fmt(313): ctx(0x240381dc00): ENTER
 [666.616] [RKMPP] [16678] rkmpp_get_queue(259): type = 9
 [666.616] [RKMPP] [16678] rkmpp_try_fmt(253): ctx(0x240381dc00): ENTER
 [666.616] [RKMPP] [16678] rkmpp_try_fmt(303): ctx(0x240381dc00): LEAVE
 [666.616] [RKMPP] [16678] rkmpp_s_fmt(341): ctx(0x240381dc00): LEAVE
 [666.616] [RKMPP] [16678] rkmpp_enc_ioctl(1226): ctx(0x240381dc00): LEAVE
 [666.616] [RKMPP] [16678] plugin_ioctl(913): ctx(0x240381dc00): VIDIOC_S_FMT  ret: 0
 [666.616] [RKMPP] [16678] plugin_ioctl(917): ctx(0x240381dc00): LEAVE
 [666.616] [RKMPP] [16678] plugin_ioctl(902): ctx(0x240381dc00): ENTER
 [666.616] [RKMPP] [16678] plugin_ioctl(906): ctx(0x240381dc00): VIDIOC_S_FMT
 [666.616] [RKMPP] [16678] rkmpp_enc_ioctl(1162): ctx(0x240381dc00): ENTER

chromium-100.log

Is there some specific logging I can turn on to help find the cause of the issue. So far I have tried updating rockchip-mpp and this plugin to the latest commits.

video framerate is extramely low after commit f0d1d5364e30aa98f6d4cefd87f1e365ad5e205c

My environment:

rkr1 6.1 kernel
chromium v121
GPU driver is mainline panthor kernel driver and mainline git mesa
Mesa has patched gbm to support NV12 rendering.
mpp is from https://github.com/nyanmisaka/mpp/tree/jellyfin-mpp, from which nyanmisaka/mpp@81aef47 is necessary because kernel 6.1 has changed dma_heap. And mpp also need hack from #12 (comment) to support large av1 videos.

I encountered this issue before but at that moment I didn't configured all things right. Now I find this commit has caused the low fps issue.

Chromium 91 support

With Chromium 91+ focusing mainly on vaapi, would it be worth it to revive rockchip-va-driver instead?

does it possible to decode more than 1 video at the same time

Currently, I test with 4 video on chromium-ozone-wayland but it play only one video.

is it possible to support AV1 decoding?

chromium v121 can't use egl to render video output

I am running kernel 6.1.43 and gpu driver is panfork(mesa userspace driver and kernel arm mali driver)
Run chromium v121 with patches from meta-rockchip:

DISPLAY=:0 chromium --enable-logging --vmodule=*media/gpu*=4 /home/jfliu/bbb_sunflower_1080p_60fps_normal.mp4

Here is log from ~/.config/chromium/chrome_debug.log:
https://paste.armbian.com/kafubisuwi
We can see:

ERROR:shared_image_factory.cc(960)] Could not find SharedImageBackingFactory with params: usage: DisplayRead|Scanout, format: NV12_LEGACY, share_between_threads: 0, gmb_type: platform, debug_label: MailboxVideoFrameConverter_Cid:10_Pid:1962718

EGL rendering related code should be at media/gpu/v4l2/legacy and I don't know if we can use that since google has dropped it.
I think this line of patch makes chromium know it can render NV12 without color space conversion, but in fact it can't.

I also meet the same issue when testing mainline kernel v4l2 stateless decoding with the patch line above. Without that patch chromium can make use of libyuvimageprocessor to convert NV12 to AR24.

Use Fuse instead of libv4l-utils

@JeffyCN

https://github.com/libfuse/libfuse/blob/master/example/ioctl.c

I have briefly tested this idea. The idea is to use a libfuse filesystem. A deamon as in the example will create a filesystem with video-vepu580, video-vdpuxxx files in it.

After the daemon is mounted to a directory, the generated files will be symlinked to /dev/video-*.

When another client application opens this /dev/video-vepu580 ie, and calls V4L2 IOCTL like (VIDIOC_QUERYCAP) with this FD, the daemon can catch it and dispatch to mpp backend to handle.
ie:
https://github.com/libfuse/libfuse/blob/869a4a6fa550ae054df01f9d50db68871f88ca4f/example/ioctl.c#L204
handle it like:

libv4l-rkmpp/src/libv4l-rkmpp-enc.c

Line 1167 in 38da407

case VIDIOC_QUERYCAP:

This allows to bind v4l2 backend to mpp genericly, this can be used with
chromium
Firefox since 116 https://bugzilla.mozilla.org/show_bug.cgi?id=1833354
FFmpeg https://github.com/FFmpeg/FFmpeg/blob/8028b8260e6d0a4a469529ec5c5d504782a5f875/libavcodec/v4l2_m2m.c#L75

literally anything that supports vl42_m2m

the main difference with v4l2-utils, is that the plugin only works if the client uses libv4l2, if direct IOCTL is used then this does not work. Also with libfuse approach you do not need to patch anything at all, it is straight forward.

One concern could be how much overhead that the libfuse would bring but i think it should be minuscule.

What do you think? i think most the code already avaibale in this repo already..

关于libv4l-rkmpp的使用

您好，我这边想使用你的这个libv4l-rkmpp进行H264码流和YUV码流的获取，想看看有没有对应的例子，有几点疑问：

我如何选择是哪个video节点，或者rkisp节点？v4l2直接通过open可以打开指定节点，但是您这个貌似没有；
我想通过select或者poll获取数据，监控的文件句柄是哪个啊？ v4l2直接通过监控open返回的句柄得到的；
如果我想添加HEVC，即H265编码的支持，该如何进行修改啊？
打扰了，谢谢。

Handling 10-bit HEVC/VP9/AV1 videos?

Some self-hosted media servers and Youtube/Bilibili can query 10-bit video decode support via HTMLMediaElement: canPlayType() reported by the browser and play 10-bit/HDR directly in Chromium.

If I understand correctly, the current libv4l-rkmpp code base only handles 8-bit video. For 10-bit 4:2:0 videos, the MPP decoded NV15 memory will be displayed as NV12 (corrupted image for sure) since NV15 is a compact version of P010, which is not supported by EGL and Mesa.

IMO the NV15 can be converted to P010 with RGA3 before importing into EGL to avoid loss of precision resulting in color banding, or fallback to using NV12 to ensure compatibility.

So, are there any plans to improve support for 10-bit video (HEVC Main10, VP9 Profile 2 and AV1 Main)? Thanks in advance.

Feature request: H265 decoding

Chromium can be compiled to support HEVC demuxing in an MP4 container, and decoding via v4l2:

eg: on 73.0.3683.103:

#args.gn
enable_hevc_demuxing=true

// v4l2_video_decode_accelerator.cc (73.0.3683.103)
@@ -88,6 +88,9 @@ const uint32_t V4L2VideoDecodeAccelerator::supported_input_fourccs_[] = {
 #if !BUILDFLAG(USE_LINUX_V4L2)
     V4L2_PIX_FMT_VP9,
 #endif
+#if BUILDFLAG(ENABLE_HEVC_DEMUXING) && defined(V4L2_PIX_FMT_HEVC)
+    V4L2_PIX_FMT_HEVC,
+#endif
 };

是否可以在Electron中使用vpu进行视频解码

Electron是基于chromium的桌面端开发框架，Electron也支持给上游的chromium打补丁https://www.electronjs.org/zh/docs/latest/development/patches

chromium has dropped libv4l2 since v117

FYI: chromium/chromium@72d438b
What are we going to do to make mpp work in the future?

rockchip_mpp not found

Hi sir, well, I installed rockchip_mpp through the repo as "librockchip-mpp" and also compiled it and install it into the system , but for some reason it's not catched by this project. could you give me a hint here?
librockchip_mpp.so and all the rest are on /usr/local/lib and on /usr/lib they are the one from the repo, it's hard to know why it doesnt pick them.

some videos are displayed all green

I've packaged the latest code in a ppa: https://launchpad.net/~liujianfeng1994/+archive/ubuntu/rockchip-multimedia/.
And I can play some video with hardware accelerated decode in a patched chromium 101.0.4951.15 on radxa rock5b ,a rk3588 board.
But there are some videos in bilibili.com that are displayed all green, for example: https://www.bilibili.com/video/BV1wE411M7nK.
This seems to be a rga related issue.
I Added this patch to show more log:

diff --git a/src/utils.c b/src/utils.c
index 8f090eb..3d1123a 100644
--- a/src/utils.c
+++ b/src/utils.c
@@ -126,12 +126,14 @@ static int rkmpp_copy_buffer(struct rkmpp_context *ctx,
        }
 
        /* Nothing to do for mmap buffer */
-       if (buffer->memory == V4L2_MEMORY_MMAP)
-               return 0;
+       if (buffer->memory == V4L2_MEMORY_MMAP) {
+               LOGE("Nothing to do for mmap buffer\n");
+               return 0;}
 
        queue = rkmpp_get_queue(ctx, buffer->type);
-       if (!queue)
-               return -1;
+       if (!queue) {
+               LOGE("Can't get queue\n");
+               return -1;}
 
        rkmpp_ptr = mpp_buffer_get_ptr(rkmpp_buffer->rkmpp_buf);
        rkmpp_format = queue->rkmpp_format;

And I got this debug log: https://paste.ubuntu.com/p/3VVPVcwt7W/
rkmpp_copy_buffer always return -1 because of buffer->memory == V4L2_MEMORY_MMAP. And the output of cat /proc/interrupts |grep rga shows that the rga is not used. It this the root cause?
Or any other things that I've ignored.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.