Benjamin Gaignard
September 07, 2023
Share this post:
Reading time:
The latest mainline Linux kernel (v6.5) includes 22 patches that enable support for the AV1 uAPI and for two stateless video decoders: one for the Rockchip RK3588 and one for MT8195, a MediaTek SoC. This is the result of several years of efforts by Collabora to bring up this new codec for stateless video decoding.
After also bringing support for VP8, H.264, VP9 and HEVC, our team is now well-versed in the requirements for adding a new video codec. As was the case for AV1, it requires a stable uAPI and at least two drivers using it on different hardware. The two drivers are needed as they help prove that the uAPI can meet the various hardware needs for hardware from different vendors. As a matter of fact, having more drivers would have been even better in terms of testing and coverage, but there are currently only two hardware IP blocks known to include a stateless AV1 decoder.
We also needed to prove that drivers can work with the new uAPI to generate valid frames. For this purpose, Daniel Almeida has upstreamed AV1 test files to Fluster (a video conformance test framework) in November 2021. He has also worked on the AV1 support in v4l2codecs, a GStreamer element that uses the V4L2 stateless decoder interface to decode video on hardware. This means that there is userspace support to drive the new pieces we have added to the kernel. GStreamer support has also been merged and will land on version 2.24, likely to be available to the general public around October 2023.
The first uAPI tentatives started back in March 2021 and needed 7 versions on the mailing list before being considered as stable in June 2023. The Rockchip AV1 driver needed the addition of 8800 lines of code and took 10 versions (and 1 year of work) to be accepted. In total, 305 conformance test files have been added to Fluster to cover a maximum of AV1 features including 10-bit pixel-format support and film grain, a process in which newer codecs like AV1 remove a pattern known as "grain" from the picture, owing to its random nature and thus poor compressibility.
All in all, having AV1 support heavily benefits the open source community, as AV1 itself is an open source codec. This means that AV1 has a royalty-free licensing model that makes it suitable for adoption in various open source projects. It was also designed to solve long-standing patent litigation issues that were common in other codecs in the industry. Furthermore, V4L2 support comes at a time when AV1 hardware designs are starting to become more common in consumer devices. This means that users can better leverage the capabilities of the hardware they have paid for while using Linux.
09/05/2024
This year will be Collabora's first time exhibiting at IOTSWC, in partnership with STMicroelectronics. Collabora will be showcasing how…
29/04/2024
GStreamer's support for diverse hardware and software platforms extends to its upstream Machine Learning capabilities, exemplified by its…
22/04/2024
As part of our commitment to social responsibility, we've been part of the 1% for the Planet Network since 2022. The environmental partners…
Whether writing a line of code or shaping a longer-term strategic software development plan, we'll help you navigate the ever-evolving world of Open Source.
한국어 버전의 Collabora.com 보기
Acesse Collabora.com em Português
Comments (0)
Add a Comment