The latest mainline Linux kernel (v6.5) includes 22 patches that enable support for the AV1 uAPI and for two stateless video decoders: one for the Rockchip RK3588 and one for MT8195, a MediaTek SoC. This is the result of several years of efforts by Collabora to bring up this new codec for stateless video decoding.

After also bringing support for VP8, H.264, VP9 and HEVC, our team is now well-versed in the requirements for adding a new video codec. As was the case for AV1, it requires a stable uAPI and at least two drivers using it on different hardware. The two drivers are needed as they help prove that the uAPI can meet the various hardware needs for hardware from different vendors. As a matter of fact, having more drivers would have been even better in terms of testing and coverage, but there are currently only two hardware IP blocks known to include a stateless AV1 decoder.

We also needed to prove that drivers can work with the new uAPI to generate valid frames. For this purpose, Daniel Almeida has upstreamed AV1 test files to Fluster (a video conformance test framework) in November 2021. He has also worked on the AV1 support in v4l2codecs, a GStreamer element that uses the V4L2 stateless decoder interface to decode video on hardware. This means that there is userspace support to drive the new pieces we have added to the kernel. GStreamer support has also been merged and will land on version 2.24, likely to be available to the general public around October 2023.

AV1 journey in numbers

The first uAPI tentatives started back in March 2021 and needed 7 versions on the mailing list before being considered as stable in June 2023. The Rockchip AV1 driver needed the addition of 8800 lines of code and took 10 versions (and 1 year of work) to be accepted. In total, 305 conformance test files have been added to Fluster to cover a maximum of AV1 features including 10-bit pixel-format support and film grain, a process in which newer codecs like AV1 remove a pattern known as "grain" from the picture, owing to its random nature and thus poor compressibility.

All in all, having AV1 support heavily benefits the open source community, as AV1 itself is an open source codec. This means that AV1 has a royalty-free licensing model that makes it suitable for adoption in various open source projects. It was also designed to solve long-standing patent litigation issues that were common in other codecs in the industry. Furthermore, V4L2 support comes at a time when AV1 hardware designs are starting to become more common in consumer devices. This means that users can better leverage the capabilities of the hardware they have paid for while using Linux.

Related Posts

Linux kernel 6.5: USB4v2 and Wifi7 have arrived

HEVC uAPI mainlined

Adding HEVC/H.265 support for NXP's i.MX 8M

Related Posts

Linux kernel 6.5: USB4v2 and Wifi7 have arrived

HEVC uAPI mainlined

Adding HEVC/H.265 support for NXP's i.MX 8M