LCEVC meets GStreamer

Integrating MPEG's novel enhancement codec

MPEG-5 Part 2 Low Complexity Enhancement Video Coding (LCEVC) is a new video standard by the Moving Picture Experts Group (MPEG). Originally developed by V-Nova, a leading provider of video compression solutions, it specifies an enhancement layer which, when combined with a base video encoded with a separate codec, produces an enhanced video stream with sustainable power consumption.

LCEVC works in conjunction with existing video encoders and codecs. It generates a lower resolution base image that can be encoded with any codec and specifies an enhancement layer which, when combined with the base video, produces an enhanced video stream. This results in a greater compression ratio than the base codec alone.

With the standard now published and industry support quickly growing, V-Nova sought Collabora's GStreamer expertise to further erase any barriers to a broad adoption of LCEVC by integrating the recently released LCEVCdec decoder SDK into the multimedia framework, and leverage the benefits of GStreamer's open source nature.

LCEVC and GStreamer

As the leading open source multimedia framework, GStreamer makes it easy to abstract various types of media processing elements and link them to create complex graphs. One of the framework’s strengths is its ability to substitute different implementations of the same functionality, for example a software codec with one that is hardware accelerated. Thanks to its intelligent pipeline construction facility, GStreamer can create the most optimized pipeline to decode a video given the capabilities of the platform.

Integrating a multi-step decoding process like LCEVC into such a flexible architecture while preserving the characteristics that allow for automated pipeline creation requires careful design. This is why V-Nova tasked Collabora and its team of GStreamer experts to build the right architecture to integrate the LCEVC codec into the framework.

Designing a complete architecture

With the cutting-edge nature of the LCEVC process, building the right architecture involved multiple steps including extracting compressed data, passing the base layer through a regular decoder, and combining the decoded, low quality image, with the enhancement data. As GStreamer decoders expect to receive encoded data, and output decoded data, a flexible wrapper was also needed to perform both decoding actions together while maintaining the ability to use any decoder for the base layer.

From mapping functional requirements, to making the right structural choices to bring LCEVC support, Collabora's team of multimedia experts successfully designed a complete, maintainable architecture to seamlessly integrate the LCEVC codec into the framework. As longstanding maintainers of GStreamer, Collabora will be able to bring this new LCEVC support directly into the framework's core repository, making it available to all GStreamer users on every platform.

Looking ahead

Backed by a growing ecosystem of broadcasters, streaming services and more, LCEVC is on a path to become a leading standard for enhanced video compression. With GStreamer, the de-facto standard for hardware accelerated video now providing native support, LCEVC integration has never been easier.

Beyond the support for software integration of LCEVC, Collabora is already looking at ways to leverage various hardware capabilities to accelerate LCEVC decoding. As a leading developer and contributor for GStreamer, codec support in the Linux kernel and open source GPU drivers, Collabora is uniquely positioned to assist industry members in integrating next generation video processing into GStreamer and other open source frameworks.