Xavier Claessens
April 08, 2024
Share this post:
Reading time:
GStreamer 1.24 was recently released, and with it came a new plugin to help separate pipelines into different processes.
The new unixfd plugin provides a sink and source elements. They communicate over a unix socket to provide a zero-copy transfer of buffers across pipelines in different processes. Buffers must be backed by file-descriptor memories, such as memfd (see below) or DMABUF (e.g. v4l2src).
One unixfdsink can have multiple unixfdsrc clients, acting like a cross process "tee". However, no caps negotiation happens between pipelines; caps are negotiated by the upstream pipeline alone and then sent from unixfdsink to unixfdsrc. Events and queries are also not currently transmitted, except for caps and EOS.
Example:
$ gst-launch-1.0 v4l2src device=/dev/video0 ! unixfdsink socket-path=/tmp/mysocket $ gst-launch-1.0 unixfdsrc socket-path=/tmp/mysocket ! queue ! autovideosink $ gst-launch-1.0 unixfdsrc socket-path=/tmp/mysocket ! queue ! autovideosink
Here we have tree pipelines in different processes transferring webcam frames as DMABUF (depending on webcam capabilities) and displayed twice.
In addition, unixfdsink will propose the new shared memory allocator (GstShmAllocator) in its allocation query. On Linux it uses memfd by default, but it can use POSIX shm_open as fallback. This allows for example videotestsrc to generate frames directly into file-descriptor memories that can then be transferred between processes with no copy:
$ gst-launch-1.0 videotestsrc ! unixfdsink socket-path=/tmp/mysocket $ gst-launch-1.0 unixfdsrc socket-path=/tmp/mysocket ! queue ! videoconvert ! autovideosink
In order to be able to transmit raw video frames, metadata must also be transferred, such as GstVideoMeta. New functions gst_meta_serialize() and gst_meta_deserialize() have been implemented on most GstMeta to transfer them over the unix socket. That API could also be used in the future to store buffer metadata into a file using e.g. <codegdppay.
Here's a look at this new plugin in action:
We thank our partners at Netflix for sponsoring this work.
29/04/2024
GStreamer's support for diverse hardware and software platforms extends to its upstream Machine Learning capabilities, exemplified by its…
22/04/2024
As part of our commitment to social responsibility, we've been part of the 1% for the Planet Network since 2022. The environmental partners…
17/04/2024
Monado, the cross-platform open source XR runtime, has recently received significant updates to align with the features and specifications…
Whether writing a line of code or shaping a longer-term strategic software development plan, we'll help you navigate the ever-evolving world of Open Source.
한국어 버전의 Collabora.com 보기
Acesse Collabora.com em Português
Comments (0)
Add a Comment