For remote security surveillance, like monitoring an industrial facility where expensive equipment or even human lives might be at stake, maintaining an immediate and high quality video streaming from the areas of interest is a must. With the advent of 5G networks, it's now possible to stream high quality video in real-time with a very low latency that wasn't possible with the past generations of mobile networks. In this domain, the SRT protocol has been picking up speed, and thanks to srtsrc and srtsink elements available since GStreamer 1.16 (see Olivier Crête's blog post) it's now easier than ever to incorporate low latency streaming into your application.

Here at Collabora we've been lately participating in the design and development of Hwangsaeul—a next generation security video feeds streaming platform with one of our customers, SK Telecom Co..

Hwangsaeul is a cloud relay service that gathers live security video feeds from different locations into a single service to which clients can connect to watch the feeds. Additionally, it also enables continuous recording of each feed. The SRT protocol is utilized by both camera-to-relay and relay-to-client transport in order to minimize latency.

Figure 1. Hwangsaeul architecture. (Image courtesy of Jeongseok Kim @ SK Telecom Co.)

The platform consists of a collection of modules communicating through RabbitMQ message broker. For this introduction, three components will be important. The names are in Korean. (Hwangsaeul itself means "stork's nest". It is the street in Seongnam where SK Telecom offices were located.)

Gaeul (개울, brook) is a streaming server that runs on each edge node contributing a video stream to the service. It accesses the video capture device through V4L2 and registers the node with the platform's message broker, allowing video capture and streaming to be remotely controlled from the platform's relay service.

Hwangsae (황새, stork) fulfills two distinct roles: it implements the core SRT relay that provides M:N streaming of the content (capture from one camera may be viewed by many clients at a time) and can record streams into MPEG-TS or MP4 files for later viewing.

Chamge (참게, a small crab) takes care of communication between platform components. On one hand it provides a wrapper around RabbitMQ client library for other services, on the other it implements the Arbiter service, which keeps track of all video capture and relay nodes registered with the platform and facilitates their interaction.

Installation

Hwangsaeul is being developed under Apache 2.0 license with the source codes hosted on GitHub. Nightly builds are also available as binary packages for Ubuntu 18.04 and 19.04. To try the platform in its present development shape, add the following PPA:

sudo add-apt-repository ppa:hwangsaeul/nightly
sudo apt-get update

In production, the components would be divided across multiple servers with distinct purposes as stream sources, relays, recorders. Here we can, for the sake of simplicity, install everything on one machine:

apt-get install gaeul hwangsae-relay hwangsae-recorder chamge-arbiter

Configuration

AMQP message broker

The platform needs a running AMQP broker instance, which can be either installed on-premises, or for our purposes a free account on a cloud service like [cloudamqp.com](https://www.cloudamqp.com/) will be sufficient as well. In any case you'll need your broker's URI to configure Hwangsaeul.

/etc/hwansaeul.conf

As the next step, edit the parameters in `/etc/hwangsaeul.conf`. The excerpt below shows the minimal configuration. You may change the V4L2 device node to match the camera you want to use for streaming, and don't forget to set the correct URI of your AMQP broker.

    [org/hwangsaeul/Gaeul]
    video-device='/dev/video0'
    video-source='v4l2src'
    encoding-method='general'

    [org/hwangsaeul/Chamge1/Edge/AMQP]
    amqp-uri='amqp://user:password@server.com/user'

    [org/hwangsaeul/Chamge1/Arbiter/AMQP]
    amqp-uri='amqp://user:password@server.com/user'

    [org/hwangsaeul/Chamge1/Hub/AMQP]
    amqp-uri='amqp://user:password@server.com/user'

When the configuration is in place, make sure you restart all Hwangsaeul services.

    systemctl restart chamge-arbiter
    systemctl restart gaeul
    systemctl restart hwangsae-relay
    systemctl restart hwangsae-recorder

D-Bus API

Hwangsaeul is being developed as a platform that integrates into 3rd party applications. To that end, it exposes its functionality as a set of D-Bus interfaces. Here is a couple of examples demonstrating its usage.

Streaming

In order to initiate a video stream, we first need to know its source. Streaming sources in Hwangsaeul are identified by their unique Edge ID, which can be obtained from its respective gaeul process over D-Bus using org.hwangsaeul.Gaeul.Manager interface:

    $ busctl introspect org.hwangsaeul.Gaeul /org/hwangsaeul/Gaeul/Manager  org.hwangsaeul.Gaeul.Manager
    NAME                         TYPE      SIGNATURE RESULT/VALUE FLAGS
    .GetEdgeId                   method    -         s            -
    .State                       property  i         0            emits-change

    $ busctl call org.hwangsaeul.Gaeul /org/hwangsaeul/Gaeul/Manager  org.hwangsaeul.Gaeul.Manager GetEdgeId
    s "9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879"

Streaming itself is controlled through org.hwangsaeul.Hwangsae1.EdgeInterface of Hwangsae relay agent:

    $ busctl introspect org.hwangsaeul.Hwangsae1.RelayAgent /org/hwangsaeul/Hwangsae1/EdgeInterface \
      org.hwangsaeul.Hwangsae1.EdgeInterface
    NAME                                   TYPE      SIGNATURE RESULT/VALUE FLAGS
    .ChangeParameters                      method    siiii     -            -
    .Delete                                method    s         -            -
    .GetEdgeStatus                         method    s         u            -
    .Register                              method    su        -            -
    .Start                                 method    siiii     s            -
    .Stop                                  method    s         s            -
    .Edges                                 property  u         0            emits-change

To start streaming from the edge device whose ID we got in the previous step, one can run:

    $ busctl call org.hwangsaeul.Hwangsae1.RelayAgent /org/hwangsaeul/Hwangsae1/EdgeInterface org.hwangsaeul.Hwangsae1.EdgeInterface \
      Start 'siiii' '9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879' 640 480 30 2048000

The call arguments that follow the Edge ID are the requested width, height, framerate and bitrate in bits per second of the stream. Upon receiving the command, Hwangsaeul will turn on the camera and initiate SRT connection from the edge device to the relay service. If everything goes well, the D-Bus call will return a URL on the relay server from which clients can play the video stream.

    s "srt://192.168.47.155:9999"

The GStreamer's gst-play-1.0 tool supports srt:// URLs, but they should also work in recent versions of common media players like VLC or GNOME Videos (Totem).

    gst-play-1.0 srt://192.168.47.155:9999

Figure 2. Receiving a SRT stream with gst-play-1.0.

Recording

Stream recording is controlled through org.hwangsaeul.Hwangsae1.RecorderInterface D-Bus interface:

    $ busctl introspect org.hwangsaeul.Hwangsae1.RecorderAgent /org/hwangsaeul/Hwangsae1/RecorderInterface org.hwangsaeul.Hwangsae1.RecorderInterface
    NAME                                       TYPE      SIGNATURE RESULT/VALUE FLAGS
    .Delete                                    method    ss        -            -
    .LookupByEdge                              method    sxx       a(ssxxx)     -
    .LookupByRecord                            method    sxx       sa(sxxx)     -
    .Start                                     method    s         s            -
    .Stop                                      method    s         -            -
    .Url                                       method    ss        s            -

To begin recording from an edge device, we only need to know its ID.

    $ busctl call org.hwangsaeul.Hwangsae1.RecorderAgent /org/hwangsaeul/Hwangsae1/RecorderInterface org.hwangsaeul.Hwangsae1.RecorderInterface \
      Start 's' '9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879'

Hwangsaeul will continuously save the stream into a file. In the case when the connection with the stream source gets broken, the recorder will keep making attempts to re-establish it until explicitly told to stop.

Stopping the recording is straightforward too:

    $ busctl call org.hwangsaeul.Hwangsae1.RecorderAgent /org/hwangsaeul/Hwangsae1/RecorderInterface org.hwangsaeul.Hwangsae1.RecorderInterface \
      Stop 's' '9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879'

The file we've just created is kept in the recorder's internal storage. Hwangsaeul provides several methods for searching through the directory of recordings; we will use LookupByEdge:

<!--
LookupByEdge:
@edgeId:          a unique ID of an edge device
@from:            start timestamp of the time range in which to search
                  in microseconds since Jan 01 1970
@to:              end timestamp of the time range in which to search
                  in microseconds since Jan 01 1970

@records:         an array where each item represents a found recording 

Looks up recordings from an edge device taken during given time range.

The meaning of the values in each array item is: recording ID, fileID,
recording start, recording end (both μs since Epoch) and file size in Bytes.
-->
<method name="LookupByEdge">
  <arg name="edgeId" direction="in" type="s"/>
  <arg name="from" direction="in" type="x"/>
  <arg name="to" direction="in" type="x"/>
  <arg name="records" direction="out" type="a(ssxxx)"/>
</method>

So, to find our recording run the following request. Because we don't want to filter the results by any time range, passing 0 and 0xffffffffffffff as from and to will encompass all possible timestamps.

    $ busctl call org.hwangsaeul.Hwangsae1.RecorderAgent /org/hwangsaeul/Hwangsae1/RecorderInterface org.hwangsaeul.Hwangsae1.RecorderInterface \
      LookupByEdge 'sxx' '9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879' 0 0xffffffffffffff

The call returns one entry. Meaning of the values is described above.

    a(ssxxx) 1 "1580479418277126" "1580479418277126-1580479418631803-1580479735978654" 1580479418631803 1580479735978654 140987404

Let's download the file. Normally, the recordings would be served by a full-fledged server like NGINX. In our demonstration we'll take advantage of the small HTTP server included in Hwangsae Recorder. First we need to convert the ID of the requested file to its URL:

<!--
Url:
@edgeId:          a unique ID of an edge device
@fileId:          a unique ID of a recording file

@url:             URL using which the recording can be downloaded from the
                  embedded HTTP server.  

Gets the URL of a file.
-->
<method name="Url">
  <arg name="edgeId" direction="in" type="s"/>
  <arg name="fileId" direction="in" type="s"/>
  <arg name="url" direction="out" type="s"/>
</method>

Thus, knowing edge and file IDs:

    $ busctl call org.hwangsaeul.Hwangsae1.RecorderAgent /org/hwangsaeul/Hwangsae1/RecorderInterface org.hwangsaeul.Hwangsae1.RecorderInterface \
      Url 'ss' '9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879' '1580479418277126-1580479418631803-1580479735978654'
    s "http://192.168.47.155:8090/9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879/1580479418277126-1580479418631803-1580479735978654"

Now we can download the recording using your preferred tool.

    $ wget --content-disposition http://192.168.47.155:8090/9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879/1580479418277126-1580479418631803-1580479735978654
    --2020-02-04 16:00:55--  http://192.168.47.155:8090/9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879/1580479418277126-1580479418631803-1580479735978654
    Connecting to 192.168.47.155:8090... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 140987404 (134M)
    Saving to: ‘hwangsae-recording-1580479418277126-1580479418631803-1580479735978654.ts’

    hwangsae-recording-1580479418277126- 100%[======================================================================>] 134.46M   779MB/s    in 0.2s    

    2020-02-04 16:00:55 (779 MB/s) - ‘hwangsae-recording-1580479418277126-1580479418631803-1580479735978654.ts’ saved [140987404/140987404]

Next steps

Hwangsaeul streaming service is still in its early development stage, but it already provides the most fundamental APIs for 3rd party consumption. A key missing requirement is stream encryption, ideally through a mechanism that would be compatible also with common web browsers. There are also possibilities to extend SRT support in GStreamer, for example by implementing network adaptive encoding in order to improve the service quality.

Our team at Collabora would love to help you integrate SRT into your platform, using GStreamer, ffmpeg, VLC or your own multimedia framework. Contact us today to see how we can help!

Related Posts

SRT in GStreamer

SRT, typical examples

Secure video comes of age

Related Posts

SRT in GStreamer

SRT, typical examples

Secure video comes of age