Commit 6fe35047 authored by Louis-Philippe Véronneau's avatar Louis-Philippe Véronneau

Merge branch 'docs/streaming' into 'master'

Add documentation for the streaming architecture

See merge request !87
parents 313c4f08 d1a03787
Pipeline #34533 passed with stages
in 2 minutes and 26 seconds
This source diff could not be displayed because it is too large. You can view the blob instead.
......@@ -9,6 +9,7 @@ Ansible Documentation
voctomix.rst
gateway.rst
opsis.rst
streaming.rst
advanced_usage.rst
usb.rst
contact.rst
.. _streaming:
Streaming architecture
==============================
The DebConf video team streaming setup gives users access to `HLS
<https://en.wikipedia.org/wiki/HTTP_Live_Streaming>`_ (HTTP Live Streaming)
streams, that are very easy to wrap for web-based consumption. To push the
streams to the backends, `RTMP
<https://en.wikipedia.org/wiki/Real-Time_Messaging_Protocol>`_ streams are used
on the backend.
Our streaming infrastructure is split in two layers: :ref:`streaming backends
<streaming-backend>` receive a H.264 encoded stream from each room via RTMP, and
generate HLS (HTTP Live Streaming) streams on HTTPS. :ref:`Streaming frontends
<streaming-frontend>` are currently simple caching HTTPS reverse proxies,
allowing load distribution and geographic redirections to reduce the latency
downlading video segments for the clients.
.. image:: _static/streaming.svg
:alt: Graphical representation of all components in the streaming setup
:target: _static/streaming.svg
All TLS certificates (for HTTPS) are handled using either `Let's Encrypt
<https://letsencrypt.org>`_ or by generating self-signed certificates. To
accommodate geographic redirections or other DNS manipulations, when using Let's
Encrypt, all of the challenges are centralized on the streaming backend host,
while all frontends redirect their requests there.
.. _streaming-backend:
Streaming backend
--------------------------
Streaming backends are nginx instances, using the `nginx RTMP module
<https://github.com/arut/nginx-rtmp-module>`_ to listen to RTMP feeds pushed by
the rooms.
The backend is configured as two RTMP applications (``stream`` and ``show``), as well as a HTTPS virtual host.
The ``stream`` RTMP application
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The ``videoteam-stream`` script in each room pushes the feeds to the ``stream``
RTMP application, at URL ``rtmp://<backend>:1935/stream/<room>``. This
application has two purposes:
1. It dumps the incoming stream as a last ditch backup
1. It runs ffmpeg to downscale the incoming stream to lower bandwidth variants.
This ffmpeg instance pushes its synchronized downscaled streams to the
``show`` RTMP application at ``rtmp://<backend>:1935/show/<room>_<quality>``.
The original stream is also pushed unchanged to
``rtmp://<backend>:1935/show/<room>_src``
The ``show`` RTMP application
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The ``show`` RTMP application generates the client-oriented adaptive bandwidth
HLS playlists, which are in the end served through its HTTPS virtual host
``https://<backend>/live/<room>.m3u8``
main adaptive HLS playlist, referencing all the following playlists, with
the bandwidth settings from the configuration
``https://<backend>/live/<room>_src.m3u8``
HLS playlist for the "source quality" stream
``https://<backend>/live/<room>_<quality>.m3u8``
HLS playlist for downscaled streams
The streaming backend HTTPS virtual host
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The streaming backend virtual host mostly serves the live HLS data under the
``/live/`` directory. It also serves the centralized ``.well-known`` directory
used for the frontends to answer the Let's Encrypt challenges.
.. _streaming-frontend:
Streaming frontend
--------------------------
Streaming frontends are nginx instances as well, with a single HTTPS virtual
host. As of now, these nginx instances only perform caching of the HLS playlists
and segments for use by clients, using the nginx proxy module for the ``/live/``
directory.
.. _streaming-geodns:
Geographic redirections
^^^^^^^^^^^^^^^^^^^^^^^
To improve latency of client connections to streams, our streaming frontends are
geographically distributed.
Frontends are expected to have a virtual host that supports all possible domain
names under the "live" subdomain, allowing on-the-fly DNS rearrangements:
- af (Africa)
- an (Antarctica)
- as (Asia)
- as (Asia)
- eu (Europe)
- na (North America)
- oc (Oceania)
- sa (South America)
- local (Clients local to the conference venue)
When using Let's Encrypt to generate certificates, challenge data in the
``.well-known`` directory is centralized on the backend host, and all frontends
redirect to it, so that they can generate certificates for all domains
indifferently of the current DNS configuration.
To help geographic-aware redirections of clients, all streaming frontends
support two specific (sets of) HTTPS endpoints:
``https://<current-frontend>/local-server``
This single endpoint returns a ``text/plain`` response containing the GeoIP
continent code, or ``local`` for clients within the registered local
networks. This allows JavaScript streaming frontends to generate a
geographic-aware URI.
``https://<current-frontend>/redir/<uri>``
Redirects clients to ``https://<code>.<live-domain>/<uri>``, i.e. the proper
geographic frontend detected for their IP address.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment