Skip to main content
FOSDEM PWA

Open Source video mixing and syncing with real-time control

K.4.601 | Day 1 | 14:10 - 14:30 | Speakers: Jaron Viëtor

Open Source video mixing and syncing with real-time control
A picture of a devroom at FOSDEM 2024
Open in browser
Get involved in the conversation!Join the chat

Notes

Abstract

As of the 3.10 release, the public domain (Unlicense) media server MistServer (https://mistserver.org) gained a new feature: the ability to mix raw (UYVY pixel format only, for now) video streams, raw audio streams (PCM) and PNG images with resizing, overlapping, aspect ratio keeping and support for non-uniform frame rates between sources. Not only that - but it's even possible to control the configuration in real time without any downtime. This talk shows off what is possible, and explains how we did it in technical detail.

Covered topics:

  • How to efficiently store a multi-frame raw video buffer in shared memory
  • Synchronization handling between multiple sources
  • Handling sources being added or removed without interruptions
  • How we implemented decoding and encoding between raw and encoded formats
  • The user interface that was built to control the mixing in a user-friendly way (though "raw" control through JSON is also possible)

Speakers

Jaron Viëtor

Jaron (AKA "Thulinma") is one of the founders of the MistServer project and currently serves as the team lead. He's been writing code since 1993 and is active in the media industry since 2009, with all of his work on MistServer being released into the public domain (Unlicense).

Notice: The placeholder video image is licensed under CC BY-SA 4.0. The original image can be found hereChanges made to the image are: Cropped the image to a new ratio, part of the image was cut off.