GStreamer is a pipeline-based multimedia framework that links together a wide variety of media processing systems to complete complex workflows. For instance, GStreamer can be used to build a system that reads files in one format, processes them, and exports them in another. The formats and processes can be changed in a plug and play fashion.
GStreamer supports a wide variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing. The pipeline design serves as a base to create many types of multimedia applications such as video editors, transcoders, streaming media broadcasters and media players.
It is designed to work on a variety of operating systems, e.g. the BSDs, OpenSolaris, Android, macOS, iOS, Windows, OS/400.
GStreamer is free and open-source software subject to the terms of the LGPL-2.1-or-later and is being hosted at freedesktop.org.
Distribution and adoption
The GNOME desktop environment, a heavy user of GStreamer, has included GStreamer since GNOME version 2.2 and encourages GNOME and GTK applications to use it. Other projects also use or support it, such as the Phonon media framework and the Songbird media player. It is also used in the WebKit browser engine.
GStreamer also operates in embedded devices like the Jolla Phone, the Palm Pre, Tizen and the Nokia 770, N800, N810, N900 and N9 Internet Tablets running the Maemo operating system.
In addition to source code releases, the GStreamer project provides binary builds for Android, iOS, OSX and Windows.
The LIGO Laboratory make use of GStreamer to simulate and analyze gravitational wave data. The GStreamer interface is called GstLAL.
Software architecture
GStreamer is written in the C programming language with the type system based on GObject and the GLib 2.0 object model.
Language bindings
A library written in one programming language may be used in another language if bindings are written; GStreamer has a range of bindings for various languages such as Go, Python, Rust, Vala, C++, Perl, GNU Guile, C# and Ruby.
Overview
GStreamer processes media by connecting a number of processing elements into a pipeline. Each element is provided by a plug-in. Elements can be grouped into bins, which can be further aggregated, thus forming a hierarchical graph. This is an example of a filter graph.
Elements communicate by means of pads. A source pad on one element can be connected to a sink pad on another. When the pipeline is in the playing state, data buffers flow from the source pad to the sink pad. Pads negotiate the kind of data that will be sent using capabilities.
The diagram to the right could exemplify playing an MP3 file using GStreamer. The file source reads an MP3 file from a computer's hard-drive and sends it to the MP3 decoder. The decoder decodes the file data and converts it into PCM samples which then pass to the sound-driver. The sound-driver sends the PCM sound samples to the computer's speakers.
Plug-ins
GStreamer uses a plug-in architecture which makes the most of GStreamer's functionality implemented as shared libraries. GStreamer's base functionality contains functions for registering and loading plug-ins and for providing the fundamentals of all classes in the form of base classes. Plug-in libraries get dynamically loaded to support a wide spectrum of codecs, container formats, input/output drivers and effects.
Plug-ins can be installed semi-automatically when they are first needed. For that purpose distributions can register a backend that resolves feature-descriptions to package-names.
Since version 0.9, the plug-ins come grouped into three sets (named after the film The Good, the Bad and the Ugly).
Individual distributions may further sub-classify these plug-ins: for example Ubuntu groups the "bad" and "ugly" sets into the "Universe" or the "Multiverse" components.
In addition, there is a GStreamer FFmpeg plug-in (called gst-libav for historic reasons) that extends the number of supported media formats.
Video acceleration
There are various SIP blocks that can do the computations to decode certain video codecs, such as PureVideo, UVD, QuickSync Video, TI Ducati and more. Such needs to be supported by the device driver, which in turn provides one or multiple interfaces, like VDPAU, VAAPI, Distributed Codec Engine or DXVA to end-user software like MPlayer to access this hardware and offload computation to it.