GStreamer Multimedia Framework Part 4
Contents
Description
Part 1
Introduction to GStreamer
Part 2
GStreamer Plugin Internals
Part 3
Advanced GStreamer Concepts
Part 4
Concepts of GStreamer Application
Part 5
An Example Application Code
2
The pipeline’s message bus Mechanism for interaction with Application
Introduction to GstBus • GstBus is a facility provided by the GStreamer Pipeline which applications can use to receive messages from framework core. • The messages delivered over GstBus are represented using the GstMessage data type. • Messages are delivered on the bus in FIFO order. If application do not pick up, messages will remain in the bus. • Example list of messages include: End of stream State change notification for each element in the pipeline Error, Info and Warning messages posted by elements in the pipeline Clock notifications, etc
• Applications would register a function (i.e. add a watch) with the pipeline’s bus. The registered function would get invoked whenever a message is to be delivered. • The message delivery mechanism used by GstBus fits in to the generic event loop handling used in GTK based applications.
4
GstBus and Application Threading Architecture application App Thread
Application uses GStreamer core APIs to
Messages from different streaming threads
invoke pipeline functionality
are accumulated by GstBus and delivered to
GstBus
pipeline filesrc
queue
the application in the context of application’s main loop itself.
Video dec
video sink Thread 2
demuxer Thread 1
Thread 3
queue
audio dec
audio sink
5
Signals Elements interacting with application
GObject Signals • Another method for applications to get triggered from GStreamer is through Signals. • Signaling mechanism is implemented by GObject (i.e. GLib functionality) and GStreamer directly inherits it. • Each GObject, and hence each GstElement, can raise a custom set of signals. Applications can register a callback function for each type of signal with the GstElement directly. • Examples: “have-type” signal emitted by typefind element “pad-added” signals emitted by elements which support dynamic (i.e. sometimes) pads.
• The application’s callback function would be executed in the context of GStreamer streaming thread itself.
7
When to use Bus Messages and When to use Signals? • Bus Messages: Bus messages are usually informational in nature. The pipeline would continue its functionality irrespective of whether the application processed a particular message or not. Pipeline should not get blocked when the application is processing a message as it may affect the timing constraint of pipeline. Hence messages are processed by application in the context of its own thread (glib main loop). If required, applications can register for synchronous delivery of messages where the pipeline streaming thread gets blocked till the application processes the message. The frequency of messages delivered is more compared to the frequency of signals in a pipeline. For example, there would be a message delivered for every state transition of every element in the pipeline.
8
When to use Bus Messages and When to use Signals? • Signals: Signals are more critical triggers than messages as the pipeline’s further functionality would depend on what action would the application take for a particular signal. Examples: - When a “have-type” signal is raised, the application might decide not to support that particular media type and hence tear down the complete pipeline.
- In case of dynamic pipeline construction, the complete pipeline would be constructed only after the “pad-added” signal is received by application.
The application’s callback function would get invoked from the context of the pipeline streaming thread itself. Hence the pipeline thread would be blocked till the application finishes processing the signal. In a pipeline, generally, there would be very few signals that an application needs to handle.
9
GStreamer Application Structure
The Glib main loop • GLib provides a generic event processing loop that applications can use straight away • Event processing loop is typically used in GUI based applications, where the applications needs to process events from different sources (example, window events, keypad events, application backend events, etc) • The messages transferred over GStreamer pipeline bus has support for usage of the glib event processing loop • This way, a GUI based application which uses GStreamer framework can be very easily built
11
Phases of a GStreamer Application Start Initialization of gstreamer elements and pipeline Registration of message and signal handlers
Glib Main Loop
Exit from glib main loop has to be triggered by one of the message handlers (e.g., the message handler for End
Deinitialisation
of Stream Message)
Stop
12
Thank You