GStreamer Multimedia Framework Part 1
Contents
Description
Part 1
Introduction to GStreamer
Part 2
GStreamer Plugin Internals
Part 3
Advanced GStreamer Concepts
Part 4
Concepts of GStreamer Application
Part 5
An Example Application Code
2
GStreamer Design Principles • Graph based structure of connecting various types of media handling components. This allows for arbitrary pipelines to be constructed. • Provides a comprehensive framework core with: Object oriented design based entirely on GLib 2.0 object model Multi-threaded design allowing for seamless handling of complex media scenarios A well designed and simple API An inbuilt debugging system An efficient media representation and buffer management system
3
GStreamer Design Principles – Contd. • Intelligent plugin architecture Allows for only required components to be loaded in to memory. Scalable from a low capability embedded system to high end desktop. Media capability system and the use of registry allows for run time selection of appropriate components.
• Developer’s support Powerful tools for quick prototyping and debugging Extensive API documentation available on the internet Active user community. Developers can register to various online groups and follow recent developments. The main references that anybody should refer to first: - Application Developers Manual - Plugin Writer’s Guide - Core Design Documentation
Hands on: Browse GStreamer website to locate these reference documents
4
pkg-config, glib, gtype and gobject Overview
What is GLib? • GLib is a cross platform software utility library which abstracts OS specific interfaces • Applications that use GLib can be ported across operating systems without any major changes • GLib was developed as part of GTK (which is a popular UI toolkit) but is now released a separate software library. This allows for non-GTK (i.e. non-UI) based applications to also use the facilities provided by GLib.
References References
http://library.gnome.org/devel/glib/stable/ http://library.gnome.org/devel/glib/stable/
6
GLib features • Defines portable data types 1
New easy-to-use data types that are not part of standard C
gboolean, gsize, gssize
2
New integer types that are guaranteed to be the same size across all platforms
gint8, guint8, gint16, guint16, gint32, guint32
3
Types that are easy to use than their C counterparts
gpointer, gconstpointer, guchar, guint, gushort, gulong
• Provides OS agnostic functions for 1
Managing threads, semaphores and mutexes
2
Utilities: string manipulation, date and time, random number generation, file management
3
Data types: singly linked lists, doubly linked lists, hash tables, arrays of arbitrarily sized elements, etc.
7
What is GObject? • GObject is a complete object oriented framework/system that can be used by applications implemented using pure C language. • It’s like any other software library used by C programs and doesn’t add any new syntax to the C programming language itself. The object oriented structure is provided through the use of C structures and function pointers. • GObject is an independent library by itself but it’s source code is maintained and delivered as part of the GLib package. • The cornerstone of GObject functionality is a runtime type management system called GType. • Main features of GObject framework include Fundamental GObject type implementation which can be used to build object hierarchies Signaling system for enabling objects to interact through notification mechanism Generic Framework for defining custom read/write properties for derived GObjects Reference Reference
http://library.gnome.org/devel/gobject/stable/ http://library.gnome.org/devel/gobject/stable/ (API) (API) http://library.gnome.org/devel/gobject/stable/pt02.html http://library.gnome.org/devel/gobject/stable/pt02.html (Tutorial) (Tutorial)
8
The GObject fundamental type – base class for all objects • GObject is also the name for the fundamental object type. All other objects have to derive from the fundamental GObject type. • Two structures will be used to implement each GObjects, including the GObject fundamental data type struct GObject
struct GObjectClass
Contains Contains declaration of data members
declaration of virtual member functions (function pointers)
One copy in memory per instance
Single copy in memory combined for all instances
9
Types of functions supported in GObject 1. Non virtual public methods 2. Virtual public methods 3. Virtual private methods
1. Non Virtual Public Methods • Implement a C wrapper function, which accepts a pointer to the GObject type as its first argument. • The definition of this function is exposed in the header file • No function pointer is defined in the corresponding class structure • The C wrapper function operates directly on the GObject type Non Virtual Public Method Example
10
Types of functions supported in Gobject … contd. 2. Virtual Public Methods •
Implement a C wrapper function, which accepts a pointer to the GObject type as its first argument.
•
The definition of this function is exposed in the header file
•
Define a function pointer for the internal function (not exposed through header file) for this method. The function pointer is added to the corresponding class definition.
•
The wrapper function does not operate directly on the GObject type and it just invokes the internal function through the function pointer. Virtual Public Method Example
3. Virtual Private Methods •
Same as virtual public method, except that there is no wrapper function implemented. Access is only through the function pointer defined in the class structure.
11
Deriving from GObject fundamental type – data members
3. Derived object – level 3
2. Derived object – level 1
1. GObject Structure
Opaque Data
Opaque Data
Opaque Data
Custom field 1.1
Custom field 1.1
Custom field 1.2
Custom field 1.2 Custom field 2.1
struct Gobject
struct Child1
{
{
}
Mandatory
struct Child2 {
GObject parent;
Child1 parent;
gint32 field1;
gint32 field1;
gint32 field2;
}
}
12
Deriving from GObject fundamental type – functions vmethod_1 struct GDerived1Class {
GDerived1 Class
GObjectClass parent_class;
vmethod_1_fp
void (*vmethod_1_fp) (void);
vmethod_2_fp
void (*vmethod_2_fp) (void);
vmethod_2
}
GDerived2 Class struct GDerived2Class { GDerived1Class parent_class;
vmethod_1_fp vmethod_2_fp
vmethod_2_overridden
} Function pointer reassigned during class initialisation
13
GStreamer Concepts Discussion
The core and plugins • A major and important design aspect of GStreamer is the separation it identifies between framework core and plugins. • GStreamer core: Implements the control logic required to manage the complete framework - Plugin selection, loading and branch/pipeline creation - Media type negotiation - Controlling data flow - Provides a message bus which applications can use to get notifications - Plugin registry to aid in dynamic loading of plugins
Is very generic and does not assume any codec/file-format/network protocol/peripheral specific details. Applications interact with framework and do not generally need to interact with plugins
15
The core and plugins … contd. • Plugins are independent modules (implemented as libraries) which perform a specific defined function. • To qualify as a plugin a module has to implement a well defined plugin interface. Plugins are treated as objects and hence they follow the the GObject hierarchy. • Plugins can be classified based on their functionality in to the following groups Protocol handlers (e.g. rtsp, http, etc) Sources (e.g. file source, camera source, audio (mic) source, etc) File format handlers: parsers, demuxers and muxers. Codecs (e.g. MP3 decoder, etc) Filters (e.g. colour converter, audio resamplers, etc) Sinks (e.g. audio renderer, LCD display, etc)
• The scalability feature of GStreamer is predominantly due to the clean separation between core and plugins. For example support for handling an altogether new format can be added to the system by implementing appropriate plugins and without making any changes to the core.
16
The core and plugins ‌ contd.
Image: Taken from GStreamer Application Development Manual (GStreamer Website)
17
Thank You