[[PageOutline]]

[=#audio]
= Audio =
This page will attempt to give examples of audio-based GStreamer pipelines using GStreamer-1.0, with 'enhanced' encoders and decoders from the gstreamer-imx set of plugins. Note that 'enhanced' just means that the software algorithms are especially tuned for the imx6 family.

To see GStreamer-0.10 (deprecated) examples, please refer to [http://trac.gateworks.com/wiki/Yocto/gstreamer/audio?version=2 this older revision page].

For a description of Audio devices see:
 * [wiki:ventana/audio Ventana Audio devices]


[=#gstreamer-alsa]
== GStreamer alsasrc and alsasink ==
The GStreamer {{{alsasrc}}} and {{{alsasink}}} provide audio capture and playback to Advanced Linux Sound Architecture (ALSA) devices. ALSA is the modern device-driver API for the Linux kernel.

The Gstreamer alsasrc and alsasink can be passed a {{{device}}} property which can represent either the device name (reported from {{{aplay -L}}} for playback, or {{{arecord -L}}} for record) (ie {{{sysdefault:CARD=sgtl5000audio}}}) or the hw:x,y notation (ie {{{hw:0,0}}} for first device, first subdevice)

If not specified via the {{{device}}} property, the device used for alsasrc and alsasink will depend on {{{/etc/asound.conf}}} and/or {{{~/.asoundrc}}}.

Playback and Capture also depends on your ALSA mixer ({{{amixer}}}) settings. See [wiki:ventana/audio#Mixermuxgainvolume mixer settings] for more info.


[=#audio-output]
== Audio Output ==
Generally, a 'sink' plugin is one that will take an audio stream and send it to an audio device. Please refer to [http://trac.gateworks.com/wiki/ventana/audio this audio] page for more details on audio devices on the ventana platform.

A list of output sinks on the imx6:
 * alsasink (recommended)
 * autoaudiosink
 * fakesink
 * filesink


=== alsasink ===
This audio sink is our recommended audio sink. It can accept the following formats via {{{audio/x-raw}}}: S8, U8, S16LE, S16BE, U16LE, U16BE, S24_32LE, S24_32BE, U24_32LE, U24_32BE, S32LE, S32BE, U32LE, U32BE, S24LE, S24BE, U24LE, U24BE, S20LE, S20BE, U20LE, U20BE, S18LE, S18BE, U18LE, U18BE, F32LE, F32BE, F64LE, F64BE


=== fakesink ===
This is a very useful debugging sink. It takes whatever data is given to it and drops them. This might help debugging pipelines if problems ever arise.


=== filesink ===
This sink is very useful when a user wants to record and save audio to a file.


=== autoaudiosink ===
This GStreamer sink is not really an 'audio' sink in the traditional sense. Similar to {{{playbin}}} and {{{decodebin}}}, this element selects what it thinks is the best available audio sink and uses it. Generally not recommended.


=== Examples ===
The GStreamer {{{alsasink}}} element will playback a 1KHz tone to any Advanced Linux Sound Architecture (ALSA) device driver:
{{{#!bash
gst-launch-1.0 audiotestsrc ! alsasink device="hw:0,0"
}}}
 * see [wiki:ventana/audio#devices here] for details on specifying audio input and output devices


[=#audio-input]
== Audio Input ==
An input source is anything coming from a capture device on the SBC, e.g. HDMI audio in/analog audio in. Please refer to [http://trac.gateworks.com/wiki/ventana/audio this audio] page for more details on audio devices on the ventana platform.

A list of input sources on the imx6:
 * alsasrc (recommended)
 * autoaudiosrc
 * audiotestsrc
 * filesrc


=== alsasrc ===
This is the recommended audio capture source when capturing audio onto the SBC. This plugin can output the following source types in {{{audio/x-raw}}}: S8, U8, S16LE, S16BE, U16LE, U16BE, S24_32LE, S24_32BE, U24_32LE, U24_32BE, S32LE, S32BE, U32LE, U32BE, S24LE, S24BE, U24LE, U24BE, S20LE, S20BE, U20LE, U20BE, S18LE, S18BE, U18LE, U18BE, F32LE, F32BE, F64LE, F64BE


=== autoaudiosrc ===
This GStreamer source is not really an 'audio' source in the traditional sense. Similar to {{{playbin}}} and {{{decodebin}}}, this element selects what it thinks is the best available audio source and uses it. Generally not recommended.


=== audiotestsrc ===
This is a very useful plugin for testing. It can output 16-64bit {{{audio/x-raw}}}: S16LE, S32LE, F32LE, F64LE

This can output audio signals from {{{sine}}} to {{{violet-noise}}}.


=== filesrc ===
This source is very useful when a user wants to playback audio from a file.


=== Examples ===
The GStreamer {{{alsasrc}}} element will capture audio from any Advanced Linux Sound Architecture (ALSA) device driver:
{{{#!bash
gst-launch-1.0 alsasrc device="hw:0,0" ! "audio/x-raw,rate=32000,channels=2,depth=16" ! \
  audioconvert ! avimux ! filesink location=./audio.avi
}}}
 * see [wiki:ventana/audio#devices here] for details on specifying audio input and output devices


[=#audio-encoding]
== Encoding ==
Encoding is the term used to capture audio and encode it to a new format type.

A list of audio encoders on the imx6 platform:
 * {{{gstreamer-imx}}} specific
  * imxmp3audioenc
 * Other GStreamer encoders
  * vorbisenc
  * mulawenc
  * wavenc
  * alawenc
  * flacenc
  * lamemp3enc

There are many more. You can search for your specified one by running a similar search: {{{gst-inspect-1.0 | grep enc}}}.


=== imxmp3audioenc ===
This encoder can encode {{{audio/x-raw}}} S16LE to {{{audio/mpeg}}}.


=== vorbisenc ===
This encoder can encode {{{audio/x-raw}}} F32LE to {{{audio/x-vorbis}}}.


=== mulawenc ===
This encoder can encode {{{audio/x-raw}}} S16LE to {{{audio/x-mulaw}}}.


=== wavenc ===
This encoder can encode {{{audio/x-raw}}} S32LE, S24LE, S16LE, U8, F32LE, F64LE, {{{audio/x-alaw}}}, and {{{audio/x-mulaw}}} to {{{audio/x-wav}}}.


=== alawenc ===
This encoder can encode {{{audio/x-raw}}} S16LE to {{{audio/x-alaw}}}.


=== flacenc ===
This encoder can encode {{{audio/x-raw}}} S24LE, S24_32LE, S16LE, S8 to {{{audio/x-flac}}}.


=== lamemp3enc ===
This encoder can encode {{{audio/x-raw}}} S16LE to {{{audio/mpeg}}}.


=== Examples ===
This will take in an audio source from the mic and save it as an mp3.
{{{#!bash
gst-launch-1.0 alsasrc device="hw:0,0" ! audioconvert ! imxmp3audioenc ! filesink location=/tmp/mic.mp3
}}}
 * Note the audioconvert element is used to convert audio samplerate / bitwidth / format (via software algorithms) as needed between two elements.


[=#audio-decoding]
== Decoding ==
Decoding is the term used to decode an encoded audio stream to a raw audio stream.

A list of audio encoders on the imx6 platform:
 * {{{gstreamer-imx}}} specific
  * imxuniaudiodec
 * Other GStreamer encoders
  * vorbisdec
  * ivorbisdec
  * a52dec
  * mulawdec
  * alawdec
  * flacdec


=== imxuniaudiodec ===
This decoder can decode {{{audio/mpeg}}} to {{{audio/x-raw}}} S32LE, S24LE, S16LE, S8.


=== vorbisdec ===
This decoder can decode {{{audio/x-vorbis}}} to {{{audio/x-raw}}} F32LE.


=== ivorbisdec ===
This decoder can decode {{{audio/x-vorbis}}} to {{{audio/x-raw}}} S16LE.


=== a52dec ===
This decoder can decode {{{audio/x-ac3}}}, {{{audio/ac3}}}, and {{{audio/x-private1-ac3}}} to {{{audio/x-raw}}} F32LE.


=== mulawdec ===
This decoder can decode {{{audio/x-mulaw}}} to {{{audio/x-raw}}} S16LE.


=== alawdec ===
This decoder can decode {{{audio/x-alaw}}} to {{{audio/x-raw}}} S16LE.


=== flacdec ===
This decoder can decode {{{audio/x-flac}}} to {{{audio/x-raw}}} S8, S16LE, S24_32LE, S32LE.


=== Examples ===
We can decode the MP3 audio saved previously and playback to the default audio device  with the following:
{{{#!bash
gst-launch-1.0 filesrc location=/tmp/mic.mp3 ! mpegaudioparse ! imxuniaudiodec ! alsasink
}}}


[=#audio-format]
== Audio Format Conversion ==
In specify the details of the audio format, you use a GStreamer {{{capsfilter}}} and to convert it from one format to another you use the GStreamer {{{audioconvert}}} element.
{{{#!bash
gst-launch-1.0 alsasrc device="hw:0,0" ! "audio/x-raw,rate=32000,channels=2,depth=16" ! \
  audioconvert ! avimux ! filesink location=./audio.avi
}}}


[=#audio-loopback]
== Audio loopback (Useful for testing audio input and output) ==
A simple aduio loopback test can take audio input from an input device and output it to an output device.

loopback audio from the first audio card:
{{{#!bash
gst-launch-1.0 alsasrc device="hw:0,0" ! alsasink device="hw:0,0"
}}}

To send audio from the sgtl5000 (analog input) to the imxhdmisoc (HDMI output):
 * using device/function numbers (ie from {{{aplay -l}}}):
{{{#!bash
gst-launch-1.0 alsasrc device="hw:0,0" ! alsasink device="hw:2,0"
}}}
 * or, using names (ie from {{{aplay -L}}}):
{{{#!bash
gst-launch-1.0 alsasrc device="sysdefault:CARD=sgtl5000audio" ! \
 alsasink device="sysdefault:CARD=imxhdmisoc"
}}}

To send audio from the tda1997x HDMI receiver (digital input) to the imxhdmisoc (digital output):
{{{#!bash
gst-launch-1.0 alsasrc device="sysdefault:CARD=tda1997xaudio" ! \
  "audio/x-raw,rate=44100" ! alsasink device="sysdefault:CARD=imxhdmisoc"
}}}
  * Note here we need to specify the audio sample-rate as it can vary per input stream (and gstreamer does not validate the rate). This rate must match the source stream samplrate which can be found via sysfs {{{/sys/bus/i2c/drivers/tda1997x/2-0048/audmode}}}. If your output device requires a different sample-rate than the source input device, you need to perform a sample-rate conversion.