Changes between Initial Version and Version 1 of Yocto/gstreamer/audio

Timestamp:

10/22/2017 05:28:45 AM (7 years ago)

Author:

trac

Comment:

--

Yocto/gstreamer/audio

@@ +1 @@
+[[PageOutline]]
+
+[=#audio]
+= Audio =
+This page will attempt to give examples of audio-based GStreamer pipelines using GStreamer-1.0, with 'enhanced' encoders and decoders from the gstreamer-imx set of plugins. Note that 'enhanced' just means that the software algorithms are especially tuned for the imx6 family.
+
+To see GStreamer-0.10 (deprecated) examples, please refer to [http://trac.gateworks.com/wiki/Yocto/gstreamer/audio?version=2 this older revision page].
+
+For a description of Audio devices see:
+ * [wiki:ventana/audio Ventana Audio devices]
+
+
+[=#gstreamer-alsa]
+== GStreamer alsasrc and alsasink ==
+The GStreamer {{{alsasrc}}} and {{{alsasink}}} provide audio capture and playback to Advanced Linux Sound Architecture (ALSA) devices. ALSA is the modern device-driver API for the Linux kernel.
+
+The Gstreamer alsasrc and alsasink can be passed a {{{device}}} property which can represent either the device name (reported from {{{aplay -L}}} for playback, or {{{arecord -L}}} for record) (ie {{{sysdefault:CARD=sgtl5000audio}}}) or the hw:x,y notation (ie {{{hw:0,0}}} for first device, first subdevice)
+
+If not specified via the {{{device}}} property, the device used for alsasrc and alsasink will depend on {{{/etc/asound.conf}}} and/or {{{~/.asoundrc}}}.
+
+Playback and Capture also depends on your ALSA mixer ({{{amixer}}}) settings. See [wiki:ventana/audio#Mixermuxgainvolume mixer settings] for more info.
+
+
+[=#audio-output]
+== Audio Output ==
+Generally, a 'sink' plugin is one that will take an audio stream and send it to an audio device. Please refer to [http://trac.gateworks.com/wiki/ventana/audio this audio] page for more details on audio devices on the ventana platform.
+
+A list of output sinks on the imx6:
+ * alsasink (recommended)
+ * autoaudiosink
+ * fakesink
+ * filesink
+
+
+=== alsasink ===
+This audio sink is our recommended audio sink. It can accept the following formats via {{{audio/x-raw}}}: S8, U8, S16LE, S16BE, U16LE, U16BE, S24_32LE, S24_32BE, U24_32LE, U24_32BE, S32LE, S32BE, U32LE, U32BE, S24LE, S24BE, U24LE, U24BE, S20LE, S20BE, U20LE, U20BE, S18LE, S18BE, U18LE, U18BE, F32LE, F32BE, F64LE, F64BE
+
+
+=== fakesink ===
+This is a very useful debugging sink. It takes whatever data is given to it and drops them. This might help debugging pipelines if problems ever arise.
+
+
+=== filesink ===
+This sink is very useful when a user wants to record and save audio to a file.
+
+
+=== autoaudiosink ===
+This GStreamer sink is not really an 'audio' sink in the traditional sense. Similar to {{{playbin}}} and {{{decodebin}}}, this element selects what it thinks is the best available audio sink and uses it. Generally not recommended.
+
+
+=== Examples ===
+The GStreamer {{{alsasink}}} element will playback a 1KHz tone to any Advanced Linux Sound Architecture (ALSA) device driver:
+{{{#!bash
+gst-launch-1.0 audiotestsrc ! alsasink device="hw:0,0"
+}}}
+ * see [wiki:ventana/audio#devices here] for details on specifying audio input and output devices
+
+
+[=#audio-input]
+== Audio Input ==
+An input source is anything coming from a capture device on the SBC, e.g. HDMI audio in/analog audio in. Please refer to [http://trac.gateworks.com/wiki/ventana/audio this audio] page for more details on audio devices on the ventana platform.
+
+A list of input sources on the imx6:
+ * alsasrc (recommended)
+ * autoaudiosrc
+ * audiotestsrc
+ * filesrc
+
+
+=== alsasrc ===
+This is the recommended audio capture source when capturing audio onto the SBC. This plugin can output the following source types in {{{audio/x-raw}}}: S8, U8, S16LE, S16BE, U16LE, U16BE, S24_32LE, S24_32BE, U24_32LE, U24_32BE, S32LE, S32BE, U32LE, U32BE, S24LE, S24BE, U24LE, U24BE, S20LE, S20BE, U20LE, U20BE, S18LE, S18BE, U18LE, U18BE, F32LE, F32BE, F64LE, F64BE
+
+
+=== autoaudiosrc ===
+This GStreamer source is not really an 'audio' source in the traditional sense. Similar to {{{playbin}}} and {{{decodebin}}}, this element selects what it thinks is the best available audio source and uses it. Generally not recommended.
+
+
+=== audiotestsrc ===
+This is a very useful plugin for testing. It can output 16-64bit {{{audio/x-raw}}}: S16LE, S32LE, F32LE, F64LE
+
+This can output audio signals from {{{sine}}} to {{{violet-noise}}}.
+
+
+=== filesrc ===
+This source is very useful when a user wants to playback audio from a file.
+
+
+=== Examples ===
+The GStreamer {{{alsasrc}}} element will capture audio from any Advanced Linux Sound Architecture (ALSA) device driver:
+{{{#!bash
+gst-launch-1.0 alsasrc device="hw:0,0" ! "audio/x-raw,rate=32000,channels=2,depth=16" ! \
+  audioconvert ! avimux ! filesink location=./audio.avi
+}}}
+ * see [wiki:ventana/audio#devices here] for details on specifying audio input and output devices
+
+
+[=#audio-encoding]
+== Encoding ==
+Encoding is the term used to capture audio and encode it to a new format type.
+
+A list of audio encoders on the imx6 platform:
+ * {{{gstreamer-imx}}} specific
+  * imxmp3audioenc
+ * Other GStreamer encoders
+  * vorbisenc
+  * mulawenc
+  * wavenc
+  * alawenc
+  * flacenc
+  * lamemp3enc
+
+There are many more. You can search for your specified one by running a similar search: {{{gst-inspect-1.0 | grep enc}}}.
+
+
+=== imxmp3audioenc ===
+This encoder can encode {{{audio/x-raw}}} S16LE to {{{audio/mpeg}}}.
+
+
+=== vorbisenc ===
+This encoder can encode {{{audio/x-raw}}} F32LE to {{{audio/x-vorbis}}}.
+
+
+=== mulawenc ===
+This encoder can encode {{{audio/x-raw}}} S16LE to {{{audio/x-mulaw}}}.
+
+
+=== wavenc ===
+This encoder can encode {{{audio/x-raw}}} S32LE, S24LE, S16LE, U8, F32LE, F64LE, {{{audio/x-alaw}}}, and {{{audio/x-mulaw}}} to {{{audio/x-wav}}}.
+
+
+=== alawenc ===
+This encoder can encode {{{audio/x-raw}}} S16LE to {{{audio/x-alaw}}}.
+
+
+=== flacenc ===
+This encoder can encode {{{audio/x-raw}}} S24LE, S24_32LE, S16LE, S8 to {{{audio/x-flac}}}.
+
+
+=== lamemp3enc ===
+This encoder can encode {{{audio/x-raw}}} S16LE to {{{audio/mpeg}}}.
+
+
+=== Examples ===
+This will take in an audio source from the mic and save it as an mp3.
+{{{#!bash
+gst-launch-1.0 alsasrc device="hw:0,0" ! audioconvert ! imxmp3audioenc ! filesink location=/tmp/mic.mp3
+}}}
+ * Note the audioconvert element is used to convert audio samplerate / bitwidth / format (via software algorithms) as needed between two elements.
+
+
+[=#audio-decoding]
+== Decoding ==
+Decoding is the term used to decode an encoded audio stream to a raw audio stream.
+
+A list of audio encoders on the imx6 platform:
+ * {{{gstreamer-imx}}} specific
+  * imxuniaudiodec
+ * Other GStreamer encoders
+  * vorbisdec
+  * ivorbisdec
+  * a52dec
+  * mulawdec
+  * alawdec
+  * flacdec
+
+
+=== imxuniaudiodec ===
+This decoder can decode {{{audio/mpeg}}} to {{{audio/x-raw}}} S32LE, S24LE, S16LE, S8.
+
+
+=== vorbisdec ===
+This decoder can decode {{{audio/x-vorbis}}} to {{{audio/x-raw}}} F32LE.
+
+
+=== ivorbisdec ===
+This decoder can decode {{{audio/x-vorbis}}} to {{{audio/x-raw}}} S16LE.
+
+
+=== a52dec ===
+This decoder can decode {{{audio/x-ac3}}}, {{{audio/ac3}}}, and {{{audio/x-private1-ac3}}} to {{{audio/x-raw}}} F32LE.
+
+
+=== mulawdec ===
+This decoder can decode {{{audio/x-mulaw}}} to {{{audio/x-raw}}} S16LE.
+
+
+=== alawdec ===
+This decoder can decode {{{audio/x-alaw}}} to {{{audio/x-raw}}} S16LE.
+
+
+=== flacdec ===
+This decoder can decode {{{audio/x-flac}}} to {{{audio/x-raw}}} S8, S16LE, S24_32LE, S32LE.
+
+
+=== Examples ===
+We can decode the MP3 audio saved previously and playback to the default audio device  with the following:
+{{{#!bash
+gst-launch-1.0 filesrc location=/tmp/mic.mp3 ! mpegaudioparse ! imxuniaudiodec ! alsasink
+}}}
+
+
+[=#audio-format]
+== Audio Format Conversion ==
+In specify the details of the audio format, you use a GStreamer {{{capsfilter}}} and to convert it from one format to another you use the GStreamer {{{audioconvert}}} element.
+{{{#!bash
+gst-launch-1.0 alsasrc device="hw:0,0" ! "audio/x-raw,rate=32000,channels=2,depth=16" ! \
+  audioconvert ! avimux ! filesink location=./audio.avi
+}}}
+
+
+[=#audio-loopback]
+== Audio loopback (Useful for testing audio input and output) ==
+A simple aduio loopback test can take audio input from an input device and output it to an output device.
+
+loopback audio from the first audio card:
+{{{#!bash
+gst-launch-1.0 alsasrc device="hw:0,0" ! alsasink device="hw:0,0"
+}}}
+
+To send audio from the sgtl5000 (analog input) to the imxhdmisoc (HDMI output):
+ * using device/function numbers (ie from {{{aplay -l}}}):
+{{{#!bash
+gst-launch-1.0 alsasrc device="hw:0,0" ! alsasink device="hw:2,0"
+}}}
+ * or, using names (ie from {{{aplay -L}}}):
+{{{#!bash
+gst-launch-1.0 alsasrc device="sysdefault:CARD=sgtl5000audio" ! \
+ alsasink device="sysdefault:CARD=imxhdmisoc"
+}}}
+
+To send audio from the tda1997x HDMI receiver (digital input) to the imxhdmisoc (digital output):
+{{{#!bash
+gst-launch-1.0 alsasrc device="sysdefault:CARD=tda1997xaudio" ! \
+  "audio/x-raw,rate=44100" ! alsasink device="sysdefault:CARD=imxhdmisoc"
+}}}
+  * Note here we need to specify the audio sample-rate as it can vary per input stream (and gstreamer does not validate the rate). This rate must match the source stream samplrate which can be found via sysfs {{{/sys/bus/i2c/drivers/tda1997x/2-0048/audmode}}}. If your output device requires a different sample-rate than the source input device, you need to perform a sample-rate conversion.