Changes between Version 8 and Version 9 of gstreamer


Ignore:
Timestamp:
03/02/2020 11:05:10 PM (5 years ago)
Author:
Tim Harvey
Comment:

added details about hardware JPEG encode/decode support

Legend:

Unmodified
Added
Removed
Modified

  • gstreamer

    v8 v9  
    635635Encoding video means taking the raw captured pixels and converting them to a standard compressed format.
    636636
    637 The Linux CODA driver provides access to the IMX6 hardware encode/decode codecs and the GStreamer {{{video4linux2}}} element provides encode/decode elements that tap into this.
     637The Linux CODA driver provides access to the IMX6 hardware encode/decode codecs and the GStreamer {{{video4linux2}}} plugin provides various encode/decode elements that tap into this.
    638638
    639639Notes:
    … …  
    641641{{{#!bash
    642642~# dmesg | grep coda   
    643 [   16.721698] coda 2040000.vpu: Direct firmware load for vpu_fw_imx6q.bin failed with error -2
    644 [   16.721724] coda 2040000.vpu: Falling back to syfs fallback for: vpu_fw_imx6q.bin
    645 [   18.381136] coda 2040000.vpu: Using fallback firmware vpu/vpu_fw_imx6q.bin
    646 [   18.433648] coda 2040000.vpu: Firmware code revision: 570363
    647 [   18.433683] coda 2040000.vpu: Initialized CODA960.
    648 [   18.433706] coda 2040000.vpu: Firmware version: 3.1.1
    649 [   18.442312] coda 2040000.vpu: codec registered as /dev/video[9-10]
    650 ~# cat /sys/class/video4linux/video9/name
    651 coda-encoder
    652 ~# cat /sys/class/video4linux/video10/name
    653 coda-decoder
     643[   10.798192] coda 2040000.vpu: Firmware code revision: 570363
     644[   10.804115] coda 2040000.vpu: Initialized CODA960.
     645[   10.808929] coda 2040000.vpu: Firmware version: 3.1.1
     646[   10.817951] coda 2040000.vpu: encoder registered as video8
     647[   10.825635] coda 2040000.vpu: decoder registered as video9
     648[   10.832117] coda 2040000.vpu: encoder registered as video10
     649[   10.839520] coda 2040000.vpu: decoder registered as video11
     650...
     651~# for i in $(ls -d /sys/class/video4linux/video*); do echo $i:$(cat $i/name); do
     652ne | grep encoder
     653/sys/class/video4linux/video10:coda-encoder
     654/sys/class/video4linux/video8:coda-jpeg-encoder
    654655}}}
    655656 * Ensure GStreamer encode elements exists:
    656657{{{#!bash
    657658~# gst-inspect-1.0 | grep -e "v4l2.*enc"
     659video4linux2:  v4l2jpegenc: V4L2 JPEG Encoder
    658660video4linux2:  v4l2h264enc: V4L2 H.264 Encoder
    659661video4linux2:  v4l2mpeg4enc: V4L2 MPEG4 Encoder
    … …  
    663665 * Encoder and Decoder options are exposed through the v4l2 control IDs (CID) and can be listed with the v4l2-ctl -L and v4l2-ctl -l parameters on the devices exposed by the coda driver
    664666 * Often you will need to use the mem2mem v4l2convert driver for format conversions (colorspace, colorimetry) to reconcile the format of the source device and the encoder element - notice the $GST_CONVERT env variable below.
     667 * V4L2 JPEG Encoder support was added to the CODA driver in Linux 5.6 and has been backported to the Gateworks 5.4 kernel
     668 * V4l2jpegenc JPEG Encoder support requires GStreamer-1.16+
    665669
    666670Examples:
    … …  
    681685# client on $SERVER:$PORT could be viewing via 'gst-launch-1.0 udpsrc port=$PORT ! application/x-rtp,payload=96 ! rtpjpegdepay ! jpegdec ! autovideosink'
    682686}}}
     687 * Encode to JPEG (hardware based encode) and stream via RTP/UDP:
     688{{{#!bash
     689# stream JPEG/RTP/UDP
     690gst-launch-1.0 v4l2src device=$DEVICE ! v4l2jpegenc ! rtpjpegpay ! udpsink host=$SERVER port=$PORT
     691# client on $SERVER:$PORT could be viewing via 'gst-launch-1.0 udpsrc port=$PORT ! application/x-rtp,payload=96 ! rtpjpegdepay ! jpegdec ! autovideosink'
     692}}}
     693  - requires GStreamer 1.16+ and kernel supporting CODA JPEG encode
    683694
    684695See also:
    … …  
    691702Decoding video means taking compressed formats such as MPEG and H264 and decoding them into frames comprised of raw pixels (typically so they can be displayed on a video display output device).
    692703
    693 The Linux CODA driver provides access to the IMX6 hardware encode/decode codecs and the GStreamer {{{video4linux2}}} element provides encode/decode elements that tap into this.
     704The Linux CODA driver provides access to the IMX6 hardware encode/decode codecs and the GStreamer {{{video4linux2}}} plugin provides encode/decode elements that tap into this.
    694705
    695706Notes:
    696707 * Ensure the CODA driver is in your kernel (CONFIG_VIDEO_CODA added in Linux 3.16) and that firmware was loaded:
    697708{{{#!bash
    698 ~# dmesg | grep coda   
    699 [   16.721698] coda 2040000.vpu: Direct firmware load for vpu_fw_imx6q.bin failed with error -2
    700 [   16.721724] coda 2040000.vpu: Falling back to syfs fallback for: vpu_fw_imx6q.bin
    701 [   18.381136] coda 2040000.vpu: Using fallback firmware vpu/vpu_fw_imx6q.bin
    702 [   18.433648] coda 2040000.vpu: Firmware code revision: 570363
    703 [   18.433683] coda 2040000.vpu: Initialized CODA960.
    704 [   18.433706] coda 2040000.vpu: Firmware version: 3.1.1
    705 [   18.442312] coda 2040000.vpu: codec registered as /dev/video[9-10]
    706 ~# cat /sys/class/video4linux/video9/name
    707 coda-encoder
    708 ~# cat /sys/class/video4linux/video10/name
    709 coda-decoder
     709~# dmesg | grep coda
     710[   10.798192] coda 2040000.vpu: Firmware code revision: 570363
     711[   10.804115] coda 2040000.vpu: Initialized CODA960.
     712[   10.808929] coda 2040000.vpu: Firmware version: 3.1.1
     713[   10.817951] coda 2040000.vpu: encoder registered as video8
     714[   10.825635] coda 2040000.vpu: decoder registered as video9
     715[   10.832117] coda 2040000.vpu: encoder registered as video10
     716[   10.839520] coda 2040000.vpu: decoder registered as video11
     717~# for i in $(ls -d /sys/class/video4linux/video*); do echo $i:$(cat $i/name); do
     718ne | grep decoder
     719/sys/class/video4linux/video11:coda-decoder
     720/sys/class/video4linux/video9:coda-jpeg-decoder
    710721}}}
    711722 * Ensure GStreamer decode elements exists:
    712723{{{#!bash
    713724~# gst-inspect-1.0 | grep -e "v4l2.*dec"
     725video4linux2:  v4l2jpegdec: V4L2 JPEG Decoder
    714726video4linux2:  v4l2mpeg4dec: V4L2 MPEG4 Decoder
    715727video4linux2:  v4l2mpeg2dec: V4L2 MPEG2 Decoder
    … …  
    718730 * The CODA960 decoder outputs NV12/I420/YV12 YUV pixel formats
    719731 * The CODA driver requires CMA memory for buffers when it is used. Make sure to provide the kernel with enough CMA memory with the kernel command-line (ie 'cma=64M' for 64MB which should be enough)
    720  
     732 * V4L2 JPEG Decoder support has not yet been added to the CODA driver as of Linux 5.6 - patches have been posted and are in the Gateworks 5.4 kernel
     733
    721734Note that the following examples assume you are using raw video encoded files, not container formats used for mimxed multimedia types (audio + video) such as ogg, avi, or mov (Quicktime). For information on de-muxing container formats see [#multimedia]
    722735
    … …  
    729742{{{#!bash
    730743gst-launch-1.0 filesrc location=file.mp4v ! mpeg4videoparse !  v4l2mpeg4dec ! kmssink
     744}}}
     745 * Streaming JPEG video might look like this:
     746{{{#!bash
     747# transmit JPEG/RTP/UDP on on board:
     748gst-launch-1.0 v4l2src device=$DEVICE ! v4l2jpegenc ! rtpjpegpay ! udpsink host=$SERVER port=$PORT
     749# recieve JPEG/RTP/UDP on another board:
     750gst-launch-1.0 udpsrc port=$PORT ! application/x-rtp,payload=96 ! rtpjpegdepay !  v4l2jpegdec ! autovideosink
    731751}}}
    732752