Changes between Version 7 and Version 8 of linux/media


Ignore:
Timestamp:
02/20/2019 10:53:01 PM (3 years ago)
Author:
Tim Harvey
Comment:

refactor page - split drivers and gstreamer sections

Legend:

Unmodified
Added
Removed
Modified
  • linux/media

    v7 v8  
    268268
    269269
    270 ==== setup script
     270[=#media-ctl-setup]
     271=== setup script
    271272Because the combinations of sensor, cpu, and board are plentiful we have created a way to create an appropriate pipeline configuration script that needs only to know what sensor you wish to use.
    272273
     
    290291
    291292
    292 ==== capture and streaming examples
    293 With the assumption that you have already configured your capture device and set it to the {{{DEVICE}}} env variable the following examples are useful examples for capture and streaming:
     293[=#mem2mem]
     294== IMX6 IPU MEM2MEM (Hardware CSC/scale/crop/rotate/flip)
     295A Linux V4L2 MEM2MEM imx-media driver exists that allows utilizing the IMX6 IPU Image Converter hardware blocks (IC) to perform hardware colorspace conversion (CSC), scaling, cropping, rotation, and flip operations.
     296
     297The GStreamer {{{video4linux2}}} plugin provides an element that uses this driver to expose these capabilities to GStreamer applications.
     298
     299Notes:
     300 - for GStreamer-1.14 the name of the element depends on the video device the driver registers with the kernel (ie v4l2video8convert if mem2mem driver registers /dev/video8)
     301 - for GStreamer master (in development) the name of the element is always 'v4l2videoconvert'
     302 - the {{{kmssink}}} examples below need a {{{can-scale=false}}} property to tell GStreamer not to scale via the KMS driver (as the IMX6 KMS driver does not support scaling)
     303 - ensure that the input format differs from the output format otherwise GStreamer will bypass the conversion completely; note that GStreamer doesn't understand flipping or rotation as part of the format. Gstreamer master (in development) adds a 'disable-passthrough' property to the v4l2videoconvert entity that can be set to force the conversion regardless of input and output format
     304 - when using imx entities (ie capture, encode/decode, mem2mem, display) you can specify 'output-io-mode=dmabuf-import' to share dmabuf pointers for a zero-copy pipeline however if using non imx entities (ie videotestsrc) you must omit these as you can not ensure the buffers share the alignment/stride necessary to share dmabuf pointers
     305
     306Examples:
     307 * Ensure mem2mem is in your kernel:
     308{{{#!bash
     309~# dmesg | grep mem2mem
     310[   18.356023] imx-media: Registered ipu_ic_pp mem2mem as /dev/video8
     311}}}
     312
     313See Gstreamer examples below
     314
     315
     316[=#coda]
     317== IMX6 coda encode / decode driver
     318The Linux CODA driver provides access to the IMX6 hardware encode/decode codecs and the GStreamer {{{video4linux2}}} plugin provides encode/decode elements that tap into this.
     319
     320Examples:
     321 * Ensure CODA is in your kernel:
     322{{{#!bash
     323~# dmesg | grep coda   
     324[   16.721698] coda 2040000.vpu: Direct firmware load for vpu_fw_imx6q.bin failed with error -2
     325[   16.721724] coda 2040000.vpu: Falling back to syfs fallback for: vpu_fw_imx6q.bin
     326[   18.381136] coda 2040000.vpu: Using fallback firmware vpu/vpu_fw_imx6q.bin
     327[   18.433648] coda 2040000.vpu: Firmware code revision: 570363
     328[   18.433683] coda 2040000.vpu: Initialized CODA960.
     329[   18.433706] coda 2040000.vpu: Firmware version: 3.1.1
     330[   18.442312] coda 2040000.vpu: codec registered as /dev/video[9-10]
     331}}}
     332
     333See Gstreamer examples below
     334
     335
     336[=#v4l2-ctl]
     337== Capture examples
     338With the assumption that you have already configured your capture device (see [#media-ctl-setup above) and set it to the {{{DEVICE}}} env variable the following examples are useful examples for capture and streaming:
    294339
    295340Examples:
     
    301346convert -size 720x480 -depth 16 uyvy:x.raw frame.png
    302347}}}
     348
     349
     350[=#gstreamer]
     351== GStreamer Capture, Dispaly, and Stream Examples
     352With the assumption that you have already configured your capture device (see [#media-ctl-setup above) and set it to the {{{DEVICE}}} env variable the following examples are useful examples for capture and streaming:
     353
     354
     355=== Capture / Display
    303356 * Display capture source:
    304357{{{#!bash
     
    308361gst-launch-1.0 v4l2src device=$DEVICE ! v4l2video10convert output-io-mode=dmabuf-import ! fbdevsink
    309362}}}
     363
     364=== Capture / Encode
     365
     366 * Ensure GStreamer encode elements exists:
     367{{{#!bash
     368~# gst-inspect-1.0 | grep -e "v4l2.*enc"
     369video4linux2:  v4l2h264enc: V4L2 H.264 Encoder
     370video4linux2:  v4l2mpeg4enc: V4L2 MPEG4 Encoder
     371}}}
     372
     373
    310374 * Encode to JPEG (software based encode) and stream via RTP/UDP:
    311375{{{#!bash
     
    325389
    326390
    327 [=#mem2mem]
    328 == IMX6 ipu_ic_pp mem2mem CSC/scale/crop/rotate/flip conversion
    329 A Linux V4L2 MEM2MEM imx-media driver exists that allows utilizing the IMX6 IPU Image Converter hardware blocks (IC) to perform hardware colorspace conversion (CSC), scaling, cropping, rotation, and flip operations.
    330 
     391=== Convert
    331392The GStreamer {{{video4linux2}}} plugin provides an element that uses this driver to expose these capabilities to GStreamer applications.
    332393
     
    397458     kmssink can-scale=false
    398459}}}
    399 
    400 
    401 [=#coda]
    402 == IMX6 coda encode
    403 The Linux CODA driver provides access to the IMX6 hardware encode/decode codecs and the GStreamer {{{video4linux2}}} plugin provides encode/decode elements that tap into this.
    404 
    405 Examples:
    406  * Ensure CODA is in your kernel:
    407 {{{#!bash
    408 ~# dmesg | grep coda   
    409 [   16.721698] coda 2040000.vpu: Direct firmware load for vpu_fw_imx6q.bin failed with error -2
    410 [   16.721724] coda 2040000.vpu: Falling back to syfs fallback for: vpu_fw_imx6q.bin
    411 [   18.381136] coda 2040000.vpu: Using fallback firmware vpu/vpu_fw_imx6q.bin
    412 [   18.433648] coda 2040000.vpu: Firmware code revision: 570363
    413 [   18.433683] coda 2040000.vpu: Initialized CODA960.
    414 [   18.433706] coda 2040000.vpu: Firmware version: 3.1.1
    415 [   18.442312] coda 2040000.vpu: codec registered as /dev/video[9-10]
    416 }}}
    417  * Ensure GStreamer encode elements exists:
    418 {{{#!bash
    419 ~# gst-inspect-1.0 | grep -e "v4l2.*enc"
    420 video4linux2:  v4l2h264enc: V4L2 H.264 Encoder
    421 video4linux2:  v4l2mpeg4enc: V4L2 MPEG4 Encoder
    422 }}}
    423460 * capture, scale, rotate, flip and encode using imx-media capture device mem2mem device and coda device (can use dmabufs for zero-copy)
    424461{{{#!bash