Changes between Version 21 and Version 22 of gstreamer


Ignore:
Timestamp:
12/29/2020 12:51:47 AM (3 years ago)
Author:
Tim Harvey
Comment:

remove MEM2MEM kernel details as they have changed and update examples for gstreamer 1.16

Legend:

Unmodified
Added
Removed
Modified

  • gstreamer

    v21 v22  
    469469 * show all video4linux2 devices:
    470470{{{#!bash
    471 root@bionic-armhf:~# v4l2-ctl --list-devices
    472471CODA960 (platform:coda):
     472        /dev/video9
     473        /dev/video10
     474        /dev/video11
     475        /dev/video12
     476
     477imx-media (platform:imx-media):
     478        /dev/media0
     479
     480imx-media-csc-scaler (platform:imx-media-csc-scaler):
    473481        /dev/video8
    474         /dev/video9
    475 
    476 imx-media-mem2mem (platform:imx-media-mem2mem):
    477         /dev/video10
    478 
    479 imx-media-capture (platform:ipu1_csi0):
     482
     483imx-capture (platform:ipu1_csi0):
     484        /dev/video0
     485
     486imx-capture (platform:ipu1_csi1):
     487        /dev/video3
     488
     489imx-capture (platform:ipu1_ic_prpenc):
     490        /dev/video1
     491
     492imx-capture (platform:ipu1_ic_prpvf):
     493        /dev/video2
     494
     495imx-capture (platform:ipu2_csi0):
    480496        /dev/video4
    481497
    482 imx-media-capture (platform:ipu1_csi1):
     498imx-capture (platform:ipu2_csi1):
     499        /dev/video7
     500
     501imx-capture (platform:ipu2_ic_prpenc):
    483502        /dev/video5
    484503
    485 imx-media-capture (platform:ipu1_ic_prpenc):
    486         /dev/video0
    487 
    488 imx-media-capture (platform:ipu1_ic_prpvf):
    489         /dev/video1
    490 
    491 imx-media-capture (platform:ipu2_csi0):
     504imx-capture (platform:ipu2_ic_prpvf):
    492505        /dev/video6
    493 
    494 imx-media-capture (platform:ipu2_csi1):
    495         /dev/video7
    496 
    497 imx-media-capture (platform:ipu2_ic_prpenc):
    498         /dev/video2
    499 
    500 imx-media-capture (platform:ipu2_ic_prpvf):
    501         /dev/video3
    502 
    503 }}}
     506}}}
     507  - note the output may vary depending on your kernel version, board, and processor
    504508 * display all details about /dev/video1:
    505509{{{#!bash
    … …  
    523527[=#colorspace]
    524528[=#scaling]
    525 [=#mem2mem]
    526 == Colorspace Converting and/or Video Scaling (via IMX6 IPU mem2mem driver)
     529== Colorspace Converting and/or Video Scaling (via IMX6 imx-media-csc-scaler driver)
    527530Often times, a colorspace conversion or scaling is required in order to link GStreamer elements together. This is often due to the fact that not all elements can every format available.
    528531
    … …  
    532535
    533536Notes:
    534  - For GStreamer-1.14 the name of the element depends on the video device the driver registers with the kernel (ie v4l2video8convert if mem2mem driver registers /dev/video8) but for GStreamer-1.16 and beyond the name of the element is stable and always 'v4l2videoconvert'. For the examples on this page we use the GST_CONVERT env variable that is set from the output of the media-ctl-setup script to avoid confusion.
     537 - For GStreamer-1.14 the name of the element depends on the video device the driver registers with the kernel (ie v4l2video8convert if the driver registers /dev/video8) but for GStreamer-1.16 and beyond the name of the element is stable and always 'v4l2convert'. For the examples on this page we use the GST_CONVERT env variable that is set from the output of the media-ctl-setup script to avoid confusion.
    535538 - the {{{kmssink}}} examples below need a {{{can-scale=false}}} property to tell GStreamer not to scale via the KMS driver (as the IMX6 KMS driver does not support scaling)
    536  - ensure that the input format differs from the output format otherwise GStreamer will bypass the conversion completely; note that GStreamer doesn't understand flipping or rotation as part of the format. Gstreamer master (in development) adds a 'disable-passthrough' property to the v4l2videoconvert entity that can be set to force the conversion regardless of input and output format
    537  - when using imx entities (ie capture, encode/decode, mem2mem, display) you can specify 'output-io-mode=dmabuf-import' to share dmabuf pointers for a zero-copy pipeline however if using non imx entities (ie videotestsrc) you must omit these as you can not ensure the buffers share the alignment/stride necessary to share dmabuf pointers
    538 
    539 Examples:
    540  * Ensure mem2mem is in your kernel:
    541 {{{#!bash
    542 ~# dmesg | grep mem2mem
    543 [   18.356023] imx-media: Registered ipu_ic_pp mem2mem as /dev/video8
    544 }}}
     539 - ensure that the input format differs from the output format otherwise GStreamer will bypass the conversion completely; note that GStreamer doesn't understand flipping or rotation as part of the format. Gstreamer master (in development) adds a 'disable-passthrough' property to the v4l2convert entity that can be set to force the conversion regardless of input and output format
     540 - when using imx entities (ie capture, encode/decode, csc/scaler, display) you can specify 'output-io-mode=dmabuf-import' to share dmabuf pointers for a zero-copy pipeline however if using non imx entities (ie videotestsrc) you must omit these as you can not ensure the buffers share the alignment/stride necessary to share dmabuf pointers
     541
     542Examples:
    545543 * Ensure GStreamer element exists:
    546544{{{#!bash
    … …  
    552550GST_CONVERT=$(gst-inspect-1.0 | grep -e "v4l2.*convert*" | sed -e 's/.*:\s*\(v4l2.*convert\):.*/\1/')
    553551}}}
     552  - for modern Gstreamer 1.16 this will be 'v4l2convert' but for earlier Gstreamer 1.14 it will vary depending on the video device number which is why we assign it to a variable here for the examples
    554553 * scale/rotate/flip using {{{videotestsrc}}} (can not use dmabufs for this as it is a non-imx entity)
    555554{{{#!bash
    … …  
    599598     kmssink can-scale=false
    600599}}}
    601  * capture, scale, rotate, flip and encode using imx-media capture device mem2mem device and coda device (can use dmabufs for zero-copy)
     600 * capture, scale, rotate, flip and encode using imx-media capture device, csc/scaler device and coda device (can use dmabufs for zero-copy)
    602601{{{#!bash
    603602# encode
    … …  
    628627 * note the above examples also use v4l2convert to convert to bt709 coloriemtry required by the encoder elements
    629628
    630 For anything not able to be handled by the video4linux2 element and any mem2mem drivers there are software based elements that can convert pixel colorspace and/or scale however note these are extremely CPU intensive:
     629For anything not able to be handled by the v4l2convert element and csc/scaler driver there are software based elements that can convert pixel colorspace and/or scale however note these are extremely CPU intensive:
    631630 * autovideoconvert
    632631 * videoconvert
    … …  
    683682 * The CODA driver requires CMA memory for buffers when it is used. Make sure to provide the kernel with enough CMA memory with the kernel command-line (ie 'cma=64M' for 64MB which should be enough)
    684683 * Encoder and Decoder options are exposed through the v4l2 control IDs (CID) and can be listed with the v4l2-ctl -L and v4l2-ctl -l parameters on the devices exposed by the coda driver
    685  * Often you will need to use the mem2mem v4l2convert driver for format conversions (colorspace, colorimetry) to reconcile the format of the source device and the encoder element - notice the $GST_CONVERT env variable below.
     684 * Often you will need to use the csc/scaler driver (v4l2convert gstreamer element) for format conversions (colorspace, colorimetry) to reconcile the format of the source device and the encoder element - notice the $GST_CONVERT env variable below.
    686685 * V4L2 JPEG Encoder support was added to the CODA driver in Linux 5.6 and has been backported to the Gateworks 5.4 kernel
    687686 * V4l2jpegenc JPEG Encoder support requires GStreamer-1.16+