Changes between Version 1 and Version 2 of ventana/vpu


Ignore:
Timestamp:
10/05/2018 09:27:38 PM (6 years ago)
Author:
Tim Harvey
Comment:

add coda and gstreamer details

Legend:

Unmodified
Added
Removed
Modified

  • ventana/vpu

    v1 v2  
    11[[PageOutline]]
    22
     3See also:
     4 * [wiki:ventana/ipu IMX6 Image Processing Unit]
     5 * [wiki:ventana/gpu IMX6 3D Graphics Processing Unit]
     6
    37[=#vpu]
    4 == VPU - Video Processing Unit (Codec Support)
    5 The Video Processing Unit in the IMX6 is a CODA960 licensed IP block provided by Chips&Media supporting the following codecs:
     8= IMX6 VPU - Video Processing Unit (Codec Support)
     9The Video Processing Unit (VPU) in the IMX6 is a CODA960 licensed IP block provided by Chips&Media supporting the following codecs:
    610
    711Encode:
    … …  
    4448References:
    4549 * ​[https://community.freescale.com/servlet/JiveServlet/download/317735-256858/VPU_API_RM_L3.0.35_1.1.0.pdf Freescale IMX6VPUAPI] documents the VPU API - This is from the LINUXDOCS_BUNDLE downloadable from Freescale's IMX6 site - check there for the most recent version
     50
     51
     52[=#software]
     53= VPU Software Support
     54
     55'''This page refers to the Linux coda driver and the GStreamer elements that use it. Note that coda support for the IMX6 was added to Linux in the 3.16 kernel. This page applies to Linux 3.16 and beyond. If you are using a kernel earlier than that for the Ventana IMX6 based boards (such as the Gateworks 3.14 kernel) you will need to refer the Yocto GStreamer pages documenting the proprietary VPU drivers and software from Freescale:
     56 * [wiki:Yocto/gstreamer]
     57 * [wiki:Yocto/gstreamer/compositing]
     58 * [wiki:Yocto/gstreamer/latency]
     59 * [wiki:Yocto/gstreamer/multimedia]
     60 * [wiki:Yocto/gstreamer/streaming]
     61 * [wiki:Yocto/gstreamer/video]
     62'''
     63
     64[=#coda]
     65== Kernel Driver (coda)
     66'''Note that coda support for the IMX6 was added to Linux in the 3.16 kernel. This page applies to Linux 3.16 and beyond. If you are using a kernel earlier than that for the Ventana IMX6 based boards (such as the Gateworks 3.14 kernel) you will need to refer the Yocto GStreamer pages documenting the proprietary VPU drivers and software from Freescale:
     67 * [wiki:Yocto/gstreamer]
     68 * [wiki:Yocto/gstreamer/compositing]
     69 * [wiki:Yocto/gstreamer/latency]
     70 * [wiki:Yocto/gstreamer/multimedia]
     71 * [wiki:Yocto/gstreamer/streaming]
     72 * [wiki:Yocto/gstreamer/video]
     73'''
     74
     75The Linux [https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/media/platform/coda coda] media driver as of Linux 3.16 supports the CODA960 in the IMX6 and currently provides access to H264 encode/decode, MPEG4 encode/decode, and MPEG2 decode.
     76
     77Encoder and Decoder options are exposed through the v4l2 control IDs (CID) and can be listed with the {{{v4l2-ctl -L}}} and {{{v4l2-ctl -l}}} parameters on the devices exposed by the coda driver:
     78{{{#!bash
     79# v4l2-ctl -d6 -L
     80
     81User Controls
     82
     83                horizontal_flip 0x00980914 (bool)   : default=0 value=0
     84                  vertical_flip 0x00980915 (bool)   : default=0 value=0
     85
     86Codec Controls
     87
     88                 video_gop_size 0x009909cb (int)    : min=0 max=99 step=1 default=16 value=16
     89                  video_bitrate 0x009909cf (int)    : min=0 max=32767000 step=1000 default=0 value=0
     90    number_of_intra_refresh_mbs 0x009909d6 (int)    : min=0 max=8160 step=1 default=0 value=0
     91           sequence_header_mode 0x009909d8 (menu)   : min=0 max=1 default=1 value=1
     92                                1: Joined With 1st Frame
     93       maximum_bytes_in_a_slice 0x009909db (int)    : min=1 max=1073741823 step=1 default=500 value=500
     94       number_of_mbs_in_a_slice 0x009909dc (int)    : min=1 max=1073741823 step=1 default=1 value=1
     95      slice_partitioning_method 0x009909dd (menu)   : min=0 max=2 default=0 value=0
     96                                0: Single
     97                                1: Max Macroblocks
     98                                2: Max Bytes
     99                vbv_buffer_size 0x009909de (int)    : min=0 max=262144 step=1 default=0 value=0
     100  initial_delay_for_vbv_control 0x009909e1 (int)    : min=0 max=32767 step=1 default=0 value=0
     101          h264_i_frame_qp_value 0x00990a5e (int)    : min=0 max=51 step=1 default=25 value=25
     102          h264_p_frame_qp_value 0x00990a5f (int)    : min=0 max=51 step=1 default=25 value=25
     103          h264_maximum_qp_value 0x00990a62 (int)    : min=0 max=51 step=1 default=51 value=51
     104                     h264_level 0x00990a67 (menu)   : min=0 max=11 default=11 value=11
     105                                5: 2
     106                                8: 3
     107                                9: 3.1
     108                                10: 3.2
     109                                11: 4
     110  h264_loop_filter_alpha_offset 0x00990a68 (int)    : min=0 max=15 step=1 default=0 value=0
     111   h264_loop_filter_beta_offset 0x00990a69 (int)    : min=0 max=15 step=1 default=0 value=0
     112          h264_loop_filter_mode 0x00990a6a (menu)   : min=0 max=1 default=0 value=0
     113                                0: Enabled
     114                                1: Disabled
     115                   h264_profile 0x00990a6b (menu)   : min=0 max=0 default=0 value=0
     116                                0: Baseline
     117         mpeg4_i_frame_qp_value 0x00990a90 (int)    : min=1 max=31 step=1 default=2 value=2
     118         mpeg4_p_frame_qp_value 0x00990a91 (int)    : min=1 max=31 step=1 default=2 value=2
     119                    mpeg4_level 0x00990a95 (menu)   : min=0 max=7 default=7 value=7
     120                                7: 5
     121                  mpeg4_profile 0x00990a96 (menu)   : min=0 max=0 default=0 value=0
     122                                0: Simple
     123root@bionic-armhf:~# v4l2-ctl -d6 -l
     124
     125User Controls
     126
     127                horizontal_flip 0x00980914 (bool)   : default=0 value=0
     128                  vertical_flip 0x00980915 (bool)   : default=0 value=0
     129
     130Codec Controls
     131
     132                 video_gop_size 0x009909cb (int)    : min=0 max=99 step=1 default=16 value=16
     133                  video_bitrate 0x009909cf (int)    : min=0 max=32767000 step=1000 default=0 value=0
     134    number_of_intra_refresh_mbs 0x009909d6 (int)    : min=0 max=8160 step=1 default=0 value=0
     135           sequence_header_mode 0x009909d8 (menu)   : min=0 max=1 default=1 value=1
     136       maximum_bytes_in_a_slice 0x009909db (int)    : min=1 max=1073741823 step=1 default=500 value=500
     137       number_of_mbs_in_a_slice 0x009909dc (int)    : min=1 max=1073741823 step=1 default=1 value=1
     138      slice_partitioning_method 0x009909dd (menu)   : min=0 max=2 default=0 value=0
     139                vbv_buffer_size 0x009909de (int)    : min=0 max=262144 step=1 default=0 value=0
     140  initial_delay_for_vbv_control 0x009909e1 (int)    : min=0 max=32767 step=1 default=0 value=0
     141          h264_i_frame_qp_value 0x00990a5e (int)    : min=0 max=51 step=1 default=25 value=25
     142          h264_p_frame_qp_value 0x00990a5f (int)    : min=0 max=51 step=1 default=25 value=25
     143          h264_maximum_qp_value 0x00990a62 (int)    : min=0 max=51 step=1 default=51 value=51
     144                     h264_level 0x00990a67 (menu)   : min=0 max=11 default=11 value=11
     145  h264_loop_filter_alpha_offset 0x00990a68 (int)    : min=0 max=15 step=1 default=0 value=0
     146   h264_loop_filter_beta_offset 0x00990a69 (int)    : min=0 max=15 step=1 default=0 value=0
     147          h264_loop_filter_mode 0x00990a6a (menu)   : min=0 max=1 default=0 value=0
     148                   h264_profile 0x00990a6b (menu)   : min=0 max=0 default=0 value=0
     149         mpeg4_i_frame_qp_value 0x00990a90 (int)    : min=1 max=31 step=1 default=2 value=2
     150         mpeg4_p_frame_qp_value 0x00990a91 (int)    : min=1 max=31 step=1 default=2 value=2
     151                    mpeg4_level 0x00990a95 (menu)   : min=0 max=7 default=7 value=7
     152                  mpeg4_profile 0x00990a96 (menu)   : min=0 max=0 default=0 value=0
     153}}}
     154
     155
     156[=#firmware]
     157=== VPU Firmware
     158The VPU requires firmware that is loaded by the coda driver (see [https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/coda/coda-common.c#L2477 here]). You can obtain and extract the firmware from various Freescale / NXP sources included in the Freescale / NXP Yocto BSP.
     159
     160Example:
     161 * v3.1.1 from firmware-imx_7.6
     162{{{#!bash
     163wget https://www.nxp.com/lgfiles/NMG/MAD/YOCTO/firmware-imx-7.6.bin
     164/bin/sh firmware-imx-*
     165mkdir /lib/firmware/vpu
     166mv firmware-imx-7.6/firmware/vpu/vpu_fw_imx6*.bin /lib/firmware/vpu
     167}}}
     168
     169
     170[=#gstreamer]
     171== GStreamer
     172The popular GStreamer framework provides access to hardware decode via plugins.
     173
     174The [https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/sys/v4l2/gstv4l2videodec.c gstv4l2videodec plugin] (since GStreamer 1.1) can utilize the Linux coda driver to provide hardware decode for H264, MPEG2 and MPEG4.
     175
     176The [https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/sys/v4l2/gstv4l2videoenc.c gstv4l2videoenc plugin] (since GStreamer 1.14) can utilize the Linux coda driver to provide hardware encode for H264 and MPEG4.
     177
     178The GStreamer elements that are exposed are typed elements:
     179 * gstv4l2videodec:
     180{{{#!bash
     181# gst-inspect-1.0 | grep -P "v4l2.*dec"
     182video4linux2:  v4l2mpeg4dec: V4L2 MPEG4 Decoder
     183video4linux2:  v4l2mpeg2dec: V4L2 MPEG2 Decoder
     184video4linux2:  v4l2h264dec: V4L2 H264 Decoder
     185}}}
     186 * gstv4l2videoenc
     187{{{#!bash
     188# gst-inspect-1.0 | grep -P "v4l2.*enc"
     189video4linux2:  v4l2h264enc: V4L2 H.264 Encoder
     190video4linux2:  v4l2mpeg4enc: V4L2 MPEG4 Encoder
     191}}}
     192
     193Examples:
     194 * Encode and stream 1080p videotestsrc using UDP/RTP/H264:
     195{{{#!bash
     196gst-launch-1.0 videotestsrc ! 'video/x-raw, width=1920, height=1080, framerate=30/1, format=NV12, colorimetry=bt709' ! v4l2h264enc output-io-mode=4 ! rtph264pay ! udpsink host=$SERVER port=5000
     197}}}
     198 * Decode and display an H264 video stream:
     199{{{#!bash
     200gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,payload=96 ! rtph264depay ! v4l2h264dec ! kmssink
     201}}}