wiki:Yocto/Video_In

Version 3 (modified by Tim Harvey, 7 years ago) ( diff )

fix typo

  1. Goals
  2. Video Capture Drivers (on board)
  3. Video Input Sources (on board)
    1. Analog Video Input
      1. Analog Video Input Source
      2. Analog Video Input Standard
    2. HDMI Digital Video Input
      1. HDMI Input Examples
      2. Hardware Capabilities and video data base mode
        1. Select Between yuv422smp and yuv422bt656
      3. tda1997x Linux driver
      4. Supported resolutions and formats
      5. Tested resolutions and formats
      6. Extended display identification data (EDID)
      7. High-bandwidth Digital Content Protection (HDCP)

Goals

The intended goal of this article is to provide the user with enough information so they can capture video from an on-board video input device.

For information regarding Audio / Video breakout adapters for Ventana see:

Video Capture Drivers (on board)

There Video capture API in Linux today is the ​video4linux (v4l) API. Each video capture source is represented by a /dev/video<n> device with /dev/video0 being the first. For more information see the Gateworks v4l2 wiki page. Note that this is low level and Gateworks recommends using Gstreamer as noted below.

The i.MX6 Video capture framework drivers are currently only available in the Yocto BSP using the Freescale kernels. This driver framework has not made it into mainline linux yet and no support is available for the OpenWrt BSP.

Please use our GStreamer wiki page for more information about video input via GStreamer

Video Input Sources (on board)

Various Ventana boards have different video input possibilities:

  • GW51xx: CVBS/S-Video/Component (Analog) (/dev/video0)
  • GW52xx: CVBS/S-Video/Component (Analog) (/dev/video0)
  • GW53xx: CVBS/S-Video/Component (Analog) (/dev/video0)
  • GW540x: HDMI (Digital) (/dev/video0), CVBS/S-Video/Component (Analog) (/dev/video1)
  • GW551x: HDMI (Digital) (/dev/video0)
  • GW553x: CVBS/S-Video/Component (Analog) (/dev/video0)

Analog Video Input

The GW5100/GW5200/GW5300/GW5400 all feature Analog Video CVBS (color, video, blanking, and sync).

The Video input is provided via the ​ADV7180 Analog Video decoder chip.

Notes:

Analog Video Input Source

The ADV7180 has 3 inputs (AIN1, AIN2, AIN3) which are all provided on the Ventana board Video Input connector. These can be used in the following combinations:

INSEL[3:0] Video Format Analog Input
0000 (0) Composite CVBS input on AIN1
0011 (3) Composite CVBS input on AIN2
0100 (4) Composite CVBS input on AIN3
0110 (6) Y/C (S-Video) Y input on AIN1, C input on AIN2
1001 (9) YPrPb (Component) Y input on AIN1, Pb input on AIN2, Pr input on AIN3
  • to specify the input configuration set the INSEL[3:0] value above to Address 0x00[3:0] (bits 3-0)
  • the adv7180 i2c address is 0x20 on /dev/i2c-2
  • the default is 0 - Composite CVBS input on AIN1

The adv7180 v4l2 driver does not support setting the input configuration via software, but its simple to do so manually.

Example (referencing the table above):

  • Set adv7180 mode to 3 for analog input #2 (AIN2) using the INSEL variable:
    • Copy and paste the below code, setting the INSEL variable to whichever mode desired according to the table above. 
      INSEL=3
# set 0x00[3:0] (without disturbing the other bits)
r0=$(i2cget -f -y 2 0x20 0x00)  # get current value
r0=$((r0&0xf0)) # mask out lower nibble
r0=$((r0|$INSEL)) # set lower nibble
i2cset -f -y 2 0x20 0x00 $r0
  • Set mode6 for S-Video using AIN1/AIN2 using the INSEL variable: 
    INSEL=6
# set 0x00[3:0] (without disturbing the other bits)
r0=$(i2cget -f -y 2 0x20 0x00)  # get current value
r0=$((r0&0xf0)) # mask out lower nibble
r0=$((r0|$INSEL)) # set lower nibble
i2cset -f -y 2 0x20 0x00 $r0
  • set mode9 for component video on AIN1/AIN2/AIN3: 
    INSEL=9
# set 0x00[3:0] (without disturbing the other bits)
r0=$(i2cget -f -y 2 0x20 0x00)  # get current value
r0=$((r0&0xf0)) # mask out lower nibble
r0=$((r0|$INSEL)) # set lower nibble
i2cset -f -y 2 0x20 0x00 $r0

Note that the ADV7180 and IMX6 SoC can encode/capture only a single stream at a time, however you could round-robin the Composite AIN1/AIN2/AIN3 inputs if you want to essentially capture 3 streams at a time but you can not achieve a high framerate as you need to wait for the input to lock after each change.

Analog Video Input Standard

The ADV7180 supports NTSC, PAL, and SECAM standards. You can use the Video4Linux API to determine if an input is detected and what mode by using the 'v4l2-ctl --get-standard' command which reports one of the following:

  • No supported input detected - Video Standard = 0x00ffffff
  • NTSC - Video Standard = 0x000000ff
  • PAL - Video Standard = 0x000000ff

Example:

  • Show Analog Video status on GW51xx/GW52xx/GW53xx (/dev/video0): 
    v4l2-ctl -d /dev/video0 --get-standard
  • Show Analog Video status on GW54xx (/dev/video1) 
    v4l2-ctl -d /dev/video1 --get-standard

HDMI Digital Video Input

The Ventana HDMI Digital Video input is provided via the ​TDA19971 HDMI receiver chip.

HDMI Input Examples

For examples on how to use HDMI video input on the board via GStreamer Software, visit the GStreamer wiki page for more information

Hardware Capabilities and video data base mode

The TDA19971 has a 24bit video data bus that it can present data output in a variety of formats. The IMX6 Capture Sensor Interface (CSI) has a 20bit video data bus and can accept data in a variety of formats. There are however, only two formats that overlap and are thus compatible between the two:

  • yuv422smp: YUV422 semi-planar - 16bit data bus clocked at 1x (1 clock per pixel) capturing 8bit per component YUV422 in a semi-planar arrangement using external sync signals
  • yuv422bt656: YUV422 CCIR656 - 8bit data bus clocked at 1/2x (2 clocks per pixel) capturing 8bit per component YUV422 via BT656 protocol with embedded sync signals

While the TDA19971 supports 8bit per component RGB444 and YUV444 formats, the IMX6 can not consume this data because it doesn't the required 24bit data bus available.

The TDA19971 when used with the IMX6 will always output YUV data (doing a color-conversion on the TDA19971 for HDMI input sources that are RGB).

There are trade-offs depending on the video data bus mode chosen between the TDA19971 and the IMX6 above:

  • yuv422smp: can capture up to 1080p@60Hz, but lacks interlaced video support (which is assumed to be a driver shortcomming at this time) and outputs UYVY which must use an IPU/GPU/PXP transform if needing to pass it to the encoder
  • yuv422bt656: can capture up to 1080p@30Hz (as each pixel takes 2 clock cycles) but supports interlaced video and outputs YU12 which needs no transform if needing to pass it to the encoder

To determine the video bus mode the driver is using look at the device-tree: 

echo $(cat /proc/device-tree/soc/aips-bus@02100000/i2c@021a8000/tda1977x@48/vidout_fmt)
  • a value of 422_smp indicates yuv422smp
  • a value of 422_ccir indicates yuv422bt656

Select Between yuv422smp and yuv422bt656

Currently, products with HDMI video input are defaulted to yuv422smp as this allows capturing at 1080p60, except for the GW551x-A revision board (Please see errata HW9). However, this can be changed through a bootloader environmental variable hdmiinfmt.

In order to default to yuv422smp every time, the following steps would occur:

  1. Breakout into the bootloader
  2. Type the following: 
    Ventana > setenv hdmiinfmt yuv422smp
Ventana > saveenv
  3. Boot

In order to default to yuv422bt656 every time, the following steps would occur:

  1. Breakout into the bootloader
  2. Type the following: 
    Ventana > setenv hdmiinfmt yuv422bt656
Ventana > saveenv
  3. Boot

tda1997x Linux driver

The tda1997x multi-function driver core monitors the chip for HDMI input signals and provides information about the input source via sysfs: 

# ls /sys/bus/i2c/devices/2-0048/
audmode     edid        modalias    product     uevent
colorspace  edid-ascii  name        state       vendor
driver      info        power       subsystem   vidmode

The following sysfs files are useful in determining info about the HDMI input source:

  • audmode - audio input
  • vidmode - video output mode (<width>x<height>@<framerate>Hz<i|p>)
  • colorspace - video input colorspace
  • state - initialized|configured|locked|unlocked (will be 'locked' when locked on a valid HDMI input source)
  • vendor - input device info
  • product - input device info
  • edid - read/write binary EDID presented to HDMI sources

To determine the presence, and format of a detected HDMI input signal for example you could: 

if [ "$(cat /sys/bus/i2c/devices/2-0048/state)" = "locked" ]; then
  cat /sys/bus/i2c/devices/2-0048/{vidmode,colorspace,audmode}
else
  echo "No signal"
fi

Supported resolutions and formats

Resolutions ranging from 1080p to 480i at various frequencies are supported:

  • see ​driver for details
  • regardless of what modes the tda1997x driver supports, the EDID is supposed to be honored by HDMI output sources when deciding upon formats.

The following video input colorspaces and colorimetry formats are supported:

  • RGB444 (required by HDMI spec)
  • YUV444 ITU601/ITU709
  • YUV422 ITU601/ITU709
  • deep color (30/36bit) (untested)
  • XVYCC (untested)

The following audio input configurations are supported:

  • 44.1/48/88.2/96/172.4/192kHz sample rates
  • 8 channels (although only 2 channels can be output to the IMX6)
  • HBR streams (compatible with DTS-HD master audio and Dolby TrueHD) (untested)

Note that while RGB444/YUV444 colorspace is supported on the HDMI input, the TDA19971 will apply a colorspace conversion (using the specified colorimetry) or downsampling to convert to YUV422 to output to the IMX6 CSI. This is because RGB444/YUV444 require a 24bit data bus that the IMX6 CSI does not support.

Note that while 8 channels of High Bit Rate (HBR) or L-PCM audio is supported on the HDMI input, the I2S (Inter-IC Sound) bus between the TDA19971 and the IMX6 supports one data signal allowing for 2-ch Linear PCM (IEC60958), Encoded Audio Bitstream (IEC61937), or High Bit Rate (HBR) Encoded Audio (IEC61937).

The deep color and xvYCC colorspace options added to HDMI v1.3 are supported by the TDA19971 and the driver but are currently untested. Deep color increases the available bit depth for each color component and xvYCC makes the overall color gamut larger. Neither of these modes should be useful when the TDA19971 is connected to an IMX6 CSI bus in YUV422 or YUV422-BT656 mode yet currently these are marked as supported in the default EDID.

Tested resolutions and formats

While we have not had the equipment available to test all the combinations of the above formats, the following combinations of input formats have been tested:

Resolution / Framerate AR Colorspace Audio Device Tested Notes
640x480p@60Hz 4:3 RGB 2-ch 48/44.1kHz Roku 2710X (Roku1)
1280x720p@60Hz 16:9 RGB 2-ch 48/44.1kHz Roku 2710X (Roku1)
1280x720p@60Hz 16:9 RGB 2-ch 48/44.1kHz Roku 2710X (Roku1)
1920x1080p@60Hz 16:9 RGB 2-ch 48/44.1kHz Roku 2710X (Roku1) yuv422smp1
1280x720p@60Hz 16:9 RGB 2-ch 48kHz Dino-Lite Microscope
640x480p@60Hz 4:3 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD
720x576p@50Hz PAL-2 5:4 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD
720x480p@59Hz NTSC-3 3:2 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD
720x480p@60Hz NTSC-4 3:2 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD
1280x720p@24Hz 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD
1280x720p@25Hz 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD
1280x720p@29Hz 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD
1280x720p@30Hz 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD
1280x720p@50Hz 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD
1280X720p@59Hz NTSC-5 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD
1280x720p@60Hz 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD
1920x1080p@24Hz PAL-6 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD
1920x1080p@25Hz 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD
1920x1080p@30Hz NTSC-8 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD
1920x1080i@50Hz PAL-5 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD yuv422bt6562
1920x1080p@50Hz PAL-9 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD yuv422smp1
1920x1080i@59Hz 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD yuv422bt6562
1920x1080i@60Hz NTSC-7 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD yuv422bt6562
1920x1080p@60Hz NTSC-9 16:9 YUV444 ITU601 2-ch 48kHz Hauppauge MediaMVP-HD yuv422smp1
  1. 1080p@60Hz requires yuv422smp video bus format
  2. interlaced video requires yuv422bt656 video bus format

Notes:

  • the above was tested using Yocto v1.7 with the gateworks_3.10.17_1.0.0-ga-g4d177f6 kernel

Extended display identification data (EDID)

Extended display identification data or EDID is a standard that defines a compact binary file format describing a digital display's capabilities and supported graphics mode. Because an HDMI input is an HDMI sink it can be thought of as a display and must support EDID so that an HDMI source can choose a supported display format and resolution.

The EDID is electrically accessed via the DDC channel over an HDMI cable. The DDC channels is an i2c channel and the EDID of an HDMI sink is powered by the HDMI source through a +5V signal on the HDMI connector capable of providing up to 50mA. This allows an HDMI source to query the EDID of an HDMI sink even if the HDMI sink is powered off. Typically the EDID is stored in a i2c based EEPROM device powered off this +5V signal from the HDMI source.

The Ventana boards using the tda1997x HDMI receiver on the other hand, provide the EDID from the tda1997x itself. The EDID is programmed into the tda1997x's volatile memory from the tda1997x-core.c driver. If necessary, this driver allows the edid to be get/set by accessing the sysfs property /sys/bus/i2c/drivers/tda1997x/2-0048/edid as an alternate method to modifying the source code directly. For example:

  • Get EDID from HDMI Receiver: 
    hexdump -C /sys/bus/i2c/devices/2-0048/edid
00000000  00 ff ff ff ff ff ff 00  1e e3 00 50 01 00 00 00  |...........P....|
00000010  01 17 01 03 a2 00 00 00  02 ee 95 a3 54 4c 99 26  |............TL.&|
00000020  0f 50 54 20 00 00 01 01  01 01 01 01 01 01 01 01  |.PT ............|
00000030  01 01 01 01 01 01 02 3a  80 18 71 38 2d 40 58 2c  |.......:..q8-@X,|
00000040  45 00 c4 8e 21 00 00 1e  00 00 00 fc 00 47 57 20  |E...!........GW |
00000050  56 65 6e 74 61 6e 61 0a  20 20 00 00 00 fd 00 1e  |Ventana.  ......|
00000060  3c 0f 44 09 00 0a 20 20  20 20 20 20 00 00 00 fe  |<.D...      ....|
00000070  00 31 30 38 30 70 44 43  78 76 43 33 44 0a 01 89  |.1080pDCxvC3D...|
00000080  02 03 21 f0 4c 22 20 21  04 13 03 12 05 14 07 16  |..!.L" !........|
00000090  01 23 09 07 07 83 01 00  00 67 03 0c 00 00 00 b8  |.#.......g......|
000000a0  2d 01 1d 80 18 71 38 2d  40 58 2c 45 00 80 38 74  |-....q8-@X,E..8t|
000000b0  00 00 3f 01 1d 00 72 51  d0 1e 20 6e 50 55 00 00  |..?...rQ.. nPU..|
000000c0  d0 52 00 00 39 a0 0f 20  00 31 58 1c 20 28 80 14  |.R..9.. .1X. (..|
000000d0  00 20 58 32 00 00 3f d7  09 80 a0 20 e0 2d 10 08  |. X2..?.... .-..|
000000e0  60 22 01 80 e0 21 00 00  39 00 00 00 00 00 00 00  |`"...!..9.......|
000000f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 a7  |................|
00000100
  • Set updated EDID to HDMI Receiver: 
    cat edid  > /sys/bus/i2c/devices/2-0048/edid
[  377.266459] TDA1997x-core: New EDID being loaded

The EDID specification states that the timings should be preferred in the following order of priority:

  • preferred detailed mode
  • other detailed modes from base block
  • detailed modes from extension blocks
  • CVT 3-byte code modes
  • standard timing codes
  • established timing codes
  • modes inferred from GTF or CVT range information

For Ventana's default EDID the above should translate to the following priority:

  • 1920x1080p@60Hz (Preferred Detailed mode Block 1)
  • 1920x1080p@30Hz (extension block Detailed Timing Block1)
  • 1280x720p@60Hz (extension block Detailed Timing Block2)
  • 800x600@60Hz (extension block Detailed Timing Block3)
  • 640x480@59Hz (extension block Detailed Timing Block4)
  • 1920x1080p@29Hz (SVD1)
  • 1920x1080p@24Hz (SVD2)
  • 1920x1080p@25Hz (SVD3)
  • 1280x720p@59Hz (SVD4)
  • 1280x720p@50Hz (SVD5)
  • 720x480p@59Hz (SVD6)
  • 720x576p@50Hz (SVD7)
  • 1920x1080i@59Hz (SVD8)
  • 1920x1080i@50Hz (SVD9)
  • 720x480i@59Hz (SVD10)
  • 720x576i@50Hz (SVD11)
  • 640x480p@60Hz (SVD12)

Why would you want to edit the HDMI Receiver EDID?

  • use your product information (vendor ID, model string)
  • limit or extend the various modes advertised as supported for example:
    • limit to specific resolutions
    • limit to sRGB colorspace only (by clearing bits 4 and 4 in byte 3 of the CEA extension marking support for YUV444/YUV422)
    • remove deep-color support (by clearing bits 6-3 in VSD byte 6)

You can use a variety of tools to view/edit the EDID information. Some examples we have used:

References:

High-bandwidth Digital Content Protection (HDCP)

HDMI can use High-bandwidth Digital Content Protection (HDCP) to encrypt the signal if required by the source device as a form of digital copy protection. The system is meant to stop HDCP-encrypted content from being played on unauthorized devices or devices which have been modified to copy HDCP content. Before sending data, a transmitting device checks that the receiver is authorized to receive it and if so, the transmitter encrypts the data to prevent eavesdropping as it flows to the receiver.

In order to make a device that can playback material protected by HDCP the manufacturer must obtain a license from Intel subsidiary Digital Content Protection LLC, pay an annual fee, and submit to various conditions for example:

  • it must 'frustrate attempts to defeat the content protection requirements"
  • it must not transmit high definition protected video to non-HDCP receivers
  • DVD-audio material can only be played at CD-audio quality

The HDMI input receiver used on the GW5510/GW5400 Ventana boards is based upon the NXP TDA19971 chip. The standard version that Gateworks loads does not contain HDCP support however a pin-compatible version of the chip is available (TDA19971A) and if you require HDCP support you can contact sales@… for more information. I have received pricing back from my supplier.The pricing for the HDCP input chip is the same price, however, because it is special, the order is NC/NR (non-cancellable and non-refundable). There is a 1,000 piece minimum order.

Note that using the HDCP enabled TDA19971 would mean that your end device must adhere to the stipulations required by HDCP. The HDCP enabled version includes a HDCP 1.4 engine with pre-programmed keys stored into an internal non-volatile memory to provide a highly secured solution which simplifies the manufacturing process. The HDCP enabled version also features repeater capability (requires software driver).

For HDMI output, the Ventana boards use the NXP i.MX6's HDMI output port. The standard i.MX6 CPU loaded on Gateworks boards does not support HDCP however special parts are available from NXP to qualified vendors with HDCP revision 1.4 capability. The HDCP transmitter in the i.MX6 implements the three layers of the HDCP cipher, including LFSR and other functions required to generate the encryption key bytes that are then XORed with the data.

You may encounter a blank or 'all black' video signal due to HDCP encrypted content in the following scenarios for example:

  • A Roku device user interface works fine, but as soon as you try to play encrypted content such as that from Google Play, Netflix, Amazon Video, the display goes black
  • A Google Chromecast apparently does not even display the user interface

Devices called HDCP strippers can remove the HDCP information from the video signal and allow the video to be playable on non-HDCP-compliant displays.

References:

Note: See TracWiki for help on using the wiki.