Changes between Initial Version and Version 1 of Yocto/Video_Out


Ignore:
Timestamp:
10/22/2017 05:28:45 AM (23 months ago)
Author:
trac
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Yocto/Video_Out

    v1 v1  
     1[[PageOutline]]
     2
     3= Goals =
     4The intended goal of this article is to provide the user with enough information so they can display an image onto a screen of their choosing.
     5
     6For information regarding Audio / Video breakout adapters for Ventana see:
     7 * [wiki:ventana/breakoutadapters Audio / Video breakout Adapters]
     8
     9
     10[=#drivers]
     11= Video Output Drivers =
     12There are two different display driver API's in Linux today:
     13 * [https://www.kernel.org/doc/Documentation/fb/framebuffer.txt framebuffer]
     14  * Very simple
     15  * Low level
     16  * [https://www.kernel.org/doc/Documentation/fb/framebuffer.txt See this link for much more detail]
     17
     18 * [https://www.kernel.org/doc/Documentation/video4linux/v4l2-framework.txt video4linux (v4l)]
     19  * More abstract
     20  * Mainly used for video and other graphical applications
     21  * [https://www.kernel.org/doc/Documentation/video4linux/v4l2-framework.txt See this link for much more detail]
     22
     23The i.MX6 Video output framework drivers are currently only available in the Yocto BSP using the Freescale 3.0.35 kernel.  This driver framework has not made it fully into mainline linux yet and no support is available for OpenWrt.
     24
     25
     26[=#devices]
     27= Displays Devices =
     28The IMX6Q (Quad Core) CPU has 2 IPU's (image processing units) where as the IMX6DL (Dual-Lite 2-core) has 1 IPU. Each IPU can have 2 display output paths and can therefore output to 4 different displays simultaneously (either the same or unique data).  Each of these display output paths is represented by a 'mxcfb' device.
     29
     30Various Ventana boards have different output display possibilities (listed in the order the devices are registered):
     31 * GW51xx: HDMI (Digital)
     32 * GW52xx: HDMI (Digital), LVDS (Digital)
     33 * GW53xx: HDMI (Digital), LVDS (Digital)
     34 * GW54xx: HDMI (Digital), CVBS (Analog), LVDS (Digital)
     35 * GW551x: HDMI (Digital)
     36 * GW552x: HDMI (Digital)
     37 * GW553x: HDMI (Digital)
     38
     39These four devices get registered in the kernel in a specific order (per model, see above), but can be overwritten in the bootloader (see below). This order dictates the 'framebuffer' and 'v4l' output device numbers. By convention, the first framebuffer is used for bootup and is known as the 'Primary Display'.
     40
     41As an example, the GW5400's default display order is HDMI, CVBS, then LVDS.  The table below shows how this ordering will translate in linux:
     42
     43||= Device Registration =||= Display =||= Framebuffer =||= Video =||
     44|| First               || HDMI    || fb0, fb1    || video16, video17 ||
     45|| Second              || CVBS    || fb2, fb3    || video18, video19 ||
     46|| Third               || LVDS    || fb4         || video20          ||
     47|| Fourth              || Off     || -           || -                ||
     48
     49Notice how the first two displays get two framebuffers devices and two video devices. This is because the first two displays that were registered have overlay support (subtitles, picture-in-picture etc.). Keep this in mind when determining your order.
     50
     51
     52[=#kernel-param]
     53== How to Change Device Registration Order ==
     54In order to change how the devices get registered in the kernel, we must first understand the 'video=' argument.
     55
     56There are several components to the 'video=' argument in the form of: <BR>
     57{{{video=mxcfb<n>:dev=<devicename>,<format>[,if=<interface-fmt>][,bpp=<bits-per-pixel>][,fbpix=<fb-pix-format>]}}} where:
     58||= Component   =||= What it stands for =||= Definition =||
     59|| <n>           || Framebuffer Device   || specifies the ordering of displays in Linux ||
     60|| dev=<device>  || Device Name (adv739x,hdmi,ldb) || specifies the interface driver by name ||
     61|| if=<pixelfmt> || Interface Pixel format || the pixel format from the device to the IPU (ie RGB666 is for a 18-bit display) ||
     62|| fbpix=<pixelfmt> || Framebuffer Pixel Format  || the format the /dev/fb* device accepts ||
     63|| <format>      || A modedb format - see [http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/fb/modedb.txt modedb.txt] || The resolution, framerate, and pixeldepth of the framebuffer device ||
     64
     65
     66Interface examples:
     67 * CVBS:
     68  * video=mxcfb0:dev=adv739x,BT656-NTSC,if=BT656,fbpix=RGB565
     69   * the BT656-NTSC dictates the interconnect format between the CVBS decoder chip and the IPU (hardware-specific) and is the only thing currently supported
     70 * LVDS:
     71  * For Yocto v1.6 and Yocto v1.7 the device should be specified as: '''video=mxcfb0:dev=ldb,LDB-XGA,if=RGB666''' where the LDB-XGA defines the display 'mode' detailing the resolution and timings:
     72    * LDB-XGA (1024x768 suitable for Freescale LVDS1 or DLC800FIGT3 displays)
     73    * LDB-WSVGA (1024x600 suitable for DLC700JMGT4 display)
     74  * For Yocto v1.8 the display mode is omitted and can be specified via the device-tree 'display' env var
     75    * see [wiki:ventana/LVDS] for more info
     76 * HDMI:
     77  * video=mxcfb0:dev=hdmi,1280x720M@60,if=RGB24
     78   * the 1280x720M defines the resolution and is somewhat flexible depending on your display. See Linux mode.db for details
     79
     80The instance of the 'mxcfb' device determines the framebuffer order. For example, mxcfb0 is the first display, mxcfb1 is the second, mxcfb2 is the third, and mxcfb3 is the fourth. If you are not using all four output devices be sure to disable them by setting them to 'off' to avoid mis-allocating resources (see examples below).
     81
     82Here are the steps to make HDMI the primary display, LVDS have overlay support, and completely turn off CVBS and the fourth display lane (and thus save power)
     83
     84Making sure that the jtag dongle is connected, power on the board and hit any key to break into the bootloader and type the following:
     85{{{
     86Ventana> setenv video 'video=mxcfb0:dev=hdmi,1280x720M@60,if=RGB24 video=mxcfb1:dev=ldb,LDB-XGA,if=RGB666 video=mxcfb2:off video=mxcfb3:off'
     87Ventana> saveenv
     88Ventana> reset
     89}}}
     90
     91Here are some other combinations:
     92||= Displays =||= Configuration =||= bootloader setenv command =||
     93|| 1          || HDMI            || setenv video 'video=mxcfb0:dev=hdmi,1280x720M@60,if=RGB24 video=mxcfb1:off video=mxcfb2:off video=mxcfb3:off' ||
     94|| 2          || HDMI+LVDS       || setenv video 'video=mxcfb0:dev=hdmi,1280x720M@60,if=RGB24 video=mxcfb1:dev=ldb,LDB-XGA,if=RGB666 video=mxcfb2:off video=mxcfb3:off' ||
     95|| 2          || HDMI+CVBS       || setenv video 'video=mxcfb0:dev=hdmi,1280x720M@60,if=RGB24 video=mxcfb1:dev=adv739x,BT656-NTSC,if=BT656,fbpix=RGB565 video=mxcfb2:off video=mxcfb3:off'
     96|| 2          || LVDS+CVBS       || setenv video 'video=mxcfb0:dev=ldb,LDB-XGA,if=RGB666 video=mxcfb1:dev=adv739x,BT656-NTSC,if=BT656,fbpix=RGB565 video=mxcfb2:off video=mxcfb3:off'
     97|| 3          || CVBS+LVDS+HDMI (default) || setenv video 'video=mxcfb0:dev=adv739x,BT656-NTSC,if=BT656,fbpix=RGB565 video=mxcfb1:dev=ldb,LDB-XGA,if=RGB666 video=mxcfb2:dev=hdmi,1280x720M@60,if=RGB24 video=mxcfb3:off'
     98
     99
     100[=#bootscript]
     101== U-Boot Video Script ==
     102In our Yocto images with video out support, we include a bootscript. Below are links to each of these scripts (per release):
     103* [https://github.com/Freescale/meta-freescale-3rdparty/blob/master/recipes-bsp/u-boot/u-boot-script-gateworks-imx/6x_bootscript-yocto.txt Yocto v2.3 (Pyro)]
     104* [https://github.com/Gateworks/meta-gateworks/blob/fido/recipes-bsp/u-boot/u-boot-script-gateworks/6x_bootscript-yocto.txt Yocto v1.8 (Fido)]
     105* [https://github.com/Gateworks/meta-gateworks/blob/dizzy/recipes-bsp/u-boot/u-boot-script-gateworks/6x_bootscript-yocto.txt Yocto v1.7 (Dizzy)]
     106* [https://github.com/Gateworks/meta-gateworks/blob/daisy/recipes-bsp/u-boot/u-boot-script-gateworks/6x_bootscript-yocto.txt Yocto v1.6 (Daisy)]
     107
     108Our most recent bootscript (v1.06) allows for displays to be configured for use within Linux. It currently supports the following:
     109||= Name       =||= Description                                      =||
     110|| HDMI         || HDMI Video Out                                     ||
     111|| Hannstar-XGA || Freescale MCIMX-LVDS1 10" XGA Touchscreen Display  ||
     112|| DLC700JMGT4  || GW17029 DLC700JMGT4 7" WSVGA Touchscreen Display   ||
     113|| DLC800FIGT3  || GW17030 DLC800FIGT3 8" XGA Touchscreen Display     ||
     114|| CVBS         || Analog Video Out                                   ||
     115|| none         || Turn all displays off                              ||
     116
     117Setting the {{{display}}} environmental variable will allow video to be automatically configured. Please see below for examples:
     118* For example, the following would place HDMI on the first framebuffer, and CVBS on the second:
     119 {{{
     120#!bash
     121setenv display "HDMI CVBS"
     122saveenv # Only if you want the setting to persist
     123}}}
     124
     125* This example will configure an LVDS display on the first framebuffer:
     126 {{{
     127#!bash
     128setenv display "Hannstar-XGA"
     129saveenv # Only if you want the setting to persist
     130}}}
     131
     132* This example will mark all displays to 'off':
     133 {{{
     134#!bash
     135setenv display "none"
     136saveenv # Only if you want the setting to persist
     137}}}
     138
     139Some things to note:
     140 * If the environmental variable {{{displays}}} is set, it will take precedent over {{{display}}}
     141 * If neither {{{display}}} or {{{displays}}} are set, the bootscript will try to detect a HDMI monitor and use it
     142 * Only one type of display is allowed to be set, i.e. two LVDS displays cannot be set. If this is the case, the first one specified is chosen.
     143 * The environmental variable {{{hdmi}}} can be set to dictate the HDMI output resolution: {{{1080p}}}, {{{720p}}}, {{{480p}}}. If unset, it will default to {{{1080p}}}
     144
     145Finally, this bootscript also configures how much [https://lwn.net/Articles/486301/ CMA] to allocate for the GPU/VPU on the imx6. If the environmental variable {{{mem}}} is unset, the bootscript will attempt to dynamically configure CMA. If you want to manually configure mem, you can set it to something like {{{setenv mem "cma=96M"}}}. However, if you do not want to configure mem for anything, you can set it to {{{setenv mem 'NA'}}} and the bootscript will gloss over it and allow kernel default.
     146
     147== HDMI Resolutions Supported ==
     148The {{{mxc_hdmi}}} driver by default supports the following progressive CEA modes:
     149 * !#1: 640x480p@59.94/60Hz 4:3
     150 * !#2: 720x480p@59.94/60Hz 4:3
     151 * !#3: 720x480p@59.94/60Hz 16:9
     152 * !#4: 1280x720p@59.94/60Hz 16:9
     153 * !#8: 720(1440)x240pH@59.94/60Hz 4:3
     154 * !#9: 720(1440)x240pH@59.94/60Hz 16:9
     155 * !#14: 1440x480p@59.94/60Hz 4:3
     156 * !#15: 1440x480p@59.94/60Hz 16:9
     157 * !#16: 1920x1080p@60Hz 16:9
     158 * !#17: 720x576pH@50Hz 4:3
     159 * !#18: 720x576pH@50Hz 16:9
     160 * !#19: 1280x720p@50Hz
     161 * !#23: 720(1440)x288pH@50Hz 4:3
     162 * !#24: 720(1440)x288pH@50Hz 16:9
     163 * !#29: 720(1440)x576pH@50Hz 4:3
     164 * !#30: 720(1440)x576pH@50Hz 16:9
     165 * !#31: 1920x1080p@50Hz
     166 * !#32: 1920x1080p@23.98/24Hz
     167 * !#33: 1920x1080p@25Hz
     168 * !#34: 1920x1080p@30Hz
     169 * !#41: 1280x720p@100Hz 16:9
     170 * !#47: 1280x720p@119.88/120Hz 16:9
     171
     172Out of the above list of supported modes only those present in the monitor's Extended Display Identification Data ([https://en.wikipedia.org/wiki/Extended_Display_Identification_Data EDID]) are considered.
     173
     174If your monitor supports a non-CEA mode or mode not listed above you can set the {{{only_cea=0}}} kernel commandline parameter to allow it, however you may find that this disables HDMI audio output. This can be set in the bootloader for example:
     175{{{#!bash
     176setenv extra 'mxc_hdmi.only_cea=0'
     177saveenv
     178}}}
     179
     180The default mode is specified in the kernel device-tree (for example, [https://github.com/Gateworks/linux-imx6/blob/gateworks_fslc_3.14_1.0.x_ga/arch/arm/boot/dts/imx6qdl-gw52xx.dtsi#L141 here] for the GW52xx) as '1920x1080M@60' for 1080p@60Hz. This can be overridden with a {{{video=}}} kernel parameter (see [#kernel-param above]). For example, if you monitor is defaulting to 1920x1080p@60Hz and you want it instead to be 1680x1050@59Hz from a matching mode of 'D:1680x1050p-59' in {{{/sys/class/graphics/fb0/modes}}} you could use:
     181{{{
     182setenv video 'video=mxcfb0:dev=hdmi,1680x1050M@59 video=mxcfb1:off'
     183saveenv
     184}}}
     185
     186Note that the Yocto BSP has a [https://github.com/Gateworks/meta-gateworks/blob/fido/recipes-bsp/u-boot/u-boot-script-gateworks/6x_bootscript-yocto.txt bootloader script] which will set the kernel commandline (unless the {{{video}}} bootloader env variable is set to override it). The script will use the {{{hdmi}}} env variable (and defaults to 1080p if not set) to determine the resolution and use this for HDMI if an HDMI monitor is connected at bootup. For example:
     187 * specify 480p
     188{{{#!bash
     189setenv video # make sure video var is blank for auto-configuration to work
     190setenv hdmi 480p # set to 720x480M@60
     191saveenv
     192}}}
     193 * specify 720p
     194{{{#!bash
     195setenv video # make sure video var is blank for auto-configuration to work
     196setenv hdmi 720p # set to 1280x720M@60
     197saveenv
     198}}}
     199 * specify 1080p
     200{{{#!bash
     201setenv video # make sure video var is blank for auto-configuration to work
     202setenv hdmi 1080p # set to 1920x1080M@60
     203saveenv
     204}}}
     205
     206Note that if you specify a video mode that the monitor's EDID does not support (or supports per the EDID but doesn't appear in {{{/sys/class/graphics/fb0/modes}}} because only_cea=1) the {{{mxc_hdmi}}} driver will try to find the nearest supported resolution to what you specified.
     207
     208Once booted, you can see the list of available video modes via /sys/class/graphics:
     209 * see current video mode:
     210{{{#!bash
     211# cat /sys/class/graphics/fb0/mode
     212D:1680x1050p-59
     213}}}
     214 * see available modes:
     215{{{#!bash
     216# cat /sys/class/graphics/fb0/modes
     217S:1280x1024p-60
     218S:1152x864p-75
     219V:1280x1024p-75
     220V:1024x768p-75
     221V:1024x768p-60
     222V:800x600p-75
     223V:800x600p-60
     224V:640x480p-75
     225V:640x480p-60
     226U:720x400p-70
     227D:1680x1050p-59
     228V:640x480p-60
     229}}}
     230  - 'S' refers to an EDID 'Standard Timing' in EDID
     231  - 'V' refers to a VESA mode from 'Established Timings' in EDID
     232  - 'D' refers to the 'Detailed mode' in EDID (which should be the monitors preferred or native mode)
     233 * change the resolution (must provide a mode matching those show from {{{modes}}}):
     234{{{#!bash
     235echo "V:1024x768p-60" > /sys/class/graphics/fb0/mode
     236}}}
     237
     238== Tested Resolutions and Pipelines ==
     239
     240All of these tests were run using gstreamer's '''videotestsrc''' to display a color bar on the video device.
     241
     242
     243||= OS =||= Display Type =||= Resolution & Framerate =||= result =||= Yocto 1.7 Pipeline Command =||
     244|| Yocto 1.3 || HDMI || 1920x1080p @ 60hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
     245|| Yocto 1.3 || HDMI || 1920x1080p @ 50hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
     246|| Yocto 1.3 || HDMI || 1280x720p @ 60hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
     247|| Yocto 1.3 || HDMI || 1280x720p @ 50hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
     248|| Yocto 1.3 || HDMI || 720x576p @ 60hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
     249|| Yocto 1.3 || HDMI || 720x480p @ 60hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
     250|| Yocto 1.3 || HDMI || 640x480p @ 60hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
     251|| Yocto 1.3 || HDMI || 720x480p @ 59hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
     252|| Yocto 1.3 || LVDS || 1024x768p @ 60hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video20 ||
     253|| Yocto 1.3 || CVBS || 720x480i @ 60hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video18 ||
     254
     255The pipeline command uses the gst-launch action to use the source element[videotestsrc] and pipe it into the sink element[mfw_v4lsink] and output it through the the device specified.
     256
     257'''Note''': The pipeline commands are based on the default configuration[HDMI, then CVBS, and finally LVDS].
     258
     259= Analog Video Output Format =
     260The Ventana Analog Video output is provided via the [http://www.analog.com/static/imported-files/data_sheets/ADV7390_7391_7392_7393.pdf ADV7393] Analog Video encoder chip.  This chip has 3 outputs (DAC1, DAC2, DAC3) which are all provided on the Ventana board Video Output connector.  These can be used in the following modes:
     261 * Composite (CVBS)
     262 * Y/C (S-Video)
     263 * Component RGB
     264 * Component YPrPb
     265
     266The adv7393 driver currently does not support setting the output format via software as this is outside the scope of the V4l2 API, but you can access its registers on /dev/i2c-2 slave address 0x2a.  See [http://dev.gateworks.com/datasheets/ADV7390.pdf ADV7393 datasheet] Table 37 for details.
     267
     268Notes:
     269 * CVBS refers to "Color, Video, Blanking, and Sync." - see http://en.wikipedia.org/wiki/Composite_video
     270 * CVBS modes supported: NTSC M, PAL B/D/G/H/I/M/N, PAL 60
     271
     272
     273= Framebuffer device API and Screen Blanking =
     274A framebuffer device drives a display from a memory buffer containing raw pixel information. This means it can run regardless of the hardware present.
     275
     276In linux, the framebuffers are located at {{{/dev/fb*}}}. For the configuration above (HDMI as primary, LVDS as second, rest off), the {{{/dev/}}} folder will contain {{{fb0, fb1, fb2, fb3}}}. Remember, the reason there are four framebuffers is because the first two devices get two distinct framebuffers each for overlay.
     277
     278Framebuffer information is located at {{{/sys/class/graphics/fb*}}}. Here, you can get display properties and do things such as blanking and unblanking a screen.
     279
     280
     281If we wanted to blank the primary display and unblank the secondary screen, we would do the following:
     282{{{
     283echo 1 > /sys/class/graphics/fb0/blank
     284echo 0 > /sys/class/graphics/fb2/blank
     285}}}
     286
     287Notes:
     288* echoing a 1 into a framebuffer's blank value will make it blank the screen it is associated with.
     289* echoing a 0 into a framebuffer's blank value will unblank the screen associated with it.
     290* The default linux framebuffer support will blank a console after 600 seconds of no activity.  You can specify this with a kernel parameter 'consoleblank=<n>' where <n> is the number of seconds (0 to disable auto-blanking completely), or by using the '''setterm''' userspace application if available.
     291
     292== How to Output a Test Pattern ==
     293
     294In order to display something, you can run a utility called {{{fb-test}}}. To display a variety of colors, making sure the display is unblanked, type the following and hit return: {{{fb-test -f 0}}}
     295
     296This utility is useful as you can test a variety of colors and patterns.
     297
     298Also note: Since framebuffers are just raw pixel information in memory, you can perform many low levels tasks such as directly copying the raw data with a {{{cp /dev/fb0 /tmp/image}}}. Or writing raw data back with {{{cp /tmp/image /dev/fb0}}}. This contributes to their continued fame.
     299
     300You can also use [wiki:gstreamer gstreamer's] fbdevsink element to sink to a framebuffer device.
     301
     302= Video4Linux (v4l) device API =
     303v4l is another level of abstraction to do video in linux by providing an API. You can stream video using this driver using the mfw_v4lsink element in a [[gstreamer]] pipeline. The devices for this driver are located at {{{/dev/videoXX}}}.
     304
     305== Output to Display ==
     306You can download a small shell script [https://raw.github.com/Gateworks/meta-gateworks/master/recipes-support/gateworks-test/gateworks-test/test_video here] that will unblank/blank displays of your choosing to make typing tedious commands easier.
     307
     308At the heart of the script is a [[gstreamer]] pipeline that pipe's a "video test source" to a video4linux sink device of your choosing.
     309
     310= Troubleshooting =
     311
     312These steps are for HDMI output only.
     313
     314Confirm that the modes are properly showing from the command console with the following commands:
     315{{{
     316root@ventana:~# cat /sys/class/graphics/fb0/mode
     317S:1280x720p-60
     318
     319root@ventana:~# cat /sys/class/graphics/fb0/modes
     320S:1920x1080p-60
     321S:1920x1080p-50
     322S:1280x720p-60
     323S:1280x720p-50
     324S:720x576p-50
     325S:720x576p-50
     326S:720x480p-60
     327S:720x480p-60
     328V:640x480p-60
     329D:1280x720p-60
     330V:640x480p-60
     331}}}
     332
     333Confirm a monitor is detected on the I2C bus from the bootloader and that the EDID can be read:
     334{{{#!bash
     335Ventana > i2c dev 2 && i2c probe && i2c edid 0x50
     336Setting bus to 2
     337Valid chip addresses: 1E 20 37 3A 50
     338EDID version: 1.3
     339Product ID code: d015
     340Manufacturer: DEL
     341Serial number: 30323853
     342Manufactured in week: 14 year: 2008
     343Video input definition: digital signal, voltage level 0
     344Monitor is RGB
     345Maximum visible display size: 47 cm x 30 cm
     346Power management features: active off, suspend, standby
     347Estabilished timings:
     348        720x400         70 Hz (VGA 640x400, IBM)
     349        640x480         60 Hz (VGA)
     350        640x480         75 Hz (VESA)
     351        800x600         60 Hz (VESA)
     352        800x600         75 Hz (VESA)
     353        1024x768        60 Hz (VESA)
     354        1024x768        75 Hz (VESA)
     355        1280x1024       75 (VESA)
     356Standard timings:
     357        1152x864        75 Hz
     358        1280x1024       60 Hz
     359        1680x1050       59 Hz (detailed)
     360Monitor serial number: KU311844028S
     361Monitor range limits, horizontal sync: 30-83 kHz, vertical refresh: 56-75 Hz, max pixel clock: 140 MHz
     362Monitor name: DELL E228WFP
     363}}}