Changes between Initial Version and Version 1 of Yocto/Video_Out

Timestamp:

10/22/2017 05:28:45 AM (7 years ago)

Author:

trac

Comment:

--

Yocto/Video_Out

@@ +1 @@
+[[PageOutline]]
+
+= Goals =
+The intended goal of this article is to provide the user with enough information so they can display an image onto a screen of their choosing.
+
+For information regarding Audio / Video breakout adapters for Ventana see:
+ * [wiki:ventana/breakoutadapters Audio / Video breakout Adapters]
+
+
+[=#drivers]
+= Video Output Drivers =
+There are two different display driver API's in Linux today:
+ * [https://www.kernel.org/doc/Documentation/fb/framebuffer.txt framebuffer]
+  * Very simple
+  * Low level
+  * [https://www.kernel.org/doc/Documentation/fb/framebuffer.txt See this link for much more detail]
+
+ * [https://www.kernel.org/doc/Documentation/video4linux/v4l2-framework.txt video4linux (v4l)]
+  * More abstract
+  * Mainly used for video and other graphical applications
+  * [https://www.kernel.org/doc/Documentation/video4linux/v4l2-framework.txt See this link for much more detail]
+
+The i.MX6 Video output framework drivers are currently only available in the Yocto BSP using the Freescale 3.0.35 kernel.  This driver framework has not made it fully into mainline linux yet and no support is available for OpenWrt.
+
+
+[=#devices]
+= Displays Devices =
+The IMX6Q (Quad Core) CPU has 2 IPU's (image processing units) where as the IMX6DL (Dual-Lite 2-core) has 1 IPU. Each IPU can have 2 display output paths and can therefore output to 4 different displays simultaneously (either the same or unique data).  Each of these display output paths is represented by a 'mxcfb' device.
+
+Various Ventana boards have different output display possibilities (listed in the order the devices are registered):
+ * GW51xx: HDMI (Digital)
+ * GW52xx: HDMI (Digital), LVDS (Digital)
+ * GW53xx: HDMI (Digital), LVDS (Digital)
+ * GW54xx: HDMI (Digital), CVBS (Analog), LVDS (Digital)
+ * GW551x: HDMI (Digital)
+ * GW552x: HDMI (Digital)
+ * GW553x: HDMI (Digital)
+
+These four devices get registered in the kernel in a specific order (per model, see above), but can be overwritten in the bootloader (see below). This order dictates the 'framebuffer' and 'v4l' output device numbers. By convention, the first framebuffer is used for bootup and is known as the 'Primary Display'.
+
+As an example, the GW5400's default display order is HDMI, CVBS, then LVDS.  The table below shows how this ordering will translate in linux:
+
+||= Device Registration =||= Display =||= Framebuffer =||= Video =||
+|| First               || HDMI    || fb0, fb1    || video16, video17 ||
+|| Second              || CVBS    || fb2, fb3    || video18, video19 ||
+|| Third               || LVDS    || fb4         || video20          ||
+|| Fourth              || Off     || -           || -                ||
+
+Notice how the first two displays get two framebuffers devices and two video devices. This is because the first two displays that were registered have overlay support (subtitles, picture-in-picture etc.). Keep this in mind when determining your order.
+
+
+[=#kernel-param]
+== How to Change Device Registration Order ==
+In order to change how the devices get registered in the kernel, we must first understand the 'video=' argument.
+
+There are several components to the 'video=' argument in the form of: <BR>
+{{{video=mxcfb<n>:dev=<devicename>,<format>[,if=<interface-fmt>][,bpp=<bits-per-pixel>][,fbpix=<fb-pix-format>]}}} where:
+||= Component   =||= What it stands for =||= Definition =||
+|| <n>           || Framebuffer Device   || specifies the ordering of displays in Linux ||
+|| dev=<device>  || Device Name (adv739x,hdmi,ldb) || specifies the interface driver by name ||
+|| if=<pixelfmt> || Interface Pixel format || the pixel format from the device to the IPU (ie RGB666 is for a 18-bit display) ||
+|| fbpix=<pixelfmt> || Framebuffer Pixel Format  || the format the /dev/fb* device accepts ||
+|| <format>      || A modedb format - see [http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/fb/modedb.txt modedb.txt] || The resolution, framerate, and pixeldepth of the framebuffer device ||
+
+
+Interface examples:
+ * CVBS:
+  * video=mxcfb0:dev=adv739x,BT656-NTSC,if=BT656,fbpix=RGB565
+   * the BT656-NTSC dictates the interconnect format between the CVBS decoder chip and the IPU (hardware-specific) and is the only thing currently supported
+ * LVDS:
+  * For Yocto v1.6 and Yocto v1.7 the device should be specified as: '''video=mxcfb0:dev=ldb,LDB-XGA,if=RGB666''' where the LDB-XGA defines the display 'mode' detailing the resolution and timings:
+    * LDB-XGA (1024x768 suitable for Freescale LVDS1 or DLC800FIGT3 displays)
+    * LDB-WSVGA (1024x600 suitable for DLC700JMGT4 display)
+  * For Yocto v1.8 the display mode is omitted and can be specified via the device-tree 'display' env var
+    * see [wiki:ventana/LVDS] for more info
+ * HDMI:
+  * video=mxcfb0:dev=hdmi,1280x720M@60,if=RGB24
+   * the 1280x720M defines the resolution and is somewhat flexible depending on your display. See Linux mode.db for details
+
+The instance of the 'mxcfb' device determines the framebuffer order. For example, mxcfb0 is the first display, mxcfb1 is the second, mxcfb2 is the third, and mxcfb3 is the fourth. If you are not using all four output devices be sure to disable them by setting them to 'off' to avoid mis-allocating resources (see examples below).
+
+Here are the steps to make HDMI the primary display, LVDS have overlay support, and completely turn off CVBS and the fourth display lane (and thus save power)
+
+Making sure that the jtag dongle is connected, power on the board and hit any key to break into the bootloader and type the following:
+{{{
+Ventana> setenv video 'video=mxcfb0:dev=hdmi,1280x720M@60,if=RGB24 video=mxcfb1:dev=ldb,LDB-XGA,if=RGB666 video=mxcfb2:off video=mxcfb3:off'
+Ventana> saveenv
+Ventana> reset
+}}}
+
+Here are some other combinations:
+||= Displays =||= Configuration =||= bootloader setenv command =||
+|| 1          || HDMI            || setenv video 'video=mxcfb0:dev=hdmi,1280x720M@60,if=RGB24 video=mxcfb1:off video=mxcfb2:off video=mxcfb3:off' ||
+|| 2          || HDMI+LVDS       || setenv video 'video=mxcfb0:dev=hdmi,1280x720M@60,if=RGB24 video=mxcfb1:dev=ldb,LDB-XGA,if=RGB666 video=mxcfb2:off video=mxcfb3:off' ||
+|| 2          || HDMI+CVBS       || setenv video 'video=mxcfb0:dev=hdmi,1280x720M@60,if=RGB24 video=mxcfb1:dev=adv739x,BT656-NTSC,if=BT656,fbpix=RGB565 video=mxcfb2:off video=mxcfb3:off'
+|| 2          || LVDS+CVBS       || setenv video 'video=mxcfb0:dev=ldb,LDB-XGA,if=RGB666 video=mxcfb1:dev=adv739x,BT656-NTSC,if=BT656,fbpix=RGB565 video=mxcfb2:off video=mxcfb3:off'
+|| 3          || CVBS+LVDS+HDMI (default) || setenv video 'video=mxcfb0:dev=adv739x,BT656-NTSC,if=BT656,fbpix=RGB565 video=mxcfb1:dev=ldb,LDB-XGA,if=RGB666 video=mxcfb2:dev=hdmi,1280x720M@60,if=RGB24 video=mxcfb3:off'
+
+
+[=#bootscript]
+== U-Boot Video Script ==
+In our Yocto images with video out support, we include a bootscript. Below are links to each of these scripts (per release):
+* [https://github.com/Freescale/meta-freescale-3rdparty/blob/master/recipes-bsp/u-boot/u-boot-script-gateworks-imx/6x_bootscript-yocto.txt Yocto v2.3 (Pyro)]
+* [https://github.com/Gateworks/meta-gateworks/blob/fido/recipes-bsp/u-boot/u-boot-script-gateworks/6x_bootscript-yocto.txt Yocto v1.8 (Fido)]
+* [https://github.com/Gateworks/meta-gateworks/blob/dizzy/recipes-bsp/u-boot/u-boot-script-gateworks/6x_bootscript-yocto.txt Yocto v1.7 (Dizzy)]
+* [https://github.com/Gateworks/meta-gateworks/blob/daisy/recipes-bsp/u-boot/u-boot-script-gateworks/6x_bootscript-yocto.txt Yocto v1.6 (Daisy)]
+
+Our most recent bootscript (v1.06) allows for displays to be configured for use within Linux. It currently supports the following:
+||= Name       =||= Description                                      =||
+|| HDMI         || HDMI Video Out                                     ||
+|| Hannstar-XGA || Freescale MCIMX-LVDS1 10" XGA Touchscreen Display  ||
+|| DLC700JMGT4  || GW17029 DLC700JMGT4 7" WSVGA Touchscreen Display   ||
+|| DLC800FIGT3  || GW17030 DLC800FIGT3 8" XGA Touchscreen Display     ||
+|| CVBS         || Analog Video Out                                   ||
+|| none         || Turn all displays off                              ||
+
+Setting the {{{display}}} environmental variable will allow video to be automatically configured. Please see below for examples:
+* For example, the following would place HDMI on the first framebuffer, and CVBS on the second:
+ {{{
+#!bash
+setenv display "HDMI CVBS"
+saveenv # Only if you want the setting to persist
+}}}
+
+* This example will configure an LVDS display on the first framebuffer:
+ {{{
+#!bash
+setenv display "Hannstar-XGA"
+saveenv # Only if you want the setting to persist
+}}}
+
+* This example will mark all displays to 'off':
+ {{{
+#!bash
+setenv display "none"
+saveenv # Only if you want the setting to persist
+}}}
+
+Some things to note:
+ * If the environmental variable {{{displays}}} is set, it will take precedent over {{{display}}}
+ * If neither {{{display}}} or {{{displays}}} are set, the bootscript will try to detect a HDMI monitor and use it
+ * Only one type of display is allowed to be set, i.e. two LVDS displays cannot be set. If this is the case, the first one specified is chosen.
+ * The environmental variable {{{hdmi}}} can be set to dictate the HDMI output resolution: {{{1080p}}}, {{{720p}}}, {{{480p}}}. If unset, it will default to {{{1080p}}}
+
+Finally, this bootscript also configures how much [https://lwn.net/Articles/486301/ CMA] to allocate for the GPU/VPU on the imx6. If the environmental variable {{{mem}}} is unset, the bootscript will attempt to dynamically configure CMA. If you want to manually configure mem, you can set it to something like {{{setenv mem "cma=96M"}}}. However, if you do not want to configure mem for anything, you can set it to {{{setenv mem 'NA'}}} and the bootscript will gloss over it and allow kernel default.
+
+== HDMI Resolutions Supported ==
+The {{{mxc_hdmi}}} driver by default supports the following progressive CEA modes:
+ * !#1: 640x480p@59.94/60Hz 4:3 
+ * !#2: 720x480p@59.94/60Hz 4:3 
+ * !#3: 720x480p@59.94/60Hz 16:9 
+ * !#4: 1280x720p@59.94/60Hz 16:9 
+ * !#8: 720(1440)x240pH@59.94/60Hz 4:3 
+ * !#9: 720(1440)x240pH@59.94/60Hz 16:9 
+ * !#14: 1440x480p@59.94/60Hz 4:3 
+ * !#15: 1440x480p@59.94/60Hz 16:9 
+ * !#16: 1920x1080p@60Hz 16:9 
+ * !#17: 720x576pH@50Hz 4:3 
+ * !#18: 720x576pH@50Hz 16:9 
+ * !#19: 1280x720p@50Hz 
+ * !#23: 720(1440)x288pH@50Hz 4:3 
+ * !#24: 720(1440)x288pH@50Hz 16:9 
+ * !#29: 720(1440)x576pH@50Hz 4:3 
+ * !#30: 720(1440)x576pH@50Hz 16:9 
+ * !#31: 1920x1080p@50Hz 
+ * !#32: 1920x1080p@23.98/24Hz 
+ * !#33: 1920x1080p@25Hz 
+ * !#34: 1920x1080p@30Hz 
+ * !#41: 1280x720p@100Hz 16:9 
+ * !#47: 1280x720p@119.88/120Hz 16:9 
+
+Out of the above list of supported modes only those present in the monitor's Extended Display Identification Data ([https://en.wikipedia.org/wiki/Extended_Display_Identification_Data EDID]) are considered.
+
+If your monitor supports a non-CEA mode or mode not listed above you can set the {{{only_cea=0}}} kernel commandline parameter to allow it, however you may find that this disables HDMI audio output. This can be set in the bootloader for example:
+{{{#!bash
+setenv extra 'mxc_hdmi.only_cea=0'
+saveenv
+}}}
+
+The default mode is specified in the kernel device-tree (for example, [https://github.com/Gateworks/linux-imx6/blob/gateworks_fslc_3.14_1.0.x_ga/arch/arm/boot/dts/imx6qdl-gw52xx.dtsi#L141 here] for the GW52xx) as '1920x1080M@60' for 1080p@60Hz. This can be overridden with a {{{video=}}} kernel parameter (see [#kernel-param above]). For example, if you monitor is defaulting to 1920x1080p@60Hz and you want it instead to be 1680x1050@59Hz from a matching mode of 'D:1680x1050p-59' in {{{/sys/class/graphics/fb0/modes}}} you could use:
+{{{
+setenv video 'video=mxcfb0:dev=hdmi,1680x1050M@59 video=mxcfb1:off'
+saveenv
+}}}
+
+Note that the Yocto BSP has a [https://github.com/Gateworks/meta-gateworks/blob/fido/recipes-bsp/u-boot/u-boot-script-gateworks/6x_bootscript-yocto.txt bootloader script] which will set the kernel commandline (unless the {{{video}}} bootloader env variable is set to override it). The script will use the {{{hdmi}}} env variable (and defaults to 1080p if not set) to determine the resolution and use this for HDMI if an HDMI monitor is connected at bootup. For example:
+ * specify 480p
+{{{#!bash
+setenv video # make sure video var is blank for auto-configuration to work
+setenv hdmi 480p # set to 720x480M@60
+saveenv
+}}}
+ * specify 720p
+{{{#!bash
+setenv video # make sure video var is blank for auto-configuration to work
+setenv hdmi 720p # set to 1280x720M@60
+saveenv
+}}}
+ * specify 1080p
+{{{#!bash
+setenv video # make sure video var is blank for auto-configuration to work
+setenv hdmi 1080p # set to 1920x1080M@60
+saveenv
+}}}
+
+Note that if you specify a video mode that the monitor's EDID does not support (or supports per the EDID but doesn't appear in {{{/sys/class/graphics/fb0/modes}}} because only_cea=1) the {{{mxc_hdmi}}} driver will try to find the nearest supported resolution to what you specified.
+
+Once booted, you can see the list of available video modes via /sys/class/graphics:
+ * see current video mode:
+{{{#!bash
+# cat /sys/class/graphics/fb0/mode
+D:1680x1050p-59
+}}}
+ * see available modes:
+{{{#!bash
+# cat /sys/class/graphics/fb0/modes
+S:1280x1024p-60
+S:1152x864p-75
+V:1280x1024p-75
+V:1024x768p-75
+V:1024x768p-60
+V:800x600p-75
+V:800x600p-60
+V:640x480p-75
+V:640x480p-60
+U:720x400p-70
+D:1680x1050p-59
+V:640x480p-60
+}}}
+  - 'S' refers to an EDID 'Standard Timing' in EDID
+  - 'V' refers to a VESA mode from 'Established Timings' in EDID
+  - 'D' refers to the 'Detailed mode' in EDID (which should be the monitors preferred or native mode)
+ * change the resolution (must provide a mode matching those show from {{{modes}}}):
+{{{#!bash
+echo "V:1024x768p-60" > /sys/class/graphics/fb0/mode
+}}}
+
+== Tested Resolutions and Pipelines ==
+
+All of these tests were run using gstreamer's '''videotestsrc''' to display a color bar on the video device.
+
+
+||= OS =||= Display Type =||= Resolution & Framerate =||= result =||= Yocto 1.7 Pipeline Command =||
+|| Yocto 1.3 || HDMI || 1920x1080p @ 60hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
+|| Yocto 1.3 || HDMI || 1920x1080p @ 50hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
+|| Yocto 1.3 || HDMI || 1280x720p @ 60hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
+|| Yocto 1.3 || HDMI || 1280x720p @ 50hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
+|| Yocto 1.3 || HDMI || 720x576p @ 60hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
+|| Yocto 1.3 || HDMI || 720x480p @ 60hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
+|| Yocto 1.3 || HDMI || 640x480p @ 60hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
+|| Yocto 1.3 || HDMI || 720x480p @ 59hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16 ||
+|| Yocto 1.3 || LVDS || 1024x768p @ 60hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video20 ||
+|| Yocto 1.3 || CVBS || 720x480i @ 60hz|| Success || gst-launch videotestsrc ! mfw_v4lsink device=/dev/video18 ||
+
+The pipeline command uses the gst-launch action to use the source element[videotestsrc] and pipe it into the sink element[mfw_v4lsink] and output it through the the device specified.
+
+'''Note''': The pipeline commands are based on the default configuration[HDMI, then CVBS, and finally LVDS].
+
+= Analog Video Output Format =
+The Ventana Analog Video output is provided via the [http://www.analog.com/static/imported-files/data_sheets/ADV7390_7391_7392_7393.pdf ADV7393] Analog Video encoder chip.  This chip has 3 outputs (DAC1, DAC2, DAC3) which are all provided on the Ventana board Video Output connector.  These can be used in the following modes:
+ * Composite (CVBS)
+ * Y/C (S-Video)
+ * Component RGB
+ * Component YPrPb
+
+The adv7393 driver currently does not support setting the output format via software as this is outside the scope of the V4l2 API, but you can access its registers on /dev/i2c-2 slave address 0x2a.  See [http://dev.gateworks.com/datasheets/ADV7390.pdf ADV7393 datasheet] Table 37 for details.
+
+Notes:
+ * CVBS refers to "Color, Video, Blanking, and Sync." - see http://en.wikipedia.org/wiki/Composite_video
+ * CVBS modes supported: NTSC M, PAL B/D/G/H/I/M/N, PAL 60
+
+
+= Framebuffer device API and Screen Blanking =
+A framebuffer device drives a display from a memory buffer containing raw pixel information. This means it can run regardless of the hardware present.
+
+In linux, the framebuffers are located at {{{/dev/fb*}}}. For the configuration above (HDMI as primary, LVDS as second, rest off), the {{{/dev/}}} folder will contain {{{fb0, fb1, fb2, fb3}}}. Remember, the reason there are four framebuffers is because the first two devices get two distinct framebuffers each for overlay.
+
+Framebuffer information is located at {{{/sys/class/graphics/fb*}}}. Here, you can get display properties and do things such as blanking and unblanking a screen.
+
+
+If we wanted to blank the primary display and unblank the secondary screen, we would do the following:
+{{{
+echo 1 > /sys/class/graphics/fb0/blank
+echo 0 > /sys/class/graphics/fb2/blank
+}}}
+
+Notes:
+* echoing a 1 into a framebuffer's blank value will make it blank the screen it is associated with.
+* echoing a 0 into a framebuffer's blank value will unblank the screen associated with it.
+* The default linux framebuffer support will blank a console after 600 seconds of no activity.  You can specify this with a kernel parameter 'consoleblank=<n>' where <n> is the number of seconds (0 to disable auto-blanking completely), or by using the '''setterm''' userspace application if available.
+
+== How to Output a Test Pattern ==
+
+In order to display something, you can run a utility called {{{fb-test}}}. To display a variety of colors, making sure the display is unblanked, type the following and hit return: {{{fb-test -f 0}}}
+
+This utility is useful as you can test a variety of colors and patterns.
+
+Also note: Since framebuffers are just raw pixel information in memory, you can perform many low levels tasks such as directly copying the raw data with a {{{cp /dev/fb0 /tmp/image}}}. Or writing raw data back with {{{cp /tmp/image /dev/fb0}}}. This contributes to their continued fame.
+
+You can also use [wiki:gstreamer gstreamer's] fbdevsink element to sink to a framebuffer device.
+
+= Video4Linux (v4l) device API =
+v4l is another level of abstraction to do video in linux by providing an API. You can stream video using this driver using the mfw_v4lsink element in a [[gstreamer]] pipeline. The devices for this driver are located at {{{/dev/videoXX}}}.
+
+== Output to Display ==
+You can download a small shell script [https://raw.github.com/Gateworks/meta-gateworks/master/recipes-support/gateworks-test/gateworks-test/test_video here] that will unblank/blank displays of your choosing to make typing tedious commands easier.
+
+At the heart of the script is a [[gstreamer]] pipeline that pipe's a "video test source" to a video4linux sink device of your choosing.
+
+= Troubleshooting =
+
+These steps are for HDMI output only.
+
+Confirm that the modes are properly showing from the command console with the following commands:
+{{{
+root@ventana:~# cat /sys/class/graphics/fb0/mode
+S:1280x720p-60
+
+root@ventana:~# cat /sys/class/graphics/fb0/modes
+S:1920x1080p-60
+S:1920x1080p-50
+S:1280x720p-60
+S:1280x720p-50
+S:720x576p-50
+S:720x576p-50
+S:720x480p-60
+S:720x480p-60
+V:640x480p-60
+D:1280x720p-60
+V:640x480p-60
+}}}
+
+Confirm a monitor is detected on the I2C bus from the bootloader and that the EDID can be read:
+{{{#!bash
+Ventana > i2c dev 2 && i2c probe && i2c edid 0x50
+Setting bus to 2
+Valid chip addresses: 1E 20 37 3A 50
+EDID version: 1.3
+Product ID code: d015
+Manufacturer: DEL
+Serial number: 30323853
+Manufactured in week: 14 year: 2008
+Video input definition: digital signal, voltage level 0
+Monitor is RGB
+Maximum visible display size: 47 cm x 30 cm
+Power management features: active off, suspend, standby
+Estabilished timings:
+        720x400         70 Hz (VGA 640x400, IBM)
+        640x480         60 Hz (VGA)
+        640x480         75 Hz (VESA)
+        800x600         60 Hz (VESA)
+        800x600         75 Hz (VESA)
+        1024x768        60 Hz (VESA)
+        1024x768        75 Hz (VESA)
+        1280x1024       75 (VESA)
+Standard timings:
+        1152x864        75 Hz
+        1280x1024       60 Hz
+        1680x1050       59 Hz (detailed)
+Monitor serial number: KU311844028S
+Monitor range limits, horizontal sync: 30-83 kHz, vertical refresh: 56-75 Hz, max pixel clock: 140 MHz
+Monitor name: DELL E228WFP
+}}}