Changes between Initial Version and Version 1 of Yocto/gstreamer/latency


Ignore:
Timestamp:
10/22/2017 05:28:45 AM (2 years ago)
Author:
trac
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Yocto/gstreamer/latency

    v1 v1  
     1[[PageOutline]]
     2= Latency =
     3Latency is the term used to describe how long it takes for video from the source side to get to the sink side. For example, if a video system was described as having a latency of 1 second, that would mean that it takes 1 second for the video to get from the capture (or reading a file) side to the actual display.
     4
     5This section will attempt to describe some real world latencies that we have seen using gstreamer and our product. Because latency is highly dependent on the different 'knobs' that you can turn, each section will be a very specific test setup. This is important, as you cannot reliably compare latencies when you start to introduce/remove certain feature, video capture ports/types, sink types etc.
     6
     7The below values differ in capture/display device+driver in the following ways that affect latency
     8 * Frames Per Second (FPS)
     9 * Interlace vs. Progressive
     10 * Capture device Output format (RAW vs JPEG)
     11 * Capture device's inherent latency
     12 * Display device's inherent latency
     13
     14Please note, the below values are '''not''' comparisons, but rather completely isolated and individual examples that happen to be categorized based on input types. This is because latency is affected by several things, some of which aren't always applicable to the scenario demonstrated.
     15
     16To see prior numbers, please see [http://trac.gateworks.com/wiki/Yocto/gstreamer/latency?version=3 this older page revision].
     17
     18[=#measure]
     19== Measuring Latency ==
     20We've come up with a synchronous method of testing total system latency. We wrote [https://github.com/Gateworks/gateworks-sample-apps/blob/master/gw-latency-tester/gw-latency-tester.c a program] that uses the on-board gpio's to control an ambient light sensor ([http://www.digikey.com/product-detail/en/TEPT5700/751-1059-ND/1681193 Vishay Semiconductor Opto Division TEPT5700]) and a bright blue LED. A camera would be pointed to the LED while a GStreamer pipeline would pipe it to some sink. Finally, the [https://github.com/Gateworks/gateworks-sample-apps/blob/master/gw-latency-tester/gw-latency-tester.c program] controlling the LED and light sensor turns the LED on, measures how long it took the ambient light sensor to detect it, and repeat several times. If you would like to repeat our findings, please build the [https://github.com/Gateworks/gateworks-sample-apps/blob/master/gw-latency-tester/gw-latency-tester.c gw-latency-tester] program and use in conjunction with the following hardware:
     21* [http://www.digikey.com/product-detail/en/TEPT5700/751-1059-ND/1681193 Vishay Semiconductor Opto Division TEPT5700]
     22* Bright Blue or White LED
     23
     24Pinout to the DIO connector is as follows:
     25{{{
     26DIO0 - Power to the ambient light sensor (Collector Side), tie other end to DIO3
     27DIO1 - Power to the LED, tie other end to GND
     28DIO2 - [OPTIONAL] Status LED, tie other end to GND
     29DIO3 - Receiver end of ambient light sensor (Emitter Side), tie other end to DIO0
     30GND  -
     31}}}
     32
     33Usage is as follows:
     34{{{
     35root@ventana:~# ./gw-latency-tester
     36gw-latency-tester <pwr_dio emit_dio led_dio recv_dio> [<count>] [<udelay>] [<alpha>]
     37
     38   DIO:  0  1  2  3
     39         ----------
     40GW54xx:  9 19 41 42
     41GW53xx: 16 19 17 20
     42GW52xx: 16 19 17 20
     43GW51xx: 16 19 17 18
     44GW551x: Unsupported
     45GW552x: 16 19 17 20
     46}}}
     47
     48The below results will show how the program was run, so please refer to them for examples.
     49
     50In general (unless otherwise specified), we use the following displays configured in the following ways:
     51 * HDMI: Hanns-G HL273
     52  * Resolution: 1920x1080p-60
     53  * Manufacturer specified latency: 2ms
     54 * LVDS: !HannStar HSD100PXN1
     55  * Resolution: 1024x768p-60
     56  * Manufacturer specified rise time latency: 3-6ms
     57  * Manufacturer specified fall time latency: 9-18ms
     58 * CVBS: Generic 5" inch TFT LCD Monitor (AC-530DB)
     59  * Resolution: D:720x480i-60
     60
     61=== Accounting for Latency ===
     62When we say a system has a latency of {{{66ms}}}, this number only accounts from end to end devices. However, if we know certain device latencies (such as a displays), we can subtract that off and get fine-tuned numbers.
     63
     64Using the measurement method described [wiki:Yocto/gstreamer/latency#measure above], calculating the circuitry involved RC time constants, LED Diode rise/fall times, and the response time of the ambient light sensor, we get a "circuitry latency" of: (sensor fall time response) + (led rise time response) == {{{(2us + 13us) + (200ns + 13us) = 28.200us}}}
     65
     66Because of DIO levels, we'll round that number down to 28us. Plus add in latency from an input (let's assume a camera has exactly 1framerate of latency at 30fps) at 33ms, and finally add in the display latency (we'll assume 4ms). Adding these together produces a total of: {{{28us + 33ms + 4ms = 37.28ms}}} of latency. We can now say that latency inherent to our product is {{{66ms - 37.28ms = 28.72ms}}}.
     67
     68== Latency Summary Table ==
     69Here is a summary table of the below results.
     70
     71Loopback:
     72||= Type =||= Capture =||= Display =||= End-To-End Latency =||
     73|| Analog || CVBS(NTSC:'''480i@30fps''') [[BR]] imxv4l2videosrc(IPU)        || CVBS monitor [[BR]] imxg2dvideosink(GPU) || 56ms  ||
     74|| Analog || CVBS(NTSC:'''480i@30fps''') [[BR]] imxv4l2videosrc(IPU)        || HDMI monitor [[BR]] imxg2dvideosink(GPU) || 66ms  ||
     75|| Analog || CVBS(NTSC:'''480i@30fps''') [[BR]] imxv4l2videosrc(IPU)        || LVDS monitor [[BR]] imxg2dvideosink(GPU) || 76ms  ||
     76|| ||
     77|| Analog || PCI-AVC8000(NTSC:'''480i@30fps''') [[BR]] imxv4l2videosrc(IPU) || CVBS monitor [[BR]] imxg2dvideosink(GPU) || 55ms  ||
     78|| Analog || PCI-AVC8000(NTSC:'''480i@30fps''') [[BR]] imxv4l2videosrc(IPU) || HDMI monitor [[BR]] imxg2dvideosink(GPU) || 67ms  ||
     79|| Analog || PCI-AVC8000(NTSC:'''480i@30fps''') [[BR]] imxv4l2videosrc(IPU) || LVDS monitor [[BR]] imxg2dvideosink(GPU) || 78ms  ||
     80|| ||
     81|| Digital || HDMI(GoPro:'''720p@60fps''') [[BR]] imxv4l2videosrc(IPU)      || CVBS monitor [[BR]] imxg2dvideosink(GPU) || 92ms  ||
     82|| Digital || HDMI(GoPro:'''720p@60fps''') [[BR]] imxv4l2videosrc(IPU)      || HDMI monitor [[BR]] imxg2dvideosink(GPU) || 96ms  ||
     83|| Digital || HDMI(GoPro:'''720p@60fps''') [[BR]] imxv4l2videosrc(IPU)      || LVDS monitor [[BR]] imxg2dvideosink(GPU) || 117ms ||
     84|| ||
     85|| Digital || HDMI(GoPro:'''480p@60''') [[BR]] imxv4l2videosrc(IPU)      || HDMI monitor [[BR]] imxg2dvideosink(GPU) || 101ms ||
     86|| Digital || HDMI(GoPro:'''720p@30''') [[BR]] imxv4l2videosrc(IPU)      || HDMI monitor [[BR]] imxg2dvideosink(GPU) || 113ms ||
     87|| Digital || HDMI(GoPro:'''960p@60''') [[BR]] imxv4l2videosrc(IPU)      || HDMI monitor [[BR]] imxg2dvideosink(GPU) || 157ms ||
     88|| Digital || HDMI(GoPro:'''1080p@60''') [[BR]] imxv4l2videosrc(IPU)      || HDMI monitor [[BR]] imxg2dvideosink(GPU) || 175ms ||
     89
     90RTSP Streaming:
     91||= Capture =||= Transform =||= Encode(VPU) =||= Stream =||= Decode(VPU) =||= Display =||= End-To-End Latency =||
     92|| CVBS(NTSC:'''480i@30fps''') [[BR]]yuv422smp [[BR]] imxv4l2videosrc(IPU)        || imxipuvideotransform(IPU) || imxvpuenc_h264 [[BR]] 3mbps CBR   || RTSP [[BR]] latency=10ms || imxvpudec [[BR]] (h264) || HDMI monitor [[BR]] imxg2dvideosink(GPU) || 98ms  ||
     93|| PCI-AVC8000(NTSC:'''480i@30fps''') [[BR]]yuv422smp [[BR]] imxv4l2videosrc(IPU) || imxipuvideotransform(IPU) || imxvpuenc_h264 [[BR]] 3mbps CBR   || RTSP [[BR]] latency=10ms || imxvpudec [[BR]] (h264) || HDMI monitor [[BR]] imxg2dvideosink(GPU) || 98ms  ||
     94|| HDMI(GoPro:'''720p@60fps''') [[BR]]yuv422smp [[BR]] imxv4l2videosrc(IPU)      || imxipuvideotransform(IPU) || imxvpuenc_h264 [[BR]] 5.5mbps CBR || RTSP [[BR]] latency=10ms || imxvpudec [[BR]] (h264) || HDMI monitor [[BR]] imxg2dvideosink(GPU) || 160ms ||
     95
     96Notes:
     97 1. {{{imxipuvideosink}}} and {{{imxg2dvideosink}}} have the same latencies.
     98 2. LVDS display requires a transform from YUV to RGB colorspace.
     99 2. HDMI capture using yuv422smp requires a transform before encoding while yuv422bt656 does not.
     100 3. IPU transforms with a width or height greater than 1024 (720p/720i/1080p/1080i) require striping equating to two transforms
     101 
     102
     103== CVBS (Analog) In ==
     104Using a 208C Wired Micro Mini Camera with NTSC 720x480@30FPS output captured via the Ventana Analog in ([http://www.analog.com/media/en/technical-documentation/data-sheets/ADV7180.pdf adv7180 SDTV Video Decoder]) with 8bit ITU-R BT656 output to the IMX6 CSI, the following latencies were found using a GW5400:
     105
     106=== CVBS in to HDMI out ===
     107* CVBS Input to HDMI Output
     108{{{
     109#!bash
     110gst-launch-1.0 imxv4l2videosrc device=/dev/video1 ! imxg2dvideosink framebuffer=/dev/fb0
     111}}}
     112{{{
     113#!bash
     114gw-latency-tester 9 19 41 42 60 0
     115=== Summary ===
     116Ran 60 times
     117Max Latency: 0.082425s
     118Min Latency: 0.052815s
     119Max Jitter : 0.029610s
     120Exponential Moving Average (alpha=0.1): 0.065967s
     121}}}
     122
     123=== CVBS in to LVDS out ===
     124* CVBS Input to LVDS Output
     125{{{
     126#!bash
     127gst-launch-1.0 imxv4l2videosrc device=/dev/video1 ! imxg2dvideosink framebuffer=/dev/fb4
     128}}}
     129{{{
     130#!bash
     131gw-latency-tester 9 19 41 42 60 0
     132=== Summary ===
     133Ran 60 times
     134Max Latency: 0.082155s
     135Min Latency: 0.055859s
     136Max Jitter : 0.026296s
     137Exponential Moving Average (alpha=0.1): 0.076021s
     138}}}
     139
     140=== CVBS in to CVBS out ===
     141* CVBS Input to CVBS Output
     142{{{
     143#!bash
     144gst-launch-1.0 imxv4l2videosrc device=/dev/video1 ! imxg2dvideosink framebuffer=/dev/fb2
     145}}}
     146{{{
     147#!bash
     148gw-latency-tester 9 19 41 42 60 0
     149=== Summary ===
     150Ran 60 times
     151Max Latency: 0.066111s
     152Min Latency: 0.045768s
     153Max Jitter : 0.020343s
     154Exponential Moving Average (alpha=0.1): 0.056305s
     155}}}
     156
     157=== CVBS in to RTSP out ===
     158RTSP is an audio/video streaming standard that is commonly supported by various player clients.
     159
     160Notes:
     161 * [https://www.ietf.org/rfc/rfc2326.txt RTSP RFC]
     162 * Note that latency=0 on the receiver will get it to display frames as soon as it receives them. However, depending on network congestion/frame sizes, this could cause video stuttering due to varying latencies.
     163 * These tests were done through a ZyXEL GS1100-24 Switch with 4 ports having active traffic.
     164 * RTSP clients (players) will have a configuration that specifically adds latency by stream buffering to allow for audio/video synchronization which can be on the order of 10 to 1000ms. For gstreamer rtspsrc this is the 'latency' parameter - we set it to 10ms to minimize latency as a value of 0 can confuse gstreamer.
     165 * See the [wiki:Yocto/gstreamer/streaming#rtsp gstreamer/streaming] page for more examples on RTSP streaming
     166
     167The below sections show various gstreamer sinks running on a GW5304 running Yocto 1.8 and an Ubuntu based PC.
     168
     169==== Yocto 1.8 GW5304 ====
     170Receiver latency set to 10, bitrate set to 3mbit/s:
     171* CVBS input to RTSP Server (gst-variable-rtsp-server)
     172* receiver:
     173{{{
     174gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
     175 rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
     176}}}
     177* sender: {{{(imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)}}}
     178{{{
     179#!bash
     180gst-variable-rtsp-server -i /dev/video1 -s imxv4l2videosrc -r 0 -b 3000
     181}}}
     182* latency:
     183{{{
     184#!bash
     185gw-latency-tester 9 19 41 42 60 500000
     186=== Summary ===
     187Ran 60 times
     188Max Latency: 0.100264334s
     189Min Latency: 0.057308000s
     190Max Jitter : 0.042956334s
     191Exponential Moving Average (alpha=0.1): 0.092784375s
     192}}}
     193
     194Here is an example of using {{{udpsink}}} instead of the {{{gst-variable-rtsp-server}}} application. Receiver latency set to 10, bitrate set to 3mbit/s:
     195* CVBS input to RTSP Server (gst-variable-rtsp-server)
     196* sender:
     197{{{
     198gst-launch-1.0 imxv4l2videosrc device=/dev/video3 ! \
     199 imxipuvideotransform ! imxvpuenc_h264 bitrate=3000 ! \
     200 rtph264pay ! udpsink host=172.24.10.63 port=9001
     201}}}
     202* receiver:
     203{{{
     204#!bash
     205gst-launch-1.0 udpsrc port=9001 \
     206 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264" ! \
     207 rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
     208}}}
     209* latency:
     210{{{
     211#!bash
     212gw-latency-tester 9 19 41 42 60 500000
     213=== Summary ===
     214Ran 60 times
     215Max Latency: 0.116625333s
     216Min Latency: 0.083219333s
     217Max Jitter : 0.033406000s
     218Exponential Moving Average (alpha=0.1): 0.098617987s
     219}}}
     220
     221Receiver latency set to 100, bitrate set to 3mbit/s:
     222* CVBS input to RTSP Server (gst-variable-rtsp-server)
     223* sender: {{{(imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)}}}
     224{{{
     225gst-variable-rtsp-server -i /dev/video1 -s imxv4l2videosrc -r 0 -b 3000
     226}}}
     227* receiver:
     228{{{
     229#!bash
     230gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=100 ! \
     231 rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
     232}}}
     233* latency:
     234{{{
     235#!bash
     236gw-latency-tester 9 19 41 42 60 500000
     237=== Summary ===
     238Ran 60 times
     239Max Latency: 0.166653334s
     240Min Latency: 0.131223333s
     241Max Jitter : 0.035430001s
     242Exponential Moving Average (alpha=0.1): 0.139983921s
     243}}}
     244
     245==== Ubuntu 14.04 i5 Desktop PC ====
     246Receiver latency set to 10, bitrate set to 3mbit/s:
     247* CVBS input to RTSP Server (gst-variable-rtsp-server)
     248* sender: {{{(imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)}}}
     249{{{
     250#!bash
     251gst-variable-rtsp-server -i /dev/video1 -s imxv4l2videosrc -r 0 -b 3000
     252}}}
     253* receiver:
     254{{{
     255#!bash
     256gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
     257 decodebin ! autovideosink
     258}}}
     259* latency:
     260{{{
     261#!bash
     262gw-latency-tester 9 19 41 42 60 500000
     263=== Summary ===
     264Ran 60 times
     265Max Latency: 0.118248000s
     266Min Latency: 0.056305334s
     267Max Jitter : 0.061942666s
     268Exponential Moving Average (alpha=0.1): 0.081248197s
     269}}}
     270
     271Receiver latency set to 100, bitrate set to 3mbit/s:
     272* CVBS input to RTSP Server (gst-variable-rtsp-server)
     273* sender: {{{(imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)}}}
     274{{{
     275#!bash
     276gst-variable-rtsp-server -i /dev/video1 -s imxv4l2videosrc -r 0 -b 3000
     277}}}
     278* receiver:
     279{{{
     280#!bash
     281gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=100 ! \
     282 decodebin ! autovideosink
     283}}}
     284* latency:
     285{{{
     286#!bash
     287gw-latency-tester 9 19 41 42 60 500000
     288=== Summary ===
     289Ran 60 times
     290Max Latency: 0.167262000s
     291Min Latency: 0.138012667s
     292Max Jitter : 0.029249333s
     293Exponential Moving Average (alpha=0.1): 0.164639100s
     294}}}
     295
     296
     297== HDMI (Digital) In ==
     298This section is for boards using hdmi capture to various outputs. The in-depth results are based off of the !GoPro Hero3 camera.
     299
     300Please note that the camera itself is responsible for a lot of the latency. Here is an example of how various HDMI output devices can vary (using the same output format and capture format):
     301 * HDMI input (yuv422smp - '''Dino-Lite Digital Microscope @ 1280x720p@60''') to HDMI Output
     302{{{
     303#!bash
     304gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
     305}}}
     306{{{
     307#!bash
     308gw-latency-tester 9 19 41 42 60
     309=== Summary ===
     310Ran 60 times
     311Max Latency: 0.116650000s
     312Min Latency: 0.068762333s
     313Max Jitter : 0.047887667s
     314Exponential Moving Average (alpha=0.1): 0.109525081s
     315}}}
     316 * HDMI input (yuv422smp - '''Incredisonic Dash Cam @ 1280x720p@60''') to HDMI Output
     317{{{
     318#!bash
     319gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
     320}}}
     321{{{
     322#!bash
     323gw-latency-tester 9 19 41 42 60
     324=== Summary ===
     325Ran 60 times
     326Max Latency: 0.130328000s
     327Min Latency: 0.085225000s
     328Max Jitter : 0.045103000s
     329Exponential Moving Average (alpha=0.1): 0.117414606s
     330}}}
     331 * HDMI input (yuv422smp - '''!GoPro Hero3 @ 1280x720p@60''') to HDMI Output
     332{{{
     333#!bash
     334gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
     335}}}
     336{{{
     337#!bash
     338gw-latency-tester 9 19 41 42 60
     339=== Summary ===
     340Ran 60 times
     341Max Latency: 0.115457666s
     342Min Latency: 0.095496333s
     343Max Jitter : 0.019961333s
     344Exponential Moving Average (alpha=0.1): 0.096331845s
     345}}}
     346
     347As you can see, camera choice plays a huge factor in latency numbers. In this case, the !GoPro had 20ms less latency than the Incredisonic Dash Cam, and 10ms less latency than the Dino-Lite Digital Microscope.
     348
     349Also note that HDMI sources have variying latencies based on their output formats. For example 1080p@30 on the !GoPro has increased latencies due it its own performance.
     350
     351Here are some !GoPro Hero3 latency numbers at different output modes:
     352 * HDMI input (yuv422smp - !GoPro Hero3 @ '''1080p@60''') to HDMI Output
     353{{{
     354#!bash
     355gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
     356}}}
     357{{{
     358#!bash
     359gw-latency-tester 9 19 41 42 60 0
     360=== Summary ===
     361Ran 60 times
     362Max Latency: 0.198507s
     363Min Latency: 0.167948s
     364Max Jitter : 0.030559s
     365Exponential Moving Average (alpha=0.1): 0.175236s
     366}}}
     367 * HDMI input (yuv422smp - !GoPro Hero3 @ '''960p@60''') to HDMI Output
     368{{{
     369#!bash
     370gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
     371}}}
     372{{{
     373#!bash
     374gw-latency-tester 9 19 41 42 60
     375=== Summary ===
     376Ran 60 times
     377Max Latency: 0.176905000s
     378Min Latency: 0.137760667s
     379Max Jitter : 0.039144333s
     380Exponential Moving Average (alpha=0.1): 0.156957710s
     381}}}
     382 * HDMI input (yuv422smp - !GoPro Hero3 @ '''720p@30''') to HDMI Output
     383{{{
     384#!bash
     385gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
     386}}}
     387{{{
     388#!bash
     389gw-latency-tester 9 19 41 42 60
     390=== Summary ===
     391Ran 60 times
     392Max Latency: 0.138299333s
     393Min Latency: 0.096406667s
     394Max Jitter : 0.041892666s
     395Exponential Moving Average (alpha=0.1): 0.112956985s
     396}}}
     397 * HDMI input (yuv422smp - !GoPro Hero3 @ '''480p@60''') to HDMI Output
     398{{{
     399#!bash
     400gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
     401}}}
     402{{{
     403#!bash
     404gw-latency-tester 9 19 41 42 60
     405=== Summary ===
     406Ran 60 times
     407Max Latency: 0.116750333s
     408Min Latency: 0.093424667s
     409Max Jitter : 0.023325666s
     410Exponential Moving Average (alpha=0.1): 0.101177281s
     411}}}
     412
     413Also note that any resolution larger than 1080x720 (720p/720i) requires striping for any IPU transform that needs to be performed on the IMX (ie colorspace transform for encoding when capturing in yuv422smp mode and output transform when displaying on a device that differs in format).
     414
     415
     416==== HDMI in to HDMI out ====
     417With the !GoPro configured to '''output 1280x720p@60''', pixel format YUV420, the following latencies were found using a GW5400:
     418
     419===== While capturing in yuv422smp: =====
     420
     421* HDMI input ('''yuv422smp''' - !GoPro Hero3) to HDMI Output
     422{{{
     423#!bash
     424gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
     425}}}
     426{{{
     427#!bash
     428gw-latency-tester 9 19 41 42 60
     429=== Summary ===
     430Ran 60 times
     431Max Latency: 0.115457666s
     432Min Latency: 0.095496333s
     433Max Jitter : 0.019961333s
     434Exponential Moving Average (alpha=0.1): 0.096331845s
     435}}}
     436
     437===== While capturing in bt656: =====
     438
     439* HDMI input ('''yuv422bt656''' - !GoPro Hero3) to HDMI Output
     440{{{
     441#!bash
     442gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
     443}}}
     444{{{
     445#!bash
     446gw-latency-tester 9 19 41 42 60 0
     447=== Summary ===
     448Ran 60 times
     449Max Latency: 0.113161667s
     450Min Latency: 0.085511333s
     451Max Jitter : 0.027650334s
     452Exponential Moving Average (alpha=0.1): 0.097953780s
     453}}}
     454
     455==== HDMI in to LVDS out ====
     456===== While capturing in yuv422smp: =====
     457
     458* HDMI input ('''yuv422smp''' - !GoPro Hero3) to LVDS Output
     459{{{
     460#!bash
     461gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb4
     462}}}
     463{{{
     464#!bash
     465gw-latency-tester 9 19 41 42 60 0
     466=== Summary ===
     467Ran 60 times
     468Max Latency: 0.127060667s
     469Min Latency: 0.106143666s
     470Max Jitter : 0.020917001s
     471Exponential Moving Average (alpha=0.1): 0.117725942s
     472}}}
     473
     474===== While capturing in bt656: =====
     475
     476* HDMI input ('''yuv422bt656''' - !GoPro Hero3) to LVDS Output
     477{{{
     478#!bash
     479gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb4
     480}}}
     481{{{
     482#!bash
     483gw-latency-tester 9 19 41 42 60 0
     484=== Summary ===
     485Ran 60 times
     486Max Latency: 0.173456000s
     487Min Latency: 0.120619667s
     488Max Jitter : 0.052836333s
     489Exponential Moving Average (alpha=0.1): 0.131980293s
     490}}}
     491
     492==== HDMI in to CVBS out ====
     493===== While capturing in yuv422smp: =====
     494* HDMI input ('''yuv422smp''' - !GoPro Hero3) to CVBS Output
     495{{{
     496#!bash
     497gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb2
     498}}}
     499{{{
     500#!bash
     501gw-latency-tester 9 19 41 42 60 0
     502=== Summary ===
     503Ran 60 times
     504Max Latency: 0.103408000s
     505Min Latency: 0.091112000s
     506Max Jitter : 0.012296000s
     507Exponential Moving Average (alpha=0.1): 0.092611801s
     508}}}
     509
     510===== While capturing in bt656: =====
     511* HDMI input ('''yuv422bt656''' - !GoPro Hero3) to CVBS Output
     512{{{
     513#!bash
     514gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb2
     515}}}
     516{{{
     517#!bash
     518gw-latency-tester 9 19 41 42 60 0
     519=== Summary ===
     520Ran 60 times
     521Max Latency: 0.114230334s
     522Min Latency: 0.099739667s
     523Max Jitter : 0.014490667s
     524Exponential Moving Average (alpha=0.1): 0.100241924s
     525}}}
     526
     527==== HDMI in to RTSP out ====
     528RTSP is an audio/video streaming standard that is commonly supported by various player clients.
     529
     530Notes:
     531 * [https://www.ietf.org/rfc/rfc2326.txt RTSP RFC]
     532 * Note that latency=0 on the receiver will get it to display frames as soon as it receives them. However, depending on network congestion/frame sizes, this could cause video stuttering due to varying latencies.
     533 * These tests were done through a ZyXEL GS1100-24 Switch with 4 ports having active traffic.
     534 * RTSP clients (players) will have a configuration that specifically adds latency by stream buffering to allow for audio/video synchronization which can be on the order of 10 to 1000ms. For gstreamer rtspsrc this is the 'latency' parameter - we set it to 10ms to minimize latency as a value of 0 can confuse gstreamer.
     535 * See the [wiki:Yocto/gstreamer/streaming#rtsp gstreamer/streaming] page for more examples on RTSP streaming
     536
     537The below sections show various gstreamer sinks running on a GW5304 running Yocto 1.8 and an Ubuntu based PC.
     538
     539===== Yocto 1.8 GW5304 =====
     540Receiver latency set to 10, bitrate set to 5.5mbit/s:
     541* HDMI input (yuv422smp - !GoPro Hero3) to HDMI Output
     542* sender: {{{(imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)}}}
     543{{{
     544#!bash
     545gst-variable-rtsp-server -i /dev/video0 -s imxv4l2videosrc -r 0 -b 5500
     546}}}
     547* receiver:
     548{{{
     549#!bash
     550gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
     551 rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
     552}}}
     553* latency:
     554{{{
     555#!bash
     556gw-latency-tester 9 19 41 42 60
     557=== Summary ===
     558Ran 60 times
     559Max Latency: 0.183257333s
     560Min Latency: 0.149575667s
     561Max Jitter : 0.033681666s
     562Exponential Moving Average (alpha=0.1): 0.160407397s
     563}}}
     564
     565===== Ubuntu 14.04 i5 Desktop PC =====
     566Receiver latency set to 10, bitrate set to 5.5mbit/s:
     567* HDMI input (yuv422smp - GoPro Hero3) to HDMI Output
     568* sender: {{{(imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)}}}
     569{{{
     570#!bash
     571gst-variable-rtsp-server -i /dev/video0 -s imxv4l2videosrc -r 0 -b 5500
     572}}}
     573* receiver:
     574{{{
     575#!bash
     576gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
     577 decodebin ! autovideosink
     578}}}
     579* latency:
     580{{{
     581#!bash
     582gw-latency-tester 9 19 41 42 60
     583=== Summary ===
     584Ran 60 times
     585Max Latency: 0.127060667s
     586Min Latency: 0.106143666s
     587Max Jitter : 0.020917001s
     588Exponential Moving Average (alpha=0.1): 0.117725942s
     589}}}
     590
     591== avc8000nano (miniPCIe capture card) in ==
     592This section is for boards using the avc8000nano miniPCIe capture card to various outputs.
     593
     594Using a 208C Wired Micro Mini Camera with NTSC 720x480@30FPS output captured via the Ventana Analog in ([http://www.analog.com/media/en/technical-documentation/data-sheets/ADV7180.pdf adv7180 SDTV Video Decoder]) with 8bit ITU-R BT656 output to the IMX6 CSI, the following latencies were found using a GW5400:
     595
     596=== avc8000nano in to HDMI out ===
     597* avc8000nano Input to HDMI Output
     598{{{
     599#!bash
     600gst-launch-1.0 imxv4l2videosrc device=/dev/video3 ! imxg2dvideosink framebuffer=/dev/fb0
     601}}}
     602* latency:
     603{{{
     604#!bash
     605gw-latency-tester 9 19 41 42 60 0
     606=== Summary ===
     607Ran 60 times
     608Max Latency: 0.067630s
     609Min Latency: 0.058137s
     610Max Jitter : 0.009493s
     611Exponential Moving Average (alpha=0.1): 0.066845s
     612}}}
     613
     614=== avc8000nano in to LVDS out ===
     615* avc8000nano Input to LVDS Output
     616{{{
     617#!bash
     618gst-launch-1.0 imxv4l2videosrc device=/dev/video3 ! imxg2dvideosink framebuffer=/dev/fb4
     619}}}
     620* latency:
     621{{{
     622#!bash
     623gw-latency-tester 9 19 41 42 60 0
     624=== Summary ===
     625Ran 60 times
     626Max Latency: 0.090251s
     627Min Latency: 0.059275s
     628Max Jitter : 0.030976s
     629Exponential Moving Average (alpha=0.1): 0.078264s
     630}}}
     631
     632=== avc8000nano in to CVBS out ===
     633* avc8000nano Input to CVBS Output
     634{{{
     635#!bash
     636gst-launch-1.0 imxv4l2videosrc device=/dev/video3 ! imxg2dvideosink framebuffer=/dev/fb2
     637}}}
     638* latency:
     639{{{
     640#!bash
     641gw-latency-tester 9 19 41 42 60 0
     642=== Summary ===
     643Ran 60 times
     644Max Latency: 0.062595s
     645Min Latency: 0.049624s
     646Max Jitter : 0.012971s
     647Exponential Moving Average (alpha=0.1): 0.055531s
     648}}}
     649
     650=== avc8000nano in to RTSP out ===
     651RTSP is an audio/video streaming standard that is commonly supported by various player clients.
     652
     653Notes:
     654 * [https://www.ietf.org/rfc/rfc2326.txt RTSP RFC]
     655 * Note that latency=0 on the receiver will get it to display frames as soon as it receives them. However, depending on network congestion/frame sizes, this could cause video stuttering due to varying latencies.
     656 * These tests were done through a ZyXEL GS1100-24 Switch with 4 ports having active traffic.
     657 * RTSP clients (players) will have a configuration that specifically adds latency by stream buffering to allow for audio/video synchronization which can be on the order of 10 to 1000ms. For gstreamer rtspsrc this is the 'latency' parameter - we set it to 10ms to minimize latency as a value of 0 can confuse gstreamer.
     658 * See the [wiki:Yocto/gstreamer/streaming#rtsp gstreamer/streaming] page for more examples on RTSP streaming
     659
     660The below sections show various gstreamer sinks running on a GW5304 running Yocto 1.8 and an Ubuntu based PC.
     661
     662==== Yocto 1.8 GW5304 ====
     663Receiver latency set to 10, bitrate set to 3mbit/s:
     664* avc8000nano input to RTSP Server (gst-variable-rtsp-server)
     665* sender: {{{(imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)}}}
     666{{{
     667#!bash
     668gst-variable-rtsp-server -i /dev/video3 -s imxv4l2videosrc -r 0 -b 3000
     669}}}
     670* receiver:
     671{{{
     672#!bash
     673gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
     674 rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
     675}}}
     676* latency:
     677{{{
     678#!bash
     679gw-latency-tester 9 19 41 42 60 500000
     680=== Summary ===
     681Ran 60 times
     682Max Latency: 0.116920334s
     683Min Latency: 0.059414667s
     684Max Jitter : 0.057505667s
     685Exponential Moving Average (alpha=0.1): 0.098957740s
     686}}}
     687
     688Receiver latency set to 100, bitrate set to 3mbit/s:
     689* avc8000nano input to RTSP Server (gst-variable-rtsp-server)
     690* sender: {{{(imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)}}}
     691{{{
     692#!bash
     693gst-variable-rtsp-server -i /dev/video3 -s imxv4l2videosrc -r 0 -b 3000
     694}}}
     695* receiver:
     696{{{
     697#!bash
     698gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=100 ! \
     699 rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
     700}}}
     701* latency:
     702{{{
     703#!bash
     704gw-latency-tester 9 19 41 42 60 500000
     705=== Summary ===
     706Ran 60 times
     707Max Latency: 0.166873667s
     708Min Latency: 0.039013333s
     709Max Jitter : 0.127860334s
     710Exponential Moving Average (alpha=0.1): 0.156437334s
     711}}}
     712
     713==== Ubuntu 14.04 i5 Desktop PC ====
     714Receiver latency set to 10, bitrate set to 3mbit/s:
     715* avc8000nano input to RTSP Server (gst-variable-rtsp-server)
     716* sender: {{{(imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)}}}
     717{{{
     718#!bash
     719gst-variable-rtsp-server -i /dev/video3 -s imxv4l2videosrc -r 0 -b 3000
     720}}}
     721* receiver:
     722{{{
     723#!bash
     724gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
     725 decodebin ! autovideosink
     726}}}
     727* latency:
     728{{{
     729#!bash
     730gw-latency-tester 9 19 41 42 60 500000
     731=== Summary ===
     732Ran 60 times
     733Max Latency: 0.129218s
     734Min Latency: 0.067037s
     735Max Jitter : 0.062181s
     736Exponential Moving Average (alpha=0.1): 0.091255s
     737}}}
     738
     739Receiver latency set to 100, bitrate set to 3mbit/s:
     740* avc8000nano input to RTSP Server (gst-variable-rtsp-server)
     741* sender: {{{(imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)}}}
     742{{{
     743#!bash
     744gst-variable-rtsp-server -i /dev/video3 -s imxv4l2videosrc -r 0 -b 3000
     745}}}
     746* receiver:
     747{{{
     748#!bash
     749gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=100 ! \
     750 decodebin ! autovideosink
     751}}}
     752* latency:
     753{{{
     754#!bash
     755gw-latency-tester 9 19 41 42 60 500000
     756=== Summary ===
     757Ran 60 times
     758Max Latency: 0.201775s
     759Min Latency: 0.165823s
     760Max Jitter : 0.035952s
     761Exponential Moving Average (alpha=0.1): 0.177493s
     762}}}