wiki:Yocto/gstreamer/latency

Version 2 (modified by Chris Lang, 7 years ago) ( diff )

--

Latency

Latency is the term used to describe how long it takes for video from the source side to get to the sink side. For example, if a video system was described as having a latency of 1 second, that would mean that it takes 1 second for the video to get from the capture (or reading a file) side to the actual display.

This section will attempt to describe some real world latencies that we have seen using gstreamer and our product. Because latency is highly dependent on the different 'knobs' that you can turn, each section will be a very specific test setup. This is important, as you cannot reliably compare latencies when you start to introduce/remove certain feature, video capture ports/types, sink types etc.

The below values differ in capture/display device+driver in the following ways that affect latency

  • Frames Per Second (FPS)
  • Interlace vs. Progressive
  • Capture device Output format (RAW vs JPEG)
  • Capture device's inherent latency
  • Display device's inherent latency

Please note, the below values are not comparisons, but rather completely isolated and individual examples that happen to be categorized based on input types. This is because latency is affected by several things, some of which aren't always applicable to the scenario demonstrated.

To see prior numbers, please see this older page revision.

Measuring Latency

We've come up with a synchronous method of testing total system latency. We wrote a program that uses the on-board gpio's to control an ambient light sensor (Vishay Semiconductor Opto Division TEPT5700) and a bright blue LED. A camera would be pointed to the LED while a GStreamer pipeline would pipe it to some sink. Finally, the program controlling the LED and light sensor turns the LED on, measures how long it took the ambient light sensor to detect it, and repeat several times. If you would like to repeat our findings, please build the gw-latency-tester program and use in conjunction with the following hardware:

Pinout to the DIO connector is as follows:

DIO0 - Power to the ambient light sensor (Collector Side), tie other end to DIO3
DIO1 - Power to the LED, tie other end to GND
DIO2 - [OPTIONAL] Status LED, tie other end to GND
DIO3 - Receiver end of ambient light sensor (Emitter Side), tie other end to DIO0
GND  -

Usage is as follows:

root@ventana:~# ./gw-latency-tester
gw-latency-tester <pwr_dio emit_dio led_dio recv_dio> [<count>] [<udelay>] [<alpha>]

   DIO:  0  1  2  3
         ----------
GW54xx:  9 19 41 42
GW53xx: 16 19 17 20
GW52xx: 16 19 17 20
GW51xx: 16 19 17 18
GW551x: Unsupported
GW552x: 16 19 17 20

The below results will show how the program was run, so please refer to them for examples.

In general (unless otherwise specified), we use the following displays configured in the following ways:

  • HDMI: Hanns-G HL273
    • Resolution: 1920x1080p-60
    • Manufacturer specified latency: 2ms
  • LVDS: HannStar HSD100PXN1
    • Resolution: 1024x768p-60
    • Manufacturer specified rise time latency: 3-6ms
    • Manufacturer specified fall time latency: 9-18ms
  • CVBS: Generic 5" inch TFT LCD Monitor (AC-530DB)
    • Resolution: D:720x480i-60

Accounting for Latency

When we say a system has a latency of 66ms, this number only accounts from end to end devices. However, if we know certain device latencies (such as a displays), we can subtract that off and get fine-tuned numbers.

Using the measurement method described above, calculating the circuitry involved RC time constants, LED Diode rise/fall times, and the response time of the ambient light sensor, we get a "circuitry latency" of: (sensor fall time response) + (led rise time response) == (2us + 13us) + (200ns + 13us) = 28.200us

Because of DIO levels, we'll round that number down to 28us. Plus add in latency from an input (let's assume a camera has exactly 1framerate of latency at 30fps) at 33ms, and finally add in the display latency (we'll assume 4ms). Adding these together produces a total of: 28us + 33ms + 4ms = 37.28ms of latency. We can now say that latency inherent to our product is 66ms - 37.28ms = 28.72ms.

Latency Summary Table

Here is a summary table of the below results.

Note that customers have reported down to 4ms latency on 1080p30 HDMI input to 1080p30 HDMI output eliminating the source (camera) latency. Note the GoPro has been reported to have 60-100ms of latency itself, however, this is unverified by Gateworks.

Loopback:

Type Capture Display End-To-End Latency
Analog CVBS(NTSC:480i@30fps)
imxv4l2videosrc(IPU)
CVBS monitor
imxg2dvideosink(GPU)
56ms
Analog CVBS(NTSC:480i@30fps)
imxv4l2videosrc(IPU)
HDMI monitor
imxg2dvideosink(GPU)
66ms
Analog CVBS(NTSC:480i@30fps)
imxv4l2videosrc(IPU)
LVDS monitor
imxg2dvideosink(GPU)
76ms
Analog PCI-AVC8000(NTSC:480i@30fps)
imxv4l2videosrc(IPU)
CVBS monitor
imxg2dvideosink(GPU)
55ms
Analog PCI-AVC8000(NTSC:480i@30fps)
imxv4l2videosrc(IPU)
HDMI monitor
imxg2dvideosink(GPU)
67ms
Analog PCI-AVC8000(NTSC:480i@30fps)
imxv4l2videosrc(IPU)
LVDS monitor
imxg2dvideosink(GPU)
78ms
Digital HDMI(GoPro:720p@60fps)
imxv4l2videosrc(IPU)
CVBS monitor
imxg2dvideosink(GPU)
92ms
Digital HDMI(GoPro:720p@60fps)
imxv4l2videosrc(IPU)
HDMI monitor
imxg2dvideosink(GPU)
96ms
Digital HDMI(GoPro:720p@60fps)
imxv4l2videosrc(IPU)
LVDS monitor
imxg2dvideosink(GPU)
117ms
Digital HDMI(GoPro:480p@60)
imxv4l2videosrc(IPU)
HDMI monitor
imxg2dvideosink(GPU)
101ms
Digital HDMI(GoPro:720p@30)
imxv4l2videosrc(IPU)
HDMI monitor
imxg2dvideosink(GPU)
113ms
Digital HDMI(GoPro:960p@60)
imxv4l2videosrc(IPU)
HDMI monitor
imxg2dvideosink(GPU)
157ms
Digital HDMI(GoPro:1080p@60)
imxv4l2videosrc(IPU)
HDMI monitor
imxg2dvideosink(GPU)
175ms

RTSP Streaming:

Capture Transform Encode(VPU) Stream Decode(VPU) Display End-To-End Latency
CVBS(NTSC:480i@30fps)
yuv422smp
imxv4l2videosrc(IPU)
imxipuvideotransform(IPU) imxvpuenc_h264
3mbps CBR
RTSP
latency=10ms
imxvpudec
(h264)
HDMI monitor
imxg2dvideosink(GPU)
98ms
PCI-AVC8000(NTSC:480i@30fps)
yuv422smp
imxv4l2videosrc(IPU)
imxipuvideotransform(IPU) imxvpuenc_h264
3mbps CBR
RTSP
latency=10ms
imxvpudec
(h264)
HDMI monitor
imxg2dvideosink(GPU)
98ms
HDMI(GoPro:720p@60fps)
yuv422smp
imxv4l2videosrc(IPU)
imxipuvideotransform(IPU) imxvpuenc_h264
5.5mbps CBR
RTSP
latency=10ms
imxvpudec
(h264)
HDMI monitor
imxg2dvideosink(GPU)
160ms

Notes:

  1. imxipuvideosink and imxg2dvideosink have the same latencies.
  2. LVDS display requires a transform from YUV to RGB colorspace.
  3. HDMI capture using yuv422smp requires a transform before encoding while yuv422bt656 does not.
  4. IPU transforms with a width or height greater than 1024 (720p/720i/1080p/1080i) require striping equating to two transforms

CVBS (Analog) In

Using a 208C Wired Micro Mini Camera with NTSC 720x480@30FPS output captured via the Ventana Analog in (adv7180 SDTV Video Decoder) with 8bit ITU-R BT656 output to the IMX6 CSI, the following latencies were found using a GW5400:

CVBS in to HDMI out

  • CVBS Input to HDMI Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video1 ! imxg2dvideosink framebuffer=/dev/fb0
    
    gw-latency-tester 9 19 41 42 60 0
    === Summary ===
    Ran 60 times
    Max Latency: 0.082425s
    Min Latency: 0.052815s
    Max Jitter : 0.029610s
    Exponential Moving Average (alpha=0.1): 0.065967s
    

CVBS in to LVDS out

  • CVBS Input to LVDS Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video1 ! imxg2dvideosink framebuffer=/dev/fb4
    
    gw-latency-tester 9 19 41 42 60 0
    === Summary ===
    Ran 60 times
    Max Latency: 0.082155s
    Min Latency: 0.055859s
    Max Jitter : 0.026296s
    Exponential Moving Average (alpha=0.1): 0.076021s
    

CVBS in to CVBS out

  • CVBS Input to CVBS Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video1 ! imxg2dvideosink framebuffer=/dev/fb2
    
    gw-latency-tester 9 19 41 42 60 0
    === Summary ===
    Ran 60 times
    Max Latency: 0.066111s
    Min Latency: 0.045768s
    Max Jitter : 0.020343s
    Exponential Moving Average (alpha=0.1): 0.056305s
    

CVBS in to RTSP out

RTSP is an audio/video streaming standard that is commonly supported by various player clients.

Notes:

  • RTSP RFC
  • Note that latency=0 on the receiver will get it to display frames as soon as it receives them. However, depending on network congestion/frame sizes, this could cause video stuttering due to varying latencies.
  • These tests were done through a ZyXEL GS1100-24 Switch with 4 ports having active traffic.
  • RTSP clients (players) will have a configuration that specifically adds latency by stream buffering to allow for audio/video synchronization which can be on the order of 10 to 1000ms. For gstreamer rtspsrc this is the 'latency' parameter - we set it to 10ms to minimize latency as a value of 0 can confuse gstreamer.
  • See the gstreamer/streaming page for more examples on RTSP streaming

The below sections show various gstreamer sinks running on a GW5304 running Yocto 1.8 and an Ubuntu based PC.

Yocto 1.8 GW5304

Receiver latency set to 10, bitrate set to 3mbit/s:

  • CVBS input to RTSP Server (gst-variable-rtsp-server)
  • receiver:
    gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
     rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
    
  • sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)
    gst-variable-rtsp-server -i /dev/video1 -s imxv4l2videosrc -r 0 -b 3000
    
  • latency:
    gw-latency-tester 9 19 41 42 60 500000
    === Summary ===
    Ran 60 times
    Max Latency: 0.100264334s
    Min Latency: 0.057308000s
    Max Jitter : 0.042956334s
    Exponential Moving Average (alpha=0.1): 0.092784375s
    

Here is an example of using udpsink instead of the gst-variable-rtsp-server application. Receiver latency set to 10, bitrate set to 3mbit/s:

  • CVBS input to RTSP Server (gst-variable-rtsp-server)
  • sender:
    gst-launch-1.0 imxv4l2videosrc device=/dev/video3 ! \
     imxipuvideotransform ! imxvpuenc_h264 bitrate=3000 ! \
     rtph264pay ! udpsink host=172.24.10.63 port=9001
    
  • receiver:
    gst-launch-1.0 udpsrc port=9001 \
     caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264" ! \
     rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
    
  • latency:
    gw-latency-tester 9 19 41 42 60 500000
    === Summary ===
    Ran 60 times
    Max Latency: 0.116625333s
    Min Latency: 0.083219333s
    Max Jitter : 0.033406000s
    Exponential Moving Average (alpha=0.1): 0.098617987s
    

Receiver latency set to 100, bitrate set to 3mbit/s:

  • CVBS input to RTSP Server (gst-variable-rtsp-server)
  • sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)
    gst-variable-rtsp-server -i /dev/video1 -s imxv4l2videosrc -r 0 -b 3000
    
  • receiver:
    gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=100 ! \
     rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
    
  • latency:
    gw-latency-tester 9 19 41 42 60 500000
    === Summary ===
    Ran 60 times
    Max Latency: 0.166653334s
    Min Latency: 0.131223333s
    Max Jitter : 0.035430001s
    Exponential Moving Average (alpha=0.1): 0.139983921s
    

Ubuntu 14.04 i5 Desktop PC

Receiver latency set to 10, bitrate set to 3mbit/s:

  • CVBS input to RTSP Server (gst-variable-rtsp-server)
  • sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)
    gst-variable-rtsp-server -i /dev/video1 -s imxv4l2videosrc -r 0 -b 3000
    
  • receiver:
    gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
     decodebin ! autovideosink
    
  • latency:
    gw-latency-tester 9 19 41 42 60 500000
    === Summary ===
    Ran 60 times
    Max Latency: 0.118248000s
    Min Latency: 0.056305334s
    Max Jitter : 0.061942666s
    Exponential Moving Average (alpha=0.1): 0.081248197s
    

Receiver latency set to 100, bitrate set to 3mbit/s:

  • CVBS input to RTSP Server (gst-variable-rtsp-server)
  • sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)
    gst-variable-rtsp-server -i /dev/video1 -s imxv4l2videosrc -r 0 -b 3000
    
  • receiver:
    gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=100 ! \
     decodebin ! autovideosink
    
  • latency:
    gw-latency-tester 9 19 41 42 60 500000
    === Summary ===
    Ran 60 times
    Max Latency: 0.167262000s
    Min Latency: 0.138012667s
    Max Jitter : 0.029249333s
    Exponential Moving Average (alpha=0.1): 0.164639100s
    

HDMI (Digital) In

This section is for boards using hdmi capture to various outputs. The in-depth results are based off of the GoPro Hero3 camera.

Please note that the camera itself is responsible for a lot of the latency. Here is an example of how various HDMI output devices can vary (using the same output format and capture format):

  • HDMI input (yuv422smp - Dino-Lite Digital Microscope @ 1280x720p@60) to HDMI Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
    
    gw-latency-tester 9 19 41 42 60
    === Summary ===
    Ran 60 times
    Max Latency: 0.116650000s
    Min Latency: 0.068762333s
    Max Jitter : 0.047887667s
    Exponential Moving Average (alpha=0.1): 0.109525081s
    
  • HDMI input (yuv422smp - Incredisonic Dash Cam @ 1280x720p@60) to HDMI Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
    
    gw-latency-tester 9 19 41 42 60
    === Summary ===
    Ran 60 times
    Max Latency: 0.130328000s
    Min Latency: 0.085225000s
    Max Jitter : 0.045103000s
    Exponential Moving Average (alpha=0.1): 0.117414606s
    
  • HDMI input (yuv422smp - GoPro Hero3 @ 1280x720p@60) to HDMI Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
    
    gw-latency-tester 9 19 41 42 60
    === Summary ===
    Ran 60 times
    Max Latency: 0.115457666s
    Min Latency: 0.095496333s
    Max Jitter : 0.019961333s
    Exponential Moving Average (alpha=0.1): 0.096331845s
    

As you can see, camera choice plays a huge factor in latency numbers. In this case, the GoPro had 20ms less latency than the Incredisonic Dash Cam, and 10ms less latency than the Dino-Lite Digital Microscope.

Also note that HDMI sources have variying latencies based on their output formats. For example 1080p@30 on the GoPro has increased latencies due it its own performance.

Here are some GoPro Hero3 latency numbers at different output modes:

  • HDMI input (yuv422smp - GoPro Hero3 @ 1080p@60) to HDMI Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
    
    gw-latency-tester 9 19 41 42 60 0
    === Summary ===
    Ran 60 times
    Max Latency: 0.198507s
    Min Latency: 0.167948s
    Max Jitter : 0.030559s
    Exponential Moving Average (alpha=0.1): 0.175236s
    
  • HDMI input (yuv422smp - GoPro Hero3 @ 960p@60) to HDMI Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
    
    gw-latency-tester 9 19 41 42 60
    === Summary ===
    Ran 60 times
    Max Latency: 0.176905000s
    Min Latency: 0.137760667s
    Max Jitter : 0.039144333s
    Exponential Moving Average (alpha=0.1): 0.156957710s
    
  • HDMI input (yuv422smp - GoPro Hero3 @ 720p@30) to HDMI Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
    
    gw-latency-tester 9 19 41 42 60
    === Summary ===
    Ran 60 times
    Max Latency: 0.138299333s
    Min Latency: 0.096406667s
    Max Jitter : 0.041892666s
    Exponential Moving Average (alpha=0.1): 0.112956985s
    
  • HDMI input (yuv422smp - GoPro Hero3 @ 480p@60) to HDMI Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
    
    gw-latency-tester 9 19 41 42 60
    === Summary ===
    Ran 60 times
    Max Latency: 0.116750333s
    Min Latency: 0.093424667s
    Max Jitter : 0.023325666s
    Exponential Moving Average (alpha=0.1): 0.101177281s
    

Also note that any resolution larger than 1080x720 (720p/720i) requires striping for any IPU transform that needs to be performed on the IMX (ie colorspace transform for encoding when capturing in yuv422smp mode and output transform when displaying on a device that differs in format).

HDMI in to HDMI out

With the GoPro configured to output 1280x720p@60, pixel format YUV420, the following latencies were found using a GW5400:

While capturing in yuv422smp:
  • HDMI input (yuv422smp - GoPro Hero3) to HDMI Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
    
    gw-latency-tester 9 19 41 42 60
    === Summary ===
    Ran 60 times
    Max Latency: 0.115457666s
    Min Latency: 0.095496333s
    Max Jitter : 0.019961333s
    Exponential Moving Average (alpha=0.1): 0.096331845s
    
While capturing in bt656:
  • HDMI input (yuv422bt656 - GoPro Hero3) to HDMI Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
    
    gw-latency-tester 9 19 41 42 60 0
    === Summary ===
    Ran 60 times
    Max Latency: 0.113161667s
    Min Latency: 0.085511333s
    Max Jitter : 0.027650334s
    Exponential Moving Average (alpha=0.1): 0.097953780s
    

HDMI in to LVDS out

While capturing in yuv422smp:
  • HDMI input (yuv422smp - GoPro Hero3) to LVDS Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb4
    
    gw-latency-tester 9 19 41 42 60 0
    === Summary ===
    Ran 60 times
    Max Latency: 0.127060667s
    Min Latency: 0.106143666s
    Max Jitter : 0.020917001s
    Exponential Moving Average (alpha=0.1): 0.117725942s
    
While capturing in bt656:
  • HDMI input (yuv422bt656 - GoPro Hero3) to LVDS Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb4
    
    gw-latency-tester 9 19 41 42 60 0
    === Summary ===
    Ran 60 times
    Max Latency: 0.173456000s
    Min Latency: 0.120619667s
    Max Jitter : 0.052836333s
    Exponential Moving Average (alpha=0.1): 0.131980293s
    

HDMI in to CVBS out

While capturing in yuv422smp:
  • HDMI input (yuv422smp - GoPro Hero3) to CVBS Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb2
    
    gw-latency-tester 9 19 41 42 60 0
    === Summary ===
    Ran 60 times
    Max Latency: 0.103408000s
    Min Latency: 0.091112000s
    Max Jitter : 0.012296000s
    Exponential Moving Average (alpha=0.1): 0.092611801s
    
While capturing in bt656:
  • HDMI input (yuv422bt656 - GoPro Hero3) to CVBS Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb2
    
    gw-latency-tester 9 19 41 42 60 0
    === Summary ===
    Ran 60 times
    Max Latency: 0.114230334s
    Min Latency: 0.099739667s
    Max Jitter : 0.014490667s
    Exponential Moving Average (alpha=0.1): 0.100241924s
    

HDMI in to RTSP out

RTSP is an audio/video streaming standard that is commonly supported by various player clients.

Notes:

  • RTSP RFC
  • Note that latency=0 on the receiver will get it to display frames as soon as it receives them. However, depending on network congestion/frame sizes, this could cause video stuttering due to varying latencies.
  • These tests were done through a ZyXEL GS1100-24 Switch with 4 ports having active traffic.
  • RTSP clients (players) will have a configuration that specifically adds latency by stream buffering to allow for audio/video synchronization which can be on the order of 10 to 1000ms. For gstreamer rtspsrc this is the 'latency' parameter - we set it to 10ms to minimize latency as a value of 0 can confuse gstreamer.
  • See the gstreamer/streaming page for more examples on RTSP streaming

The below sections show various gstreamer sinks running on a GW5304 running Yocto 1.8 and an Ubuntu based PC.

Yocto 1.8 GW5304

Receiver latency set to 10, bitrate set to 5.5mbit/s:

  • HDMI input (yuv422smp - GoPro Hero3) to HDMI Output
  • sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)
    gst-variable-rtsp-server -i /dev/video0 -s imxv4l2videosrc -r 0 -b 5500
    
  • receiver:
    gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
     rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
    
  • latency:
    gw-latency-tester 9 19 41 42 60
    === Summary ===
    Ran 60 times
    Max Latency: 0.183257333s
    Min Latency: 0.149575667s
    Max Jitter : 0.033681666s
    Exponential Moving Average (alpha=0.1): 0.160407397s
    
Ubuntu 14.04 i5 Desktop PC

Receiver latency set to 10, bitrate set to 5.5mbit/s:

  • HDMI input (yuv422smp - GoPro Hero3) to HDMI Output
  • sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)
    gst-variable-rtsp-server -i /dev/video0 -s imxv4l2videosrc -r 0 -b 5500
    
  • receiver:
    gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
     decodebin ! autovideosink
    
  • latency:
    gw-latency-tester 9 19 41 42 60
    === Summary ===
    Ran 60 times
    Max Latency: 0.127060667s
    Min Latency: 0.106143666s
    Max Jitter : 0.020917001s
    Exponential Moving Average (alpha=0.1): 0.117725942s
    

avc8000nano (miniPCIe capture card) in

This section is for boards using the avc8000nano miniPCIe capture card to various outputs.

Using a 208C Wired Micro Mini Camera with NTSC 720x480@30FPS output captured via the Ventana Analog in (adv7180 SDTV Video Decoder) with 8bit ITU-R BT656 output to the IMX6 CSI, the following latencies were found using a GW5400:

avc8000nano in to HDMI out

  • avc8000nano Input to HDMI Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video3 ! imxg2dvideosink framebuffer=/dev/fb0
    
  • latency:
    gw-latency-tester 9 19 41 42 60 0
    === Summary ===
    Ran 60 times
    Max Latency: 0.067630s
    Min Latency: 0.058137s
    Max Jitter : 0.009493s
    Exponential Moving Average (alpha=0.1): 0.066845s
    

avc8000nano in to LVDS out

  • avc8000nano Input to LVDS Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video3 ! imxg2dvideosink framebuffer=/dev/fb4
    
  • latency:
    gw-latency-tester 9 19 41 42 60 0
    === Summary ===
    Ran 60 times
    Max Latency: 0.090251s
    Min Latency: 0.059275s
    Max Jitter : 0.030976s
    Exponential Moving Average (alpha=0.1): 0.078264s
    

avc8000nano in to CVBS out

  • avc8000nano Input to CVBS Output
    gst-launch-1.0 imxv4l2videosrc device=/dev/video3 ! imxg2dvideosink framebuffer=/dev/fb2
    
  • latency:
    gw-latency-tester 9 19 41 42 60 0
    === Summary ===
    Ran 60 times
    Max Latency: 0.062595s
    Min Latency: 0.049624s
    Max Jitter : 0.012971s
    Exponential Moving Average (alpha=0.1): 0.055531s
    

avc8000nano in to RTSP out

RTSP is an audio/video streaming standard that is commonly supported by various player clients.

Notes:

  • RTSP RFC
  • Note that latency=0 on the receiver will get it to display frames as soon as it receives them. However, depending on network congestion/frame sizes, this could cause video stuttering due to varying latencies.
  • These tests were done through a ZyXEL GS1100-24 Switch with 4 ports having active traffic.
  • RTSP clients (players) will have a configuration that specifically adds latency by stream buffering to allow for audio/video synchronization which can be on the order of 10 to 1000ms. For gstreamer rtspsrc this is the 'latency' parameter - we set it to 10ms to minimize latency as a value of 0 can confuse gstreamer.
  • See the gstreamer/streaming page for more examples on RTSP streaming

The below sections show various gstreamer sinks running on a GW5304 running Yocto 1.8 and an Ubuntu based PC.

Yocto 1.8 GW5304

Receiver latency set to 10, bitrate set to 3mbit/s:

  • avc8000nano input to RTSP Server (gst-variable-rtsp-server)
  • sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)
    gst-variable-rtsp-server -i /dev/video3 -s imxv4l2videosrc -r 0 -b 3000
    
  • receiver:
    gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
     rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
    
  • latency:
    gw-latency-tester 9 19 41 42 60 500000
    === Summary ===
    Ran 60 times
    Max Latency: 0.116920334s
    Min Latency: 0.059414667s
    Max Jitter : 0.057505667s
    Exponential Moving Average (alpha=0.1): 0.098957740s
    

Receiver latency set to 100, bitrate set to 3mbit/s:

  • avc8000nano input to RTSP Server (gst-variable-rtsp-server)
  • sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)
    gst-variable-rtsp-server -i /dev/video3 -s imxv4l2videosrc -r 0 -b 3000 
    
  • receiver:
    gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=100 ! \
     rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
    
  • latency:
    gw-latency-tester 9 19 41 42 60 500000
    === Summary ===
    Ran 60 times
    Max Latency: 0.166873667s
    Min Latency: 0.039013333s
    Max Jitter : 0.127860334s
    Exponential Moving Average (alpha=0.1): 0.156437334s
    

Ubuntu 14.04 i5 Desktop PC

Receiver latency set to 10, bitrate set to 3mbit/s:

  • avc8000nano input to RTSP Server (gst-variable-rtsp-server)
  • sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)
    gst-variable-rtsp-server -i /dev/video3 -s imxv4l2videosrc -r 0 -b 3000
    
  • receiver:
    gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
     decodebin ! autovideosink
    
  • latency:
    gw-latency-tester 9 19 41 42 60 500000
    === Summary ===
    Ran 60 times
    Max Latency: 0.129218s
    Min Latency: 0.067037s
    Max Jitter : 0.062181s
    Exponential Moving Average (alpha=0.1): 0.091255s
    

Receiver latency set to 100, bitrate set to 3mbit/s:

  • avc8000nano input to RTSP Server (gst-variable-rtsp-server)
  • sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)
    gst-variable-rtsp-server -i /dev/video3 -s imxv4l2videosrc -r 0 -b 3000
    
  • receiver:
    gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=100 ! \
     decodebin ! autovideosink
    
  • latency:
    gw-latency-tester 9 19 41 42 60 500000
    === Summary ===
    Ran 60 times
    Max Latency: 0.201775s
    Min Latency: 0.165823s
    Max Jitter : 0.035952s
    Exponential Moving Average (alpha=0.1): 0.177493s
    
Note: See TracWiki for help on using the wiki.