Latency

Latency is the term used to describe how long it takes for video from the source side to get to the sink side. For example, if a video system was described as having a latency of 1 second, that would mean that it takes 1 second for the video to get from the capture (or reading a file) side to the actual display.

This section will attempt to describe some real world latencies that we have seen using gstreamer and our product. Because latency is highly dependent on the different 'knobs' that you can turn, each section will be a very specific test setup. This is important, as you cannot reliably compare latencies when you start to introduce/remove certain feature, video capture ports/types, sink types etc.

The below values differ in capture/display device+driver in the following ways that affect latency

• Frames Per Second (FPS)
• Interlace vs. Progressive
• Capture device Output format (RAW vs JPEG)
• Capture device's inherent latency
• Display device's inherent latency

Please note, the below values are not comparisons, but rather completely isolated and individual examples that happen to be categorized based on input types. This is because latency is affected by several things, some of which aren't always applicable to the scenario demonstrated.

To see prior numbers, please see this older page revision.

Measuring Latency

We've come up with a synchronous method of testing total system latency. We wrote ​a program that uses the on-board gpio's to control an ambient light sensor (​Vishay Semiconductor Opto Division TEPT5700) and a bright blue LED. A camera would be pointed to the LED while a GStreamer pipeline would pipe it to some sink. Finally, the ​program controlling the LED and light sensor turns the LED on, measures how long it took the ambient light sensor to detect it, and repeat several times. If you would like to repeat our findings, please build the ​gw-latency-tester program and use in conjunction with the following hardware:

• ​Vishay Semiconductor Opto Division TEPT5700
• Bright Blue or White LED

Pinout to the DIO connector is as follows:

DIO0 - Power to the ambient light sensor (Collector Side), tie other end to DIO3
DIO1 - Power to the LED, tie other end to GND
DIO2 - [OPTIONAL] Status LED, tie other end to GND
DIO3 - Receiver end of ambient light sensor (Emitter Side), tie other end to DIO0
GND  -

Usage is as follows:

root@ventana:~# ./gw-latency-tester
gw-latency-tester <pwr_dio emit_dio led_dio recv_dio> [<count>] [<udelay>] [<alpha>]

   DIO:  0  1  2  3
         ----------
GW54xx:  9 19 41 42
GW53xx: 16 19 17 20
GW52xx: 16 19 17 20
GW51xx: 16 19 17 18
GW551x: Unsupported
GW552x: 16 19 17 20

The below results will show how the program was run, so please refer to them for examples.

In general (unless otherwise specified), we use the following displays configured in the following ways:

• HDMI: Hanns-G HL273
  • Resolution: 1920x1080p-60
  • Manufacturer specified latency: 2ms
• LVDS: HannStar HSD100PXN1
  • Resolution: 1024x768p-60
  • Manufacturer specified rise time latency: 3-6ms
  • Manufacturer specified fall time latency: 9-18ms
• CVBS: Generic 5" inch TFT LCD Monitor (AC-530DB)
  • Resolution: D:720x480i-60

Accounting for Latency

When we say a system has a latency of 66ms, this number only accounts from end to end devices. However, if we know certain device latencies (such as a displays), we can subtract that off and get fine-tuned numbers.

Using the measurement method described above, calculating the circuitry involved RC time constants, LED Diode rise/fall times, and the response time of the ambient light sensor, we get a "circuitry latency" of: (sensor fall time response) + (led rise time response) == (2us + 13us) + (200ns + 13us) = 28.200us

Because of DIO levels, we'll round that number down to 28us. Plus add in latency from an input (let's assume a camera has exactly 1framerate of latency at 30fps) at 33ms, and finally add in the display latency (we'll assume 4ms). Adding these together produces a total of: 28us + 33ms + 4ms = 37.28ms of latency. We can now say that latency inherent to our product is 66ms - 37.28ms = 28.72ms.

Latency Summary Table

Here is a summary table of the below results.

Note that customers have reported down to 4ms latency on 1080p30 HDMI input to 1080p30 HDMI output eliminating the source (camera) latency. Note the GoPro has been reported to have 60-100ms of latency itself, however, this is unverified by Gateworks.

Loopback:

Type	Capture	Display	End-To-End Latency
Analog	CVBS(NTSC:480i@30fps) imxv4l2videosrc(IPU)	CVBS monitor imxg2dvideosink(GPU)	56ms
Analog	CVBS(NTSC:480i@30fps) imxv4l2videosrc(IPU)	HDMI monitor imxg2dvideosink(GPU)	66ms
Analog	CVBS(NTSC:480i@30fps) imxv4l2videosrc(IPU)	LVDS monitor imxg2dvideosink(GPU)	76ms
Analog	PCI-AVC8000(NTSC:480i@30fps) v4l2src	CVBS monitor imxg2dvideosink(GPU)	55ms
Analog	PCI-AVC8000(NTSC:480i@30fps) v4l2src	HDMI monitor imxg2dvideosink(GPU)	67ms
Analog	PCI-AVC8000(NTSC:480i@30fps) v4l2src	LVDS monitor imxg2dvideosink(GPU)	78ms
Digital	HDMI(GoPro:720p@60fps) imxv4l2videosrc(IPU)	CVBS monitor imxg2dvideosink(GPU)	92ms
Digital	HDMI(GoPro:720p@60fps) imxv4l2videosrc(IPU)	HDMI monitor imxg2dvideosink(GPU)	96ms
Digital	HDMI(GoPro:720p@60fps) imxv4l2videosrc(IPU)	LVDS monitor imxg2dvideosink(GPU)	117ms
Digital	HDMI(GoPro:480p@60) imxv4l2videosrc(IPU)	HDMI monitor imxg2dvideosink(GPU)	101ms
Digital	HDMI(GoPro:720p@30) imxv4l2videosrc(IPU)	HDMI monitor imxg2dvideosink(GPU)	113ms
Digital	HDMI(GoPro:960p@60) imxv4l2videosrc(IPU)	HDMI monitor imxg2dvideosink(GPU)	157ms
Digital	HDMI(GoPro:1080p@60) imxv4l2videosrc(IPU)	HDMI monitor imxg2dvideosink(GPU)	175ms

RTSP Streaming:

Capture	Transform	Encode(VPU)	Stream	Decode(VPU)	Display	End-To-End Latency
CVBS(NTSC:480i@30fps) yuv422smp imxv4l2videosrc(IPU)	imxipuvideotransform(IPU)	imxvpuenc_h264 3mbps CBR	RTSP latency=10ms	imxvpudec (h264)	HDMI monitor imxg2dvideosink(GPU)	98ms
PCI-AVC8000(NTSC:480i@30fps) yuv422smp imxv4l2videosrc(IPU)	imxipuvideotransform(IPU)	imxvpuenc_h264 3mbps CBR	RTSP latency=10ms	imxvpudec (h264)	HDMI monitor imxg2dvideosink(GPU)	98ms
HDMI(GoPro:720p@60fps) yuv422smp imxv4l2videosrc(IPU)	imxipuvideotransform(IPU)	imxvpuenc_h264 5.5mbps CBR	RTSP latency=10ms	imxvpudec (h264)	HDMI monitor imxg2dvideosink(GPU)	160ms

Notes:

1. imxipuvideosink and imxg2dvideosink have the same latencies.
2. LVDS display requires a transform from YUV to RGB colorspace.
3. HDMI capture using yuv422smp requires a transform before encoding while yuv422bt656 does not.
4. IPU transforms with a width or height greater than 1024 (720p/720i/1080p/1080i) require striping equating to two transforms

CVBS (Analog) In

Using a 208C Wired Micro Mini Camera with NTSC 720x480@30FPS output captured via the Ventana Analog in (​adv7180 SDTV Video Decoder) with 8bit ITU-R BT656 output to the IMX6 CSI, the following latencies were found using a GW5400:

CVBS in to HDMI out

• CVBS Input to HDMI Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video1 ! imxg2dvideosink framebuffer=/dev/fb0
  

  gw-latency-tester 9 19 41 42 60 0
  === Summary ===
  Ran 60 times
  Max Latency: 0.082425s
  Min Latency: 0.052815s
  Max Jitter : 0.029610s
  Exponential Moving Average (alpha=0.1): 0.065967s
  

CVBS in to LVDS out

• CVBS Input to LVDS Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video1 ! imxg2dvideosink framebuffer=/dev/fb4
  

  gw-latency-tester 9 19 41 42 60 0
  === Summary ===
  Ran 60 times
  Max Latency: 0.082155s
  Min Latency: 0.055859s
  Max Jitter : 0.026296s
  Exponential Moving Average (alpha=0.1): 0.076021s
  

CVBS in to CVBS out

• CVBS Input to CVBS Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video1 ! imxg2dvideosink framebuffer=/dev/fb2
  

  gw-latency-tester 9 19 41 42 60 0
  === Summary ===
  Ran 60 times
  Max Latency: 0.066111s
  Min Latency: 0.045768s
  Max Jitter : 0.020343s
  Exponential Moving Average (alpha=0.1): 0.056305s
  

CVBS in to RTSP out

RTSP is an audio/video streaming standard that is commonly supported by various player clients.

Notes:

• ​RTSP RFC
• Note that latency=0 on the receiver will get it to display frames as soon as it receives them. However, depending on network congestion/frame sizes, this could cause video stuttering due to varying latencies.
• These tests were done through a ZyXEL GS1100-24 Switch with 4 ports having active traffic.
• RTSP clients (players) will have a configuration that specifically adds latency by stream buffering to allow for audio/video synchronization which can be on the order of 10 to 1000ms. For gstreamer rtspsrc this is the 'latency' parameter - we set it to 10ms to minimize latency as a value of 0 can confuse gstreamer.
• See the gstreamer/streaming page for more examples on RTSP streaming

The below sections show various gstreamer sinks running on a GW5304 running Yocto 1.8 and an Ubuntu based PC.

Yocto 1.8 GW5304

Receiver latency set to 10, bitrate set to 3mbit/s:

• CVBS input to RTSP Server (gst-variable-rtsp-server)
• receiver:

  gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
   rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
  

• sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)

  gst-variable-rtsp-server -i /dev/video1 -s imxv4l2videosrc -r 0 -b 3000
  

• latency:

  gw-latency-tester 9 19 41 42 60 500000
  === Summary ===
  Ran 60 times
  Max Latency: 0.100264334s
  Min Latency: 0.057308000s
  Max Jitter : 0.042956334s
  Exponential Moving Average (alpha=0.1): 0.092784375s
  

Here is an example of using udpsink instead of the gst-variable-rtsp-server application. Receiver latency set to 10, bitrate set to 3mbit/s:

• CVBS input to RTSP Server (gst-variable-rtsp-server)
• sender:

  gst-launch-1.0 imxv4l2videosrc device=/dev/video3 ! \
   imxipuvideotransform ! imxvpuenc_h264 bitrate=3000 ! \
   rtph264pay ! udpsink host=172.24.10.63 port=9001
  

• receiver:

  gst-launch-1.0 udpsrc port=9001 \
   caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264" ! \
   rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
  

• latency:

  gw-latency-tester 9 19 41 42 60 500000
  === Summary ===
  Ran 60 times
  Max Latency: 0.116625333s
  Min Latency: 0.083219333s
  Max Jitter : 0.033406000s
  Exponential Moving Average (alpha=0.1): 0.098617987s
  

Receiver latency set to 100, bitrate set to 3mbit/s:

• CVBS input to RTSP Server (gst-variable-rtsp-server)
• sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)

  gst-variable-rtsp-server -i /dev/video1 -s imxv4l2videosrc -r 0 -b 3000
  

• receiver:

  gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=100 ! \
   rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
  

• latency:

  gw-latency-tester 9 19 41 42 60 500000
  === Summary ===
  Ran 60 times
  Max Latency: 0.166653334s
  Min Latency: 0.131223333s
  Max Jitter : 0.035430001s
  Exponential Moving Average (alpha=0.1): 0.139983921s
  

Ubuntu 14.04 i5 Desktop PC

Receiver latency set to 10, bitrate set to 3mbit/s:

• CVBS input to RTSP Server (gst-variable-rtsp-server)
• sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)

  gst-variable-rtsp-server -i /dev/video1 -s imxv4l2videosrc -r 0 -b 3000
  

• receiver:

  gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
   decodebin ! autovideosink
  

• latency:

  gw-latency-tester 9 19 41 42 60 500000
  === Summary ===
  Ran 60 times
  Max Latency: 0.118248000s
  Min Latency: 0.056305334s
  Max Jitter : 0.061942666s
  Exponential Moving Average (alpha=0.1): 0.081248197s
  

Receiver latency set to 100, bitrate set to 3mbit/s:

• CVBS input to RTSP Server (gst-variable-rtsp-server)
• sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)

  gst-variable-rtsp-server -i /dev/video1 -s imxv4l2videosrc -r 0 -b 3000
  

• receiver:

  gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=100 ! \
   decodebin ! autovideosink
  

• latency:

  gw-latency-tester 9 19 41 42 60 500000
  === Summary ===
  Ran 60 times
  Max Latency: 0.167262000s
  Min Latency: 0.138012667s
  Max Jitter : 0.029249333s
  Exponential Moving Average (alpha=0.1): 0.164639100s
  

HDMI (Digital) In

This section is for boards using hdmi capture to various outputs. The in-depth results are based off of the GoPro Hero3 camera.

Please note that the camera itself is responsible for a lot of the latency. Here is an example of how various HDMI output devices can vary (using the same output format and capture format):

• HDMI input (yuv422smp - Dino-Lite Digital Microscope @ 1280x720p@60) to HDMI Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
  

  gw-latency-tester 9 19 41 42 60
  === Summary ===
  Ran 60 times
  Max Latency: 0.116650000s
  Min Latency: 0.068762333s
  Max Jitter : 0.047887667s
  Exponential Moving Average (alpha=0.1): 0.109525081s
  

• HDMI input (yuv422smp - Incredisonic Dash Cam @ 1280x720p@60) to HDMI Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
  

  gw-latency-tester 9 19 41 42 60
  === Summary ===
  Ran 60 times
  Max Latency: 0.130328000s
  Min Latency: 0.085225000s
  Max Jitter : 0.045103000s
  Exponential Moving Average (alpha=0.1): 0.117414606s
  

• HDMI input (yuv422smp - GoPro Hero3 @ 1280x720p@60) to HDMI Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
  

  gw-latency-tester 9 19 41 42 60
  === Summary ===
  Ran 60 times
  Max Latency: 0.115457666s
  Min Latency: 0.095496333s
  Max Jitter : 0.019961333s
  Exponential Moving Average (alpha=0.1): 0.096331845s
  

As you can see, camera choice plays a huge factor in latency numbers. In this case, the GoPro had 20ms less latency than the Incredisonic Dash Cam, and 10ms less latency than the Dino-Lite Digital Microscope.

Also note that HDMI sources have variying latencies based on their output formats. For example 1080p@30 on the GoPro has increased latencies due it its own performance.

Here are some GoPro Hero3 latency numbers at different output modes:

• HDMI input (yuv422smp - GoPro Hero3 @ 1080p@60) to HDMI Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
  

  gw-latency-tester 9 19 41 42 60 0
  === Summary ===
  Ran 60 times
  Max Latency: 0.198507s
  Min Latency: 0.167948s
  Max Jitter : 0.030559s
  Exponential Moving Average (alpha=0.1): 0.175236s
  

• HDMI input (yuv422smp - GoPro Hero3 @ 960p@60) to HDMI Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
  

  gw-latency-tester 9 19 41 42 60
  === Summary ===
  Ran 60 times
  Max Latency: 0.176905000s
  Min Latency: 0.137760667s
  Max Jitter : 0.039144333s
  Exponential Moving Average (alpha=0.1): 0.156957710s
  

• HDMI input (yuv422smp - GoPro Hero3 @ 720p@30) to HDMI Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
  

  gw-latency-tester 9 19 41 42 60
  === Summary ===
  Ran 60 times
  Max Latency: 0.138299333s
  Min Latency: 0.096406667s
  Max Jitter : 0.041892666s
  Exponential Moving Average (alpha=0.1): 0.112956985s
  

• HDMI input (yuv422smp - GoPro Hero3 @ 480p@60) to HDMI Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
  

  gw-latency-tester 9 19 41 42 60
  === Summary ===
  Ran 60 times
  Max Latency: 0.116750333s
  Min Latency: 0.093424667s
  Max Jitter : 0.023325666s
  Exponential Moving Average (alpha=0.1): 0.101177281s
  

Also note that any resolution larger than 1080x720 (720p/720i) requires striping for any IPU transform that needs to be performed on the IMX (ie colorspace transform for encoding when capturing in yuv422smp mode and output transform when displaying on a device that differs in format).

HDMI in to HDMI out

With the GoPro configured to output 1280x720p@60, pixel format YUV420, the following latencies were found using a GW5400:

While capturing in yuv422smp:

• HDMI input (yuv422smp - GoPro Hero3) to HDMI Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
  

  gw-latency-tester 9 19 41 42 60
  === Summary ===
  Ran 60 times
  Max Latency: 0.115457666s
  Min Latency: 0.095496333s
  Max Jitter : 0.019961333s
  Exponential Moving Average (alpha=0.1): 0.096331845s
  

While capturing in bt656:

• HDMI input (yuv422bt656 - GoPro Hero3) to HDMI Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0
  

  gw-latency-tester 9 19 41 42 60 0
  === Summary ===
  Ran 60 times
  Max Latency: 0.113161667s
  Min Latency: 0.085511333s
  Max Jitter : 0.027650334s
  Exponential Moving Average (alpha=0.1): 0.097953780s
  

HDMI in to LVDS out

While capturing in yuv422smp:

• HDMI input (yuv422smp - GoPro Hero3) to LVDS Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb4
  

  gw-latency-tester 9 19 41 42 60 0
  === Summary ===
  Ran 60 times
  Max Latency: 0.127060667s
  Min Latency: 0.106143666s
  Max Jitter : 0.020917001s
  Exponential Moving Average (alpha=0.1): 0.117725942s
  

While capturing in bt656:

• HDMI input (yuv422bt656 - GoPro Hero3) to LVDS Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb4
  

  gw-latency-tester 9 19 41 42 60 0
  === Summary ===
  Ran 60 times
  Max Latency: 0.173456000s
  Min Latency: 0.120619667s
  Max Jitter : 0.052836333s
  Exponential Moving Average (alpha=0.1): 0.131980293s
  

HDMI in to CVBS out

While capturing in yuv422smp:

• HDMI input (yuv422smp - GoPro Hero3) to CVBS Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb2
  

  gw-latency-tester 9 19 41 42 60 0
  === Summary ===
  Ran 60 times
  Max Latency: 0.103408000s
  Min Latency: 0.091112000s
  Max Jitter : 0.012296000s
  Exponential Moving Average (alpha=0.1): 0.092611801s
  

While capturing in bt656:

• HDMI input (yuv422bt656 - GoPro Hero3) to CVBS Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb2
  

  gw-latency-tester 9 19 41 42 60 0
  === Summary ===
  Ran 60 times
  Max Latency: 0.114230334s
  Min Latency: 0.099739667s
  Max Jitter : 0.014490667s
  Exponential Moving Average (alpha=0.1): 0.100241924s
  

HDMI in to RTSP out

RTSP is an audio/video streaming standard that is commonly supported by various player clients.

Notes:

• ​RTSP RFC
• Note that latency=0 on the receiver will get it to display frames as soon as it receives them. However, depending on network congestion/frame sizes, this could cause video stuttering due to varying latencies.
• These tests were done through a ZyXEL GS1100-24 Switch with 4 ports having active traffic.
• RTSP clients (players) will have a configuration that specifically adds latency by stream buffering to allow for audio/video synchronization which can be on the order of 10 to 1000ms. For gstreamer rtspsrc this is the 'latency' parameter - we set it to 10ms to minimize latency as a value of 0 can confuse gstreamer.
• See the gstreamer/streaming page for more examples on RTSP streaming

The below sections show various gstreamer sinks running on a GW5304 running Yocto 1.8 and an Ubuntu based PC.

Yocto 1.8 GW5304

Receiver latency set to 10, bitrate set to 5.5mbit/s:

• HDMI input (yuv422smp - GoPro Hero3) to HDMI Output
• sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)

  gst-variable-rtsp-server -i /dev/video0 -s imxv4l2videosrc -r 0 -b 5500
  

• receiver:

  gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
   rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
  

• latency:

  gw-latency-tester 9 19 41 42 60
  === Summary ===
  Ran 60 times
  Max Latency: 0.183257333s
  Min Latency: 0.149575667s
  Max Jitter : 0.033681666s
  Exponential Moving Average (alpha=0.1): 0.160407397s
  

Ubuntu 14.04 i5 Desktop PC

Receiver latency set to 10, bitrate set to 5.5mbit/s:

• HDMI input (yuv422smp - GoPro Hero3) to HDMI Output
• sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)

  gst-variable-rtsp-server -i /dev/video0 -s imxv4l2videosrc -r 0 -b 5500
  

• receiver:

  gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
   decodebin ! autovideosink
  

• latency:

  gw-latency-tester 9 19 41 42 60
  === Summary ===
  Ran 60 times
  Max Latency: 0.127060667s
  Min Latency: 0.106143666s
  Max Jitter : 0.020917001s
  Exponential Moving Average (alpha=0.1): 0.117725942s
  

avc8000nano (miniPCIe capture card) in

This section is for boards using the avc8000nano miniPCIe capture card to various outputs.

Using a 208C Wired Micro Mini Camera with NTSC 720x480@30FPS output captured via the Ventana Analog in (​adv7180 SDTV Video Decoder) with 8bit ITU-R BT656 output to the IMX6 CSI, the following latencies were found using a GW5400:

avc8000nano in to HDMI out

• avc8000nano Input to HDMI Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video3 ! imxg2dvideosink framebuffer=/dev/fb0
  

• latency:

  gw-latency-tester 9 19 41 42 60 0
  === Summary ===
  Ran 60 times
  Max Latency: 0.067630s
  Min Latency: 0.058137s
  Max Jitter : 0.009493s
  Exponential Moving Average (alpha=0.1): 0.066845s
  

avc8000nano in to LVDS out

• avc8000nano Input to LVDS Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video3 ! imxg2dvideosink framebuffer=/dev/fb4
  

• latency:

  gw-latency-tester 9 19 41 42 60 0
  === Summary ===
  Ran 60 times
  Max Latency: 0.090251s
  Min Latency: 0.059275s
  Max Jitter : 0.030976s
  Exponential Moving Average (alpha=0.1): 0.078264s
  

avc8000nano in to CVBS out

• avc8000nano Input to CVBS Output

  gst-launch-1.0 imxv4l2videosrc device=/dev/video3 ! imxg2dvideosink framebuffer=/dev/fb2
  

• latency:

  gw-latency-tester 9 19 41 42 60 0
  === Summary ===
  Ran 60 times
  Max Latency: 0.062595s
  Min Latency: 0.049624s
  Max Jitter : 0.012971s
  Exponential Moving Average (alpha=0.1): 0.055531s
  

avc8000nano in to RTSP out

RTSP is an audio/video streaming standard that is commonly supported by various player clients.

Notes:

• ​RTSP RFC
• Note that latency=0 on the receiver will get it to display frames as soon as it receives them. However, depending on network congestion/frame sizes, this could cause video stuttering due to varying latencies.
• These tests were done through a ZyXEL GS1100-24 Switch with 4 ports having active traffic.
• RTSP clients (players) will have a configuration that specifically adds latency by stream buffering to allow for audio/video synchronization which can be on the order of 10 to 1000ms. For gstreamer rtspsrc this is the 'latency' parameter - we set it to 10ms to minimize latency as a value of 0 can confuse gstreamer.
• See the gstreamer/streaming page for more examples on RTSP streaming

The below sections show various gstreamer sinks running on a GW5304 running Yocto 1.8 and an Ubuntu based PC.

Yocto 1.8 GW5304

Receiver latency set to 10, bitrate set to 3mbit/s:

• avc8000nano input to RTSP Server (gst-variable-rtsp-server)
• sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)

  gst-variable-rtsp-server -i /dev/video3 -s imxv4l2videosrc -r 0 -b 3000
  

• receiver:

  gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
   rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
  

• latency:

  gw-latency-tester 9 19 41 42 60 500000
  === Summary ===
  Ran 60 times
  Max Latency: 0.116920334s
  Min Latency: 0.059414667s
  Max Jitter : 0.057505667s
  Exponential Moving Average (alpha=0.1): 0.098957740s
  

Receiver latency set to 100, bitrate set to 3mbit/s:

• avc8000nano input to RTSP Server (gst-variable-rtsp-server)
• sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)

  gst-variable-rtsp-server -i /dev/video3 -s imxv4l2videosrc -r 0 -b 3000 
  

• receiver:

  gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=100 ! \
   rtph264depay ! h264parse ! imxvpudec ! imxg2dvideosink
  

• latency:

  gw-latency-tester 9 19 41 42 60 500000
  === Summary ===
  Ran 60 times
  Max Latency: 0.166873667s
  Min Latency: 0.039013333s
  Max Jitter : 0.127860334s
  Exponential Moving Average (alpha=0.1): 0.156437334s
  

Ubuntu 14.04 i5 Desktop PC

Receiver latency set to 10, bitrate set to 3mbit/s:

• avc8000nano input to RTSP Server (gst-variable-rtsp-server)
• sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)

  gst-variable-rtsp-server -i /dev/video3 -s imxv4l2videosrc -r 0 -b 3000
  

• receiver:

  gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=10 ! \
   decodebin ! autovideosink
  

• latency:

  gw-latency-tester 9 19 41 42 60 500000
  === Summary ===
  Ran 60 times
  Max Latency: 0.129218s
  Min Latency: 0.067037s
  Max Jitter : 0.062181s
  Exponential Moving Average (alpha=0.1): 0.091255s
  

Receiver latency set to 100, bitrate set to 3mbit/s:

• avc8000nano input to RTSP Server (gst-variable-rtsp-server)
• sender: (imxv4l2videosrc ! imxipuvideotransform ! imxvpuenc_h264 ! rtph264pay pt=96)

  gst-variable-rtsp-server -i /dev/video3 -s imxv4l2videosrc -r 0 -b 3000
  

• receiver:

  gst-launch-1.0 rtspsrc location=rtsp://172.24.10.180:9099/stream latency=100 ! \
   decodebin ! autovideosink
  

• latency:

  gw-latency-tester 9 19 41 42 60 500000
  === Summary ===
  Ran 60 times
  Max Latency: 0.201775s
  Min Latency: 0.165823s
  Max Jitter : 0.035952s
  Exponential Moving Average (alpha=0.1): 0.177493s