Hailo AI

Hailo is a hardware manufacturer that produces high-performance, low-power AI accelerators and vision processors uniquely designed to accelerate embedded deep learning applications on edge devices.

They typically come in a M.2 or Mini-PCIe form factor that can be plugged into a Gateworks Single Board Computer (SBC).

Quick Start Guide

Required Equipment:

• Gateworks Venice SBC with latest software (testing was done with Ubuntu Jammy and 6.6.8 kernel)
• Hailo AI Hailo-8 M.2 module ​here
• Gateworks GW16148 Mini-PCIe to M.2 M-Key adapter
• Hailo AI Developer Account (free) ​https://hailo.ai/developer-zone/

1. Download Hailo AI Software for Gateworks SBC
2. Login to Hailo 'Developer Zone' and go to software downloads page: ​https://hailo.ai/developer-zone/software-downloads/
3. When selecting a product for downloads, select 'Accelerators' and Sub-package the 'HailoRT' radio button should be checked, Architecture radio button is ARM64 and OS is Linux. As shown in screenshot below:
   1. 
4. Download the two following files:
   1. HailoRT – Ubuntu package (deb) for arm64
   2. Hailo Integration Tool – Ubuntu package (deb) for arm64
5. Choose a way to transfer these two .deb files from the PC to the Gateworks SBC (use either wget from a http server, use SCP, or use a USB flash drive, etc).

1. Boot the Gateworks SBC with access to the serial console over JTAG
2. Come a Ubuntu command prompt:

   root@jammy-venice:~# 
   

Usage

Verify PCIe device is seen:

root@jammy-venice:~# hailortcli scan
Hailo Devices:
[-] Device: 0000:03:00.0

Run HEF File:

• Note HEF files can be downloaded here: ​https://github.com/hailo-ai/hailo_model_zoo/blob/master/docs/PUBLIC_MODELS.rst

  root@jammy-venice:~# hailortcli run my.hef
  

Venice Performance Test:

root@jammy-venice:~# hailo-integration-tool -g test.json #generate test.json file which tells which tests to run, edit the file to set tests to true and false, etc

root@jammy-venice:~# hailo-integration-tool ./test.json 
Performance test started
0000:03:00.0 starting Performance Test
0000:03:00.0 - Tested Network: basic_model, Inference Mode: hw_only, Duration: 60 sec, Measured FPS: 1328
0000:03:00.0 - Tested Network: heavy_load, Inference Mode: hw_only, Duration: 60 sec, Measured FPS: 6
0000:03:00.0 - Tested Network: basic_model, Inference Mode: streaming, Duration: 60 sec, Measured FPS: 1328
0000:03:00.0 - Tested Network: heavy_load, Inference Mode: streaming, Duration: 60 sec, Measured FPS: 6

Performance test passed
JSON summary is ready: integration_tool_reports/2023-Apr-04_23-26-02/summary.json
HTML summary is ready: integration_tool_reports/2023-Apr-04_23-26-02/summary.html
root@jammy-venice:~# cat integration_tool_reports/2023-Apr-04_23-26-02/summary.json
{
  "Integration Tool": {
    "Generic info": {
      "Date of execution": "04-04-2023 23-21-58",
      "Tested modules": [
        {
          "PCIe location": "0000:03:00.0",
          "Product name": "HAILO-8 AI ACC M.2 M KEY MODULE EXT TEMP",
          "Serial number": "HLLWM2A220500021"
        }
      ],
      "Tests summary": [
        {
          "PCIe location": "0000:03:00.0",
          "Pass / Fail": "Pass",
          "Test duration [s]": 244.0,
          "Test name": "Performance test"
        }
      ],
      "User configuration comment": "Insert user-specific comment here",
      "Version tool": "1.13.0",
      "continue_to_next_test_upon_failure": false
    },
    "Tests": [
      {
        "Performance test": {
          "Detected PCIe configuration": {
            "#Lanes": 2,
            "Generation": 1
          },
          "Device ID": "0000:03:00.0",
          "Fail description": "",
          "Pass / Fail": "Pass",
          "Results": [
            {
              "Infer Mode": "hw_only",
              "Measured FPS": 1328,
              "Reference0 FPS": -1,
              "Status": "passed",
              "Tested NN": "basic_model"
            },
            {
              "Infer Mode": "hw_only",
              "Measured FPS": 6,
              "Reference0 FPS": -1,
              "Status": "passed",
              "Tested NN": "heavy_load"
            },
            {
              "Infer Mode": "streaming",
              "Measured FPS": 1328,
              "Reference0 FPS": -1,
              "Status": "passed",
              "Tested NN": "basic_model"
            },
            {
              "Infer Mode": "streaming",
              "Measured FPS": 6,
              "Reference0 FPS": -1,
              "Status": "passed",
              "Tested NN": "heavy_load"
            }
          ]
        }
      }
    ]
  }
}