Hailo AI

Hailo is a hardware manufacturer that produces high-performance, low-power AI accelerators and vision processors uniquely designed to accelerate embedded deep learning applications on edge devices.

They typically come in a M.2 or Mini-PCIe form factor that can be plugged into a Gateworks Single Board Computer (SBC).

Quick Start Guide

Required Equipment:

• Gateworks Venice SBC with latest software (testing was done with Ubuntu Jammy and 6.6.8 kernel)
• Hailo AI Hailo-8 M.2 module ​here
• Gateworks GW16148 Mini-PCIe to M.2 M-Key adapter
• Hailo AI Developer Account (free) ​https://hailo.ai/developer-zone/

1. Download Hailo AI Software for Gateworks SBC
2. Login to Hailo 'Developer Zone' and go to software downloads page: ​https://hailo.ai/developer-zone/software-downloads/
3. When selecting a product for downloads, select 'Accelerators' and Sub-package the 'HailoRT' radio button should be checked, OS is Linux. As shown in screenshot below:
   1. 
4. Download the following files:
   1. HailoRT – PCIe driver Ubuntu package (deb) * not specific to arm64
   2. HailoRT – Ubuntu package (deb) for arm64
   3. Hailo Integration Tool – Ubuntu package (deb) for arm64 (not mandatory)
5. Boot the Gateworks SBC with access to the serial console over JTAG
6. Come a Ubuntu command prompt:

   root@jammy-venice:~# 
   

7. Choose a way to transfer these .deb files from the PC to the Gateworks SBC (use either wget from a http server, use SCP, or use a USB flash drive, etc).
8. Install pre-requisites

   sudo apt update
   sudo apt install build-essential
   

9. Install PCIe driver first

   root@jammy-venice:~# sudo dpkg -i hailort-pcie-driver_4.16.0_all.deb 
   (Reading database ... 34758 files and directories currently installed.)
   Preparing to unpack hailort-pcie-driver_4.16.0_all.deb ...
   Could not test for SecureBoot, assuming SecureBoot is disabled on this machine.
   Unpacking hailort-pcie-driver (4.16.0) over (4.16.0) ...
   Setting up hailort-pcie-driver (4.16.0) ...
   
   WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
   
   build-essential/jammy,now 12.9ubuntu3 arm64 [installed]
   Do you wish to use DKMS? [Y/n]: n
   Please reboot your computer for the installation to take effect.
   root@jammy-venice:~# 
   
   

10. Install HailoRT tools second:

    root@jammy-venice:~# sudo dpkg -i hailort_4.16.0_arm64.deb 
    Selecting previously unselected package hailort.
    (Reading database ... 28728 files and directories currently installed.)
    Preparing to unpack hailort_4.16.0_arm64.deb ...
    Unpacking hailort (4.16.0) ...
    Setting up hailort (4.16.0) ...
    Do you wish to activate hailort service? (required for most pyHailoRT use cases) [y/N]: 
    Stopping hailort.service
    root@jammy-venice:~# 
    
    

11. Reboot system
12. Use the lspci command to verify the Hailo card is seen on the PCIe bus:

    root@jammy-venice:~# lspci
    00:00.0 PCI bridge: Synopsys, Inc. DWC_usb3 / PCIe bridge (rev 01)
    01:00.0 PCI bridge: Pericom Semiconductor PI7C9X2G608GP PCIe2 6-Port/8-Lane Packet Switch
    02:01.0 PCI bridge: Pericom Semiconductor PI7C9X2G608GP PCIe2 6-Port/8-Lane Packet Switch
    02:02.0 PCI bridge: Pericom Semiconductor PI7C9X2G608GP PCIe2 6-Port/8-Lane Packet Switch
    02:03.0 PCI bridge: Pericom Semiconductor PI7C9X2G608GP PCIe2 6-Port/8-Lane Packet Switch
    02:04.0 PCI bridge: Pericom Semiconductor PI7C9X2G608GP PCIe2 6-Port/8-Lane Packet Switch
    42:00.0 Co-processor: Hailo Technologies Ltd. Hailo-8 AI Processor (rev 01)
    c0:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8057 PCI-E Gigabit Ethernet Controller
    root@jammy-venice:~# 
    

Verify Hailo PCIe device is seen with the hailo rt cli software tool:

root@jammy-venice:~# hailortcli scan
Hailo Devices:
[-] Device: 0000:03:00.0

Usage

Run HEF File:

• Note HEF files can be downloaded here:
  • ​https://github.com/hailo-ai/hailo_model_zoo/blob/master/docs/PUBLIC_MODELS.rst

    root@jammy-venice:~# hailortcli run my.hef
    

Venice Performance Test:

root@jammy-venice:~# hailo-integration-tool -g test.json #generate test.json file which tells which tests to run, edit the file to set tests to true and false, etc

root@jammy-venice:~# hailo-integration-tool ./test.json 
Performance test started
0000:03:00.0 starting Performance Test
0000:03:00.0 - Tested Network: basic_model, Inference Mode: hw_only, Duration: 60 sec, Measured FPS: 1328
0000:03:00.0 - Tested Network: heavy_load, Inference Mode: hw_only, Duration: 60 sec, Measured FPS: 6
0000:03:00.0 - Tested Network: basic_model, Inference Mode: streaming, Duration: 60 sec, Measured FPS: 1328
0000:03:00.0 - Tested Network: heavy_load, Inference Mode: streaming, Duration: 60 sec, Measured FPS: 6

Performance test passed
JSON summary is ready: integration_tool_reports/2023-Apr-04_23-26-02/summary.json
HTML summary is ready: integration_tool_reports/2023-Apr-04_23-26-02/summary.html
root@jammy-venice:~# cat integration_tool_reports/2023-Apr-04_23-26-02/summary.json
{
  "Integration Tool": {
    "Generic info": {
      "Date of execution": "04-04-2023 23-21-58",
      "Tested modules": [
        {
          "PCIe location": "0000:03:00.0",
          "Product name": "HAILO-8 AI ACC M.2 M KEY MODULE EXT TEMP",
          "Serial number": "HLLWM2A220500021"
        }
      ],
      "Tests summary": [
        {
          "PCIe location": "0000:03:00.0",
          "Pass / Fail": "Pass",
          "Test duration [s]": 244.0,
          "Test name": "Performance test"
        }
      ],
      "User configuration comment": "Insert user-specific comment here",
      "Version tool": "1.13.0",
      "continue_to_next_test_upon_failure": false
    },
    "Tests": [
      {
        "Performance test": {
          "Detected PCIe configuration": {
            "#Lanes": 2,
            "Generation": 1
          },
          "Device ID": "0000:03:00.0",
          "Fail description": "",
          "Pass / Fail": "Pass",
          "Results": [
            {
              "Infer Mode": "hw_only",
              "Measured FPS": 1328,
              "Reference0 FPS": -1,
              "Status": "passed",
              "Tested NN": "basic_model"
            },
            {
              "Infer Mode": "hw_only",
              "Measured FPS": 6,
              "Reference0 FPS": -1,
              "Status": "passed",
              "Tested NN": "heavy_load"
            },
            {
              "Infer Mode": "streaming",
              "Measured FPS": 1328,
              "Reference0 FPS": -1,
              "Status": "passed",
              "Tested NN": "basic_model"
            },
            {
              "Infer Mode": "streaming",
              "Measured FPS": 6,
              "Reference0 FPS": -1,
              "Status": "passed",
              "Tested NN": "heavy_load"
            }
          ]
        }
      }
    ]
  }
}