= Hailo AI Hailo is a hardware manufacturer that produces high-performance, low-power AI accelerators and vision processors uniquely designed to accelerate embedded deep learning applications on edge devices. They typically come in a M.2 or Mini-PCIe form factor that can be plugged into a Gateworks Single Board Computer (SBC). = Quick Start Guide Required Equipment: * Gateworks Venice SBC with latest software (testing was done with Ubuntu Jammy and 6.6.8 kernel) * Hailo AI Hailo-8 M.2 module [https://hailo.ai/products/ai-accelerators/hailo-8-m2-ai-acceleration-module/ here] * Gateworks GW16148 Mini-PCIe to M.2 M-Key adapter * Hailo AI Developer Account (free) [https://hailo.ai/developer-zone/] 1. Download Hailo AI Software for Gateworks SBC 1. Login to Hailo 'Developer Zone' and go to software downloads page: [https://hailo.ai/developer-zone/software-downloads/] 1. When selecting a product for downloads, select 'Accelerators' and Sub-package the 'HailoRT' radio button should be checked, OS is Linux. As shown in screenshot below: 1. [[Image(hailort.png,300px)]] 1. Download the following files: 1. HailoRT – PCIe driver Ubuntu package (deb) * '''not specific to arm64''' 1. HailoRT – Ubuntu package (deb) for arm64 1. Hailo Integration Tool – Ubuntu package (deb) for arm64 (not mandatory) 1. Boot the Gateworks SBC with access to the serial console over JTAG 1. Come a Ubuntu command prompt: {{{ root@jammy-venice:~# }}} 1. Choose a way to transfer these .deb files from the PC to the Gateworks SBC (use either wget from a http server, use SCP, or use a USB flash drive, etc). 1. Install pre-requisites {{{ sudo apt update sudo apt install build-essential }}} 1. Install PCIe driver first {{{ root@jammy-venice:~# sudo dpkg -i hailort-pcie-driver_4.16.0_all.deb (Reading database ... 34758 files and directories currently installed.) Preparing to unpack hailort-pcie-driver_4.16.0_all.deb ... Could not test for SecureBoot, assuming SecureBoot is disabled on this machine. Unpacking hailort-pcie-driver (4.16.0) over (4.16.0) ... Setting up hailort-pcie-driver (4.16.0) ... WARNING: apt does not have a stable CLI interface. Use with caution in scripts. build-essential/jammy,now 12.9ubuntu3 arm64 [installed] Do you wish to use DKMS? [Y/n]: n Please reboot your computer for the installation to take effect. root@jammy-venice:~# }}} 1. Install HailoRT tools second: {{{ root@jammy-venice:~# sudo dpkg -i hailort_4.16.0_arm64.deb Selecting previously unselected package hailort. (Reading database ... 28728 files and directories currently installed.) Preparing to unpack hailort_4.16.0_arm64.deb ... Unpacking hailort (4.16.0) ... Setting up hailort (4.16.0) ... Do you wish to activate hailort service? (required for most pyHailoRT use cases) [y/N]: Stopping hailort.service root@jammy-venice:~# }}} 1. Reboot system 1. Use the lspci command to verify the Hailo card is seen on the PCIe bus: {{{ root@jammy-venice:~# lspci 00:00.0 PCI bridge: Synopsys, Inc. DWC_usb3 / PCIe bridge (rev 01) 01:00.0 PCI bridge: Pericom Semiconductor PI7C9X2G608GP PCIe2 6-Port/8-Lane Packet Switch 02:01.0 PCI bridge: Pericom Semiconductor PI7C9X2G608GP PCIe2 6-Port/8-Lane Packet Switch 02:02.0 PCI bridge: Pericom Semiconductor PI7C9X2G608GP PCIe2 6-Port/8-Lane Packet Switch 02:03.0 PCI bridge: Pericom Semiconductor PI7C9X2G608GP PCIe2 6-Port/8-Lane Packet Switch 02:04.0 PCI bridge: Pericom Semiconductor PI7C9X2G608GP PCIe2 6-Port/8-Lane Packet Switch 42:00.0 Co-processor: Hailo Technologies Ltd. Hailo-8 AI Processor (rev 01) c0:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8057 PCI-E Gigabit Ethernet Controller root@jammy-venice:~# }}} 1. Verify Hailo PCIe device is seen with the hailo rt cli software tool: {{{ root@jammy-venice:~# hailortcli scan Hailo Devices: [-] Device: 0000:03:00.0 }}} == Usage Run HEF File: * Note HEF files can be downloaded here: * [https://github.com/hailo-ai/hailo_model_zoo/blob/master/docs/PUBLIC_MODELS.rst] {{{ root@jammy-venice:~# hailortcli run my.hef }}} Venice Performance Test: {{{ root@jammy-venice:~# hailo-integration-tool -g test.json #generate test.json file which tells which tests to run, edit the file to set tests to true and false, etc root@jammy-venice:~# hailo-integration-tool ./test.json Performance test started 0000:03:00.0 starting Performance Test 0000:03:00.0 - Tested Network: basic_model, Inference Mode: hw_only, Duration: 60 sec, Measured FPS: 1328 0000:03:00.0 - Tested Network: heavy_load, Inference Mode: hw_only, Duration: 60 sec, Measured FPS: 6 0000:03:00.0 - Tested Network: basic_model, Inference Mode: streaming, Duration: 60 sec, Measured FPS: 1328 0000:03:00.0 - Tested Network: heavy_load, Inference Mode: streaming, Duration: 60 sec, Measured FPS: 6 Performance test passed JSON summary is ready: integration_tool_reports/2023-Apr-04_23-26-02/summary.json HTML summary is ready: integration_tool_reports/2023-Apr-04_23-26-02/summary.html root@jammy-venice:~# cat integration_tool_reports/2023-Apr-04_23-26-02/summary.json { "Integration Tool": { "Generic info": { "Date of execution": "04-04-2023 23-21-58", "Tested modules": [ { "PCIe location": "0000:03:00.0", "Product name": "HAILO-8 AI ACC M.2 M KEY MODULE EXT TEMP", "Serial number": "HLLWM2A220500021" } ], "Tests summary": [ { "PCIe location": "0000:03:00.0", "Pass / Fail": "Pass", "Test duration [s]": 244.0, "Test name": "Performance test" } ], "User configuration comment": "Insert user-specific comment here", "Version tool": "1.13.0", "continue_to_next_test_upon_failure": false }, "Tests": [ { "Performance test": { "Detected PCIe configuration": { "#Lanes": 2, "Generation": 1 }, "Device ID": "0000:03:00.0", "Fail description": "", "Pass / Fail": "Pass", "Results": [ { "Infer Mode": "hw_only", "Measured FPS": 1328, "Reference0 FPS": -1, "Status": "passed", "Tested NN": "basic_model" }, { "Infer Mode": "hw_only", "Measured FPS": 6, "Reference0 FPS": -1, "Status": "passed", "Tested NN": "heavy_load" }, { "Infer Mode": "streaming", "Measured FPS": 1328, "Reference0 FPS": -1, "Status": "passed", "Tested NN": "basic_model" }, { "Infer Mode": "streaming", "Measured FPS": 6, "Reference0 FPS": -1, "Status": "passed", "Tested NN": "heavy_load" } ] } } ] } } }}}