GW16168 NXP Ara240 DNPU AI Accelerator

The GW16168 NXP Ara240 DNPU AI Accelerator is an M.2 2280 M-Key card by Gateworks for use in the Gateworks single board computers.

For more product information see here: ​https://www.gateworks.com/products/gw16168-m2-ai-accelerator-usa-made/

Note, there is a 8GB and 16GB variant of the GW16168:

• GW16168 - 16GB LPDDR4
• GW16168-SP520 - 8GB LPDDR4

Terminology

The following terminology is used in the Kinara documentation:

• Ara1 / Ara2 processor: An ultra low-power programmable Neural Network processor
• Kinara SDK: Kinara Software Development Kit
• DVNC: Kinara Network Compiler
• DVSim: Kinara Simulator
• DVConvert: Kinara Network Converter
• NNApp: Neural Network Application
• Development Platform: Machine to compile model and run simulator
• Target Platform: Platform which Ara1 / Ara2 processor connects to
• PPA: Power, Performance and Accuracy - metrics reported by the compiler
• SOF: Schedule Optimization Factor - a measure reported by the compiler
• CNN: Convolutional Neural Network - a deep learning model designed to analyze and process grid-like data such as images, videos and sometimes audio and text
• LLM: Large Language Model - an AI model trained on massive amounts of text data to understand, summarize, and generate human-like language
• VLM: Vision Language Model - a multimodal AI that bridges the gap between sight and language. It essentially gives an LLM the ability to "see" by integrating a vision encoder with a language
• sLLM: small Language Model - a lightweight version of an LLM designed to be more efficient, especially for "edge" devices with limited hardware resources

Documentation and Links

Public:

• ​NXP ARA SDK Landing page
• ​https://github.com/nxp-imx/rt-sdk-ara2 - NXP's repo for ara2 runtime SDK v2.04 with dynamic linked binaries
• ​https://github.com/nxp-imx-support/uiodma-driver - Kernel driver GPL-2.0

M.2 Pinout

The GW16168 follows the very generic and standard M.2 M-Key conventions.

• M2.2,4,12,14,16,18,70,72,74: 3.3VDC
• M2.1,2,9,15,21,27,33,39,45,51,57,71: GND
• M2.6: GND

• M2.40: I2C_SCL 1.8
• M2.42: I2C_SDA 1.8
• M2.50: PCIe_PERST#
• M2.52: P_CLKREQ#
• M2.56: I2C_SDA 3.3
• M2.58: I2C_SCL 3.3

• M2.5: PCIE TXN3
• M2.7: PCIE TXP3
• M2.11: PCIE RXN3
• M2.13: PCIE RXP3

• M2.17: PCIE TXN2
• M2.19: PCIE TXP2
• M2.23: PCIE RXN2
• M2.25: PCIE RXP2

• M2.29: PCIE TXN1
• M2.31: PCIE TXP1
• M2.35: PCIE RXN1
• M2.37: PCIE RXP1

• M2.41: PCIE TXN0
• M2.43: PCIE TXP0
• M2.47: PCIE RXN0
• M2.49: PCIE RXP0

• M2.53: PCIEREFCLKN
• M2.55: PCIEREFCLKP

Models

Currently NXP has the following models available for Ara240:

• ​nxp/Qwen2.5-7B-Instruct-Ara240 ~8.17GB - large language model
• ​nxp/Qwen2.5-Coder-1.5B-Ara240 ~1.87GB - code-specialized large language model
• ​nxp/Qwen2.5-VL-7B-Instruct-Ara240 ~12.3GB - multimodal vision-language model
• ​nxp/YOLOv8 ~712MB object detection, segmentation, pose 

NXP Ara240 DNPU AI Accelerator Quick Start

Ara Runtime

NXP has a runtime library for the Ara240 which consists of some statically built libraries as well as a dynamic linked GStreamer plugin.

The Ara runtime includes a couple of Python Wheels. A Python Wheel is a standard built-package format for distributing Python libraries. It is essentially a ZIP-format archive with a .whl extension that contains all the files needed for a package to run immediately after being. It's fairly standard when using Python to run into package version incompatibilities which is why user based Python virtual environments are used.

The Ara runtime provides a complete runtime environment for AI/ML acceleration using the Ara240 NPU on for aarch64. This package includes:

• Runtime libraries for Ara240 NPU integration
• Python bindings (DVAPI) for custom inference applications
• Optimum-Ara framework for LLMs and VLMs
• GStreamer plugin for Real-Time Detection Object Applications
• Helper scripts for monitoring, benchmarking, and model management
• Systemd service for automatic hardware initialization

Installation on a Gateworks board with Ubuntu based OS:

• Download and extract the self-extracting binary from NXP:

  VER=imx-nxp-ara2-2.1.1-063d56c
  wget https://www.nxp.com/lgfiles/NMG/MAD/YOCTO/$VER.bin
  sh $VER.bin
  

• take care of postinst steps
  • miscellaneous

    # create dirs (used for models)
    mkdir -pv /usr/share/{cnn,llm}
    

  • configure swap (necessary if using VLM)

    /usr/bin/enable_swap 2
    

  • install uv package manager for Python virtualization and packaging for local user (which is installed to ~/.local/bin so we create symlinks to /usr/bin)

    apt update && apt install -y curl
    curl -LsSf https://astral.sh/uv/install.sh | sh
    ln -s /root/.local/bin/uv /usr/bin/uv
    ln -s /root/.local/bin/uvx /usr/bin/uvx
    

  • build driver

    apt update && apt install -y build-essential git bc file flex bison
    git clone https://github.com/nxp-imx-support/uiodma-driver
    ( cd uiodma-driver/uiodma; make )
    # install it where the rt service expects to find it (over the top of the non-compatible one)
    cp uiodma-driver/uiodma/uiodma.ko /usr/share/rt-sdk-ara240/driver/
    

  • enable service:

    # enable service
    systemctl enable rt-sdk-ara2.service
    # start service now (unless you reboot)
    systemctl start rt-sdk-ara2.service
    

  • use 'fetch_models' to pre-compiled models for testing via the fetch_models script which will fetch models from HuggingFace.

    # list models available for nxp/ara
    fetch_models --list
    # install YOLOv8
    fetch_models --repo-id nxp/YOLOv8 # 746MB (711MiB)
    

    • the script is a python wrapper that uses uvx and the fetch-models python wheel (/usr/share/python-wheels/fetch_models-1.0.0-py3-none-any.whl) to fetch and install models from HuggingFace HUB
    • the models will be installed in either /usr/share/cnn (Convolutional Neural Network) and /usr/share/llm (Large Language Model)
    • NXP has Ara2 optimized models at ​https://huggingface.co/nxp
    • the script has a hard coded list of models available and where to install them locally. You can use 'python -m zipfile -e /usr/share/python-wheels/fetch_models-1.0.0-py3-none-any.whl ./fetch_models' to see what it's doing

Notable Files:

• /usr/lib/
  • libaraclient_aarch64.so - base library for interfacing with ara2
  • libara_vision_inference.so - inference lib that builds on libaraclient
• /usr/lib/gstreamer-1.0
  • libgstdvInf.so - GStreamer plugin
• /usr/share/rt-sdk-ara240 (symlink to a version independent dir at same location)
  • hw_utils/boot_img - firmware files
  • hw_utils/ddr_config - ddr binaries
  • hw_utils/bins/ - the hw utils for bringup/programming
  • optimum-ara/ - extension of the Hugging Face library that integrates with Ara240 DNPU
  • scripts - various wrappers around the tools etc
  • nnapp - tool for benchmarking models
  • config - various example yaml config files used for proxy/nnapp
  • include/dvapi.py - python bindings to dvapi
  • driver/uiodma.ko - driver (where the setup script expects to find it)
• /usr/share/python-wheels - python wheels for fetch_models and optimum_ara
• /usr/shar/doc/rt-sdk-ara2 - license info
• /usr/include/sdk_ara - headers for C libs
• /usr/bin - various scripts
• /etc/udev/rules.d/99-ara2.rules - udev rule which makes the PCI ID dependent on the systemd service
• /etc/systemd/system/rt-sdk-ara2.service - systemd service that handles the various hw util config
• /etc/rt-sdk-ara240/cnn_config.yaml - config for nnapp
• /etc/rt-sdk-ara240/proxy_config.yam - config for proxy

Notes:

• The 'uv' package manager is a fast all-in-one Python package and project manager written in Rust which makes it easy to work with virtual env's to avoid Python package version clashing which is essential
• on bootup make sure you wait for the console messages indicating the Proxy is launched before using it as it can take a couple of minutes
• the binary tools and libs are all static linked for compatibility
• the GStreamer libs require GStreamer 1.26 or newer and is dynamic linked

Verification steps:

1. show chip_info

   chip_info.sh
   

2. verify service

   # show service status
   systemctl status rt-sdk-ara2.service --no-pager -l 
   # view detailed service logs
   journalctl -u rt-sdk-ara2.service
   # verify proxy is running (critical)
   ps -eaf | grep proxy_ara240
   

Examples:

• Download pre-compiled models for testing:
  • The fetch_models script from the ara2-rt will fetch models from HuggingFace.

    # list models available for nxp/ara
    fetch_models --list
    # install YOLOv8
    fetch_models --repo-id nxp/YOLOv8 # 746MB (711MiB)
    

  • the 'fetch_models' script is a python wrapper that uses uvx and the fetch-models python wheel (/usr/share/python-wheels/fetch_models-1.0.0-py3-none-any.whl) to fetch and install models from HuggingFace HUB
  • the models will be installed in /usr/share/cnn (Convolutional Neural Network) and /usr/share/llm (Large Language Model)
  • NXP has Ara2 optimized models at ​https://huggingface.co/nxp
• Run performance benchmark (uses nnapp)

  run_model_perf.sh
  

  • the 'run_model_perf.sh' script makes it easy to list and show model categories and models and is a wrapper around the nnapp app which has a lot of options and a config file
• monitor real-time NPU metrics including utilization, temperature, DRAM usage and device state (interactively during benchmarking or model execution)

  ara2_metrics.sh
  

GStreamer plugins

The Ara runtime provides an OpenSource GStreamer plugin for detection models:

• ​dvInf

The plugin can sink 32bit pixel samples (ie format=BGRx using 4 bytes per pixel, blue, green, red, and a pading byte as a structural spacer)

The model is specified via the 'model' property. If using yolov8x for example you would specify the path to the yolov8x.dvm

For detection models the element frame data will contain a buffer with number of bytes (32bit) followed by a series of detection structures containing the bounding box, confidence level, and COCO class ID of the object detected.

The units for the bounding box are relative to the models size and will need to be scaled back to your original image size. For example the YOLO models operate on 640x640 pixel data. You can pass something larger in and it will essentially tile but its unclear if there is an advantage of doing that.

While the gstreamer plugin source provided is provided ​here it is included in the Ara runtime pre-compiled for convenience linked against stdlibc (libc.so.6) and libgstreamer-1.0.so.0 and compatible with GStreamer 1.26 or newer.

Install GStreamer:

apt-get update && apt install -y \
   gstreamer1.0-x \
   gstreamer1.0-tools \
   gstreamer1.0-plugins-base \
   gstreamer1.0-plugins-good \
   gstreamer1.0-plugins-bad \
   gstreamer1.0-plugins-ugly \
   gstreamer1.0-libav \
   v4l-utils

• this adds about 500MiB of disk space

Specify Plugin path:

# export now to current shell
export GST_PLUGIN_PATH=/usr/lib/gstreamer-1.0/
# put in .bashrc so it happens for any new bash shell
echo "export GST_PLUGIN_PATH=/usr/lib/gstreamer-1.0/" >> ~/.bashrc

• this tells GStreamer to look for plugins in the non-standard location of the ARA gstreamer plugins

At this point you can inspect the dvInf element:

gst-inspect-1.0 dvInf

Examples:

• gst-launch pipeline prototyping:
  • enabling debug level 6 on dvPost will show the number of object detections in its debug output but if you want to do anything with that data you need to write an application that can decode frame buffers. Still this is useful for prototyping:
    • perform detection on a v4l2 video device like a webcam:

      DEV=/dev/video_webcam
      MODEL=/usr/share/cnn/detection/yolov8n/model.dvm
      GST_DEBUG="dvInf:6" \
      gst-launch-1.0 -v \
        v4l2src device=$DEV ! \
        video/x-raw,width=640,height=480,framerate=30/1 ! \
        videoconvert ! video/x-raw,format=BGRx ! \
        dvInf model=$MODEL orig-width=640 orig-height=480 stream=0 \
              sock=/var/run/proxy.sock use-shm=true shm-path=/dev/shm/ara_shm ! \
        fakesink sync=false | grep Detected
      

      • see wiki:linux/persistent_device_naming#video for details about making video devices have persistent device names
    • perform a detection on an image:

      URI=file:///$PWD/traffic.png
      MODEL=/usr/share/cnn/detection/yolov8n/model.dvm
      GST_DEBUG="dvInf:6" \
      gst-launch-1.0 -v \
        filesrc location=traffic.png ! \
        pngdec ! imagefreeze num-buffers=10 ! \
        videoscale ! videoconvert ! video/x-raw,format=BGRx,width=640,height=480 ! \
        dvInf model=$MODEL orig-width=640 orig-height=480 stream=0 \
              sock=/var/run/proxy.sock use-shm=true shm-path=/dev/shm/ara_shm ! \
        fakesink sync=false | grep Detected
      

For a more complete example see below

eIQ AAF Connector for LLM inference

The eIQ AAF Connector (edge Intelligence Ara Application Framework) is a REST-based server that enables LLM inference on NXP i.MX processors with the ARA-240 DNPU. The API implemented is the de-facto API standard created by OpenAI for ChatGPT. It provides a simple Chat Completions-based HTTP interface for serving models to client applications.

Requirements:

• python 3.13 (we will install in a virtual env)
• uv - used for the user-specific Python virtual environment
• Optimum Ara framework for running Large Language Models (LLMs) and Vision-Language Models (VLMs) on Ara240 (part of rt-sdk)
• OpenCV (dependency of the QwenVL engine)
• Models

Source:

• ​https://github.com/nxp-imx-support/eiq-aaf-connector

For ease of use Gateworks provides a pre-built deb package of eiq-aaf-connector v2.1 built from the NXP IMX Yocto BSP which you can install with:

# fetch
wget https://dev.gateworks.com/ara/eiq-aaf-connector_2.1-r0_arm64.deb
# extract data (but don't install)
dpkg-deb --vextract eiq-aaf-connector_2.1-r0_arm64.deb /
# run the install script
/usr/share/eiq/aaf-connector/install.sh
# fetch LLM models (installed to /usr/share/llm)
fetch_models --repo-id nxp/Qwen2.5-7B-Instruct-Ara240 # 7.7GiB LLM
fetch_models --repo-id nxp/Qwen2.5-Coder-1.5B-Ara240 # 1.67GiB LLM
fetch_models --repo-id nxp/Qwen2.5-VL-7B-Instruct-Ara240 # 12GB VLM

files:

• /usr/share/eiq/aaf-connector/server_config.json (config file)
• /usr/share/eiq/aaf-connector/install.sh (install script)
• /usr/share/python-wheels/eiq_aaf_connector-2.1-py3-none-any.whl (python wheel)

The install script creates a systemd eiq-aaf-connector.service:

# Enable service on boot
systemctl enable eiq-aaf-connector.service
# Start the service now (or reboot)
systemctl start eiq-aaf-connector.service

Notes:

• By default the connector will listen on 127.0.0.1:8000. If you wish the service to be accessible externally set the host to '0.0.0.0' instead:

  sed -i 's|--host 127.0.0.1|--host 0.0.0.0|g' /etc/systemd/system/eiq-aaf-connector.service
  

• the default config file has configuration for all of the above Ara models but they are not 'enabled' by default. You must only enable 1 model at a time and doing so loads the model onto the Ara when the servoce starts. To enable a model change the appropriate 'enabled' property to 'true' in /etc/systemd/system/eiq-aaf-connector.service and restart the service
• it takes several minutes for the service to actually be ready for connections as it must process the models (monitor with 'journalctl -u eiq-aaf-connector.service --no-pager -f' and test that its ready for listening with 'ss -tulpn | grep :8000').
• the connector self-hosts API documentation at ​http://<serverip>:8000/docs (available externally if configured for a host of 0.0.0.0)

Example Usage:

• verify connector running

  # show service status
  systemctl status eiq-aaf-connector.service --no-pager -l
  # view detailed service logs
  journalctl -u eiq-aaf-connector.service
  # verify process exists
  ps -ef | grep aaf-connector
  # verify port open
  ss -tulpn | grep :8000 # show IP:PORT server is listening on
  

• view API docs and interact with server (requires changing the host to '0.0.0.0' in the ExecStart config for /etc/systemd/system/eiq-aaf-connector.service by opening http://<serverip>:8000/docs
• use API via curl/jq

  # make sure curl and jq are installed (jq allows easy interaction with json data)
  apt install -y curl jq 
  # list of models
  curl -X 'GET' \
    'http://127.0.0.1:8000/v1/models' \
    -H 'accept: application/json' | jq
  # get info about a specific model (Qwen2.5-7B-Instruct)
  curl -X 'GET' \
    'http://127.0.0.1:8000/params/Qwen2.5-7B-Instruct' \
    -H 'accept: application/json' | jq
  # send a LLM query
  curl -X POST http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "Qwen2.5-7B-Instruct",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant running on NXP i.MX hardware."},
      {"role": "user", "content": "Explain what an NPU is in one sentence."}
    ],
    "max_tokens": 50
  }' | jq
  

• run connector by hand (useful for troubleshooting or monitoring)

  systemctl stop eiq-aaf-connector.service
  source "/usr/share/eiq/aaf-connector/venv/bin/activate"
  connector --host 0.0.0.0 --port 8000 # will run until stopped
  deactivate
  

[=examples]

Examples

Here are some Ara example applications put together by Gateworks

dvapi stats

This is an ANSI c app that provides an example of using the dvapi to connect to the proxy and obtain NPU endpoint stats such as temperature, clocks and usage. Basically it's a re-implementation of the closed source /usr/share/rt-sdk-ara240/scripts/ara2_metrics_bin/hw_metrics.out.

ara_status.c:

#include <stdio.h>
#include <stdlib.h>
#include "dvapi.h"

int main() {
    dv_session_t *session = NULL;
    dv_endpoint_t *ep_list = NULL;
    int ep_count = 0;
    dv_status_code_t status;
    const char *socket_path = "/run/proxy.sock"; 

    // 1. Establish session
    status = dv_session_create_via_unix_socket(socket_path, &session);
    if (status != DV_SUCCESS) {
        fprintf(stderr, "Failed to connect: %s\n", dv_stringify_status_code(status));
        return 1;
    }

    // 2. Get list of NPU endpoints
    dv_endpoint_get_list(session, &ep_list, &ep_count);

    for (int i = 0; i < ep_count; i++) {
        dv_endpoint_t *ep = &ep_list[i];
        dv_endpoint_statistics_t *stats = NULL;
        int s_count = 0;
        bool is_busy = false;

        // 3. Retrieve status and statistics
        dv_get_endpoint_busyness(session, ep, &is_busy);
        status = dv_endpoint_get_statistics(session, ep, &stats, &s_count);

        if (status == DV_SUCCESS && s_count > 0) {
            // DRAM Calculations (Bytes to GB)
            double used_gb = (double)stats->ep_dram_stats.ep_total_dram_occupancy_size / 1073741824.0;
            double total_gb = (double)stats->ep_dram_stats.ep_total_dram_size / 1073741824.0;
            double dram_pct = (total_gb > 0) ? (used_gb / total_gb) * 100.0 : 0.0;

            // NPU Utilization (Queue occupancy)
            double npu_load = 0.0;
            if (stats->ep_infq_stats && stats->ep_infq_stats->length > 0) {
                npu_load = ((double)stats->ep_infq_stats->occupancy_count / stats->ep_infq_stats->length) * 100.0;
            }

            printf("--- NPU Endpoint %d Statistics ---\n", i);
            printf("Busy State:       %s\n", is_busy ? "TRUE" : "FALSE");
            printf("NPU Utilization:  %.1f%%\n", npu_load);
            printf("Temperature:      %.1f C\n", stats->ep_temp);
            printf("NNP Clock:        %d MHz\n", stats->ep_nnp_clk);
            printf("SBP Clock:        %d MHz\n", stats->ep_sbp_clk);
            printf("DRAM Clock:       %d MHz\n", stats->ep_dram_clk);
            
            // Format: DRAM Usage: 8.2GB/16.0GB (51.3%)
            printf("DRAM Usage:       %.1fGB/%.1fGB (%.1f%%)\n", used_gb, total_gb, dram_pct);
            printf("\n");

            dv_endpoint_free_statistics(stats, s_count);
        }
    }

    // 4. Cleanup
    dv_endpoint_free_group(ep_list);
    dv_session_close(session);
    return 0;
}

Compile:

apt update && apt install build-essentials
gcc ara_status.c -I/usr/include/sdk_ara/ -L/usr/lib/ -laraclient_aarch64 -o ara_status

Execution:

# ./ara_status
--- NPU Endpoint 0 Statistics ---
Busy State:       FALSE
NPU Utilization:  0.0%
Temperature:      56.0 C
NNP Clock:        900 MHz
SBP Clock:        355 MHz
DRAM Clock:       1066 MHz
DRAM Usage:       10.0GB/16.0GB (62.5%)

Image Detection with boxying via Python

Python is incredibly useful for accessing GStreamer and handling the ARA detection frame data and imagemagick provides excellent tools for converting and drawing on images. We use PyGObject which is a Python package that provides bindings for libraries based on GObject Introspection such as GTK, WebKit, and GStreamer. It allows you to use C-based frameworks in python.

Steps:

1. We need to install the C libs for GSTreamer and build utilities:

   apt-get install -y \
     libcairo2-dev \
     libgirepository-2.0-dev \
     python3-dev \
     python3-gst-1.0 \
     cmake pkg-config
   # we are also going to need to install gstreamer and its dev packages
   apt-get install -y \
     libgstreamer1.0-dev \
     libgstreamer-plugins-base1.0-dev \
     libgstreamer-plugins-bad1.0-dev \
     gstreamer1.0-plugins-base \
     gstreamer1.0-plugins-good \
     gstreamer1.0-plugins-bad \
     gstreamer1.0-plugins-ugly \
     gstreamer1.0-libav \
     gstreamer1.0-tools
   

2. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:

   # create a dir for the venv
   mkdir image-detect
   cd image-detect
   # create a venv (.venv)
   uv venv
   # install our scripts dependencies
   uv pip install pygobject
   

3. (optional) fetch some images for detection

   # fetch a coco validation image; it contains a dog on a bench and the dog is at 208,147 to 293,289
   wget http://images.cocodataset.org/val2017/000000546829.jpg -O dog.jpg
   # use ffmpeg to grab a frame from within an MP4
   apt install -y ffmpeg
   ffmpeg -i /usr/share/ara2-vision-examples/sample_videos/video_0.mp4 -f null - # shows how lon git is (time=00:00:15.50)
   ffmpeg -i /usr/share/ara2-vision-examples/sample_videos/video_0.mp4 -ss 00:00:5 -frames:v 1 traffic.png
   

4. fetch the script

   wget https://dev.gateworks.com/ara/examples/image_detect.py
   

5. run the script (image_detect.py <source-image> <destination-image> [model-path])

   uv run image_detect.py dog.jpg coco_detections.jpg
   

   • Note that without shm the pipeline needs to copy the raw image bytes over a local network-style socket connection. By mounting a dedicated memory path to /dev/shm you can eliminate that transfer (zero-copy): dvPre dumps the processed directly into a designated block of system RAM and dvInf uses a pointer to it
   • you would think that if your original image was 1080x1920 and you resized it to the model size of 640x640 that if you tell dvPost the orig-width=1080 orig-height=1920 that it would scale the bounding boxes properly however in practice it seems it does not unless your image has the same aspect ratio of the model. mapping it as above (telling dvPost that the image is 640x640 and scaling ourselves) resolves this
   • images:

Video Detection Webapp via Python

Python is incredibly useful for accessing GStreamer and handling the ARA detection frame data and building webapps. The script using PyGObject which is a Python package that provides bindings for libraries based on GObject Introspection such as GTK, WebKit, and GStreamer. It allows you to use C-based frameworks in python. We need to install the C libs for GStreamer for this

Steps:

1. We need to install the C libs for GStreamer and build utilities:

   apt-get install -y \
     libcairo2-dev \
     libgirepository-2.0-dev \
     python3-dev \
     python3-gst-1.0 \
     cmake pkg-config
   # we are also going to need to install GStreamer and its dev packages
   apt-get install -y \
     libgstreamer1.0-dev \
     libgstreamer-plugins-base1.0-dev \
     libgstreamer-plugins-bad1.0-dev \
     gstreamer1.0-plugins-base \
     gstreamer1.0-plugins-good \
     gstreamer1.0-plugins-bad \
     gstreamer1.0-plugins-ugly \
     gstreamer1.0-libav \
     gstreamer1.0-tools
   

2. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:

   # create a dir for the venv
   mkdir vision-webapp
   cd vision-webapp
   # create a venv (.venv)
   uv venv
   # install our scripts dependencies
   uv pip install pygobject opencv-python-headless flask
   

3. fetch the script

   wget https://dev.gateworks.com/ara/examples/vision-webapp.py
   

4. run the script (vison-webapp.py [--port <portno>] [--camera <camera-dev>] [--mp4 <mp4-dir>]

   uv run vision-webapp.py --camera /dev/video_webcam --mp4 /usr/share/media/sample_videos/
   

   • you can provide a webcam device path to enable streaming from a webcam and/or an mp4 directory to enable processing those. A dropdown will allow you to select the input stream and the model and the browser window will show you detections and statistics

command-line python eIQ chatbot (chat.py)

This is a command-line chatbot written in python using the eIQ AAF Connector

1. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:

   # create a dir for the venv
   mkdir chat
   cd chat
   # create a venv (.venv)
   uv venv
   # install our scripts dependencies
   uv pip install -q requests
   

2. fetch the script

   wget https://dev.gateworks.com/ara/examples/chat.py
   

3. run the script

   uv run chat.py
   

Example session:

--- i.MX LLM Session (Model: Qwen2.5-7B-Instruct) ---
Type 'exit' to stop.

You: Why is the sky blue
AI: The sky appears blue because of a phenomenon called Rayleigh scattering. When sunlight enters the Earth's atmosphere, it collides with molecules and small particles in the air. Sunlight is made up of different colors, each of which has a different wavelength. Blue light has a shorter wavelength and is scattered more than other colors by the gases and particles in the atmosphere. This scattering makes the sky appear blue to our eyes.

During sunrise and sunset, the sky can appear red or orange because the light has to travel through more of the Earth's atmosphere. This longer path means that more blue and green light is scattered out of the beam, leaving the red and orange wavelengths to dominate the light that reaches our eyes.

So, the blue color of the sky is primarily due to the way shorter wavelength light is scattered by the Earth's atmosphere.

--- Stats ---
Time taken: 29.18 seconds
Throughput: 5.04 tokens/sec
-------------

You: exit

Web based python eIQ chatbot (webchat.py)

This is a web based chatbot in python using eIQ AAF Connector

1. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:

   # create a dir for the venv
   mkdir webchat
   cd webchat
   # create a venv (.venv)
   uv venv
   # install our scripts dependencies
   uv pip install -q fastapi psutil uvicorn
   

2. fetch the script

   wget https://dev.gateworks.com/ara/examples/webchat.py
   

3. run the script

   uv run webchat.py
   

4. Open a web browser to your boards IP address port 8080: ​http://<ipaddr>:8080

Notes:

• By default this will listen for HTTP requests on port 8080
• On startup it will enable the model (specified in the script) and restart the eIQ server if needed - waiting for the model to load may take several minutes

Web based python VLM eIQ example (webvlm.py)

The eIQ AAF Connector can be used to analyze video and images.

Here is an example of a headless web-app based off NXP's ​vlm-edge-studio example using:

• Qwen2.5-VL-7B-Instruct-Ara240
• eIQ AAF Connector

Requirements:

• Ara runtime
• eIQ AAF Connector
• Qwen2.5-VL-7B-Instruct-Ara240 model

Steps:

1. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:

   # create a dir for the venv
   mkdir webvlm
   cd webvlm
   # create a venv (.venv)
   uv venv
   # install our scripts dependencies
   uv pip install -q httpx uvicorn fastapi argparse
   

2. fetch the script

   wget https://dev.gateworks.com/ara/examples/webvlm.py
   

3. run the script

   uv run webvlm.py
   

4. Open a web browser to your boards IP address port 8080: ​http://<ipaddr>:8080

Notes:

• By default this will listen for HTTP requests on port 8080
• On startup it will enable the model (specified in the script) and restart the eIQ server if needed - waiting for the model to load may take several minutes

Troubleshooting

Please note software support should be routed through NXP, who produces the Ara240 DNPU Chip and Software SDK.

​https://community.nxp.com