Changes between Version 28 and Version 29 of expansion/gw16168

Timestamp:

08/25/2026 07:01:13 PM (7 hours ago)

Author:

Tim Harvey

Comment:

reorder info

expansion/gw16168

@@ -86 +86 @@
 - [https://huggingface.co/nxp/Qwen2.5-VL-7B-Instruct-Ara240 nxp/Qwen2.5-VL-7B-Instruct-Ara240] ~12.3GB - multimodal vision-language model
 - [https://huggingface.co/nxp/YOLOv8 nxp/YOLOv8] ~712MB object detection, segmentation, pose 
-
-= NXP Ara240 DNPU AI Accelerator Quick Start
-
-[=#ara2-runtime]
-== Ara Runtime
-NXP has a runtime library for the Ara240 which consists of some statically built libraries as well as a dynamic linked GStreamer plugin.
-
-The Ara runtime includes a couple of Python Wheels. A Python Wheel is a standard built-package format for distributing Python libraries. It is essentially a ZIP-format archive with a .whl extension that contains all the files needed for a package to run immediately after being. It's fairly standard when using Python to run into package version incompatibilities which is why user based Python virtual environments are used.
-
-The Ara runtime provides a complete runtime environment for AI/ML acceleration using the Ara240 NPU on for aarch64. This package includes:
- * Runtime libraries for Ara240 NPU integration
- * Python bindings (DVAPI) for custom inference applications
- * Optimum-Ara framework for LLMs and VLMs
- * GStreamer plugin for Real-Time Detection Object Applications
- * Helper scripts for monitoring, benchmarking, and model management
- * Systemd service for automatic hardware initialization
-
-Installation on a Gateworks board with Ubuntu based OS:
- - Download and extract the self-extracting binary from NXP:
-{{{#!bash
-VER=imx-nxp-ara2-2.1.1-063d56c
-wget https://www.nxp.com/lgfiles/NMG/MAD/YOCTO/$VER.bin
-sh $VER.bin
-}}}
- - take care of postinst steps
-  - miscellaneous
-{{{#!bash
-# create dirs (used for models)
-mkdir -pv /usr/share/{cnn,llm}
-}}}
-  - configure swap (necessary if using VLM)
-{{{#!bash
-/usr/bin/enable_swap 2
-}}}
-  - install uv package manager for Python virtualization and packaging for local user (which is installed to ~/.local/bin so we create symlinks to /usr/bin)
-{{{#!bash
-apt update && apt install -y curl
-curl -LsSf https://astral.sh/uv/install.sh | sh
-ln -s /root/.local/bin/uv /usr/bin/uv
-ln -s /root/.local/bin/uvx /usr/bin/uvx
-}}}
-  - build driver
-{{{#!bash
-apt update && apt install -y build-essential git bc file flex bison
-git clone https://github.com/nxp-imx-support/uiodma-driver
-( cd uiodma-driver/uiodma; make )
-# install it where the rt service expects to find it (over the top of the non-compatible one)
-cp uiodma-driver/uiodma/uiodma.ko /usr/share/rt-sdk-ara240/driver/
-}}}
-  - enable service:
-{{{#!bash
-# enable service
-systemctl enable rt-sdk-ara2.service
-# start service now (unless you reboot)
-systemctl start rt-sdk-ara2.service
-}}}
-  - use 'fetch_models' to pre-compiled models for testing via the fetch_models script which will fetch models from !HuggingFace.
-{{{#!bash
-# list models available for nxp/ara
-fetch_models --list
-# install YOLOv8
-fetch_models --repo-id nxp/YOLOv8 # 746MB (711MiB)
-}}}
-   - the script is a python wrapper that uses uvx and the fetch-models python wheel (/usr/share/python-wheels/fetch_models-1.0.0-py3-none-any.whl) to fetch and install models from !HuggingFace HUB
-   - the models will be installed in either /usr/share/cnn (Convolutional Neural Network) and /usr/share/llm (Large Language Model)
-   - NXP has Ara2 optimized models at https://huggingface.co/nxp
-   - the script has a hard coded list of models available and where to install them locally. You can use 'python -m zipfile -e /usr/share/python-wheels/fetch_models-1.0.0-py3-none-any.whl ./fetch_models' to see what it's doing
-
-Notable Files:
- - /usr/lib/
-  - libaraclient_aarch64.so - base library for interfacing with ara2
-  - libara_vision_inference.so - inference lib that builds on libaraclient
- - /usr/lib/gstreamer-1.0
-  - libgstdvInf.so - GStreamer plugin
- - /usr/share/rt-sdk-ara240 (symlink to a version independent dir at same location)
-  - hw_utils/boot_img - firmware files
-  - hw_utils/ddr_config - ddr binaries
-  - hw_utils/bins/ - the hw utils for bringup/programming
-  - optimum-ara/ - extension of the Hugging Face library that integrates with Ara240 DNPU
-  - scripts - various wrappers around the tools etc
-  - nnapp - tool for benchmarking models
-  - config - various example yaml config files used for proxy/nnapp
-  - include/dvapi.py - python bindings to dvapi
-  - driver/uiodma.ko - driver (where the setup script expects to find it)
- - /usr/share/python-wheels - python wheels for fetch_models and optimum_ara
- - /usr/shar/doc/rt-sdk-ara2 - license info
- - /usr/include/sdk_ara - headers for C libs
- - /usr/bin - various scripts
- - /etc/udev/rules.d/99-ara2.rules - udev rule which makes the PCI ID dependent on the systemd service
- - /etc/systemd/system/rt-sdk-ara2.service - systemd service that handles the various hw util config
- - /etc/rt-sdk-ara240/cnn_config.yaml - config for nnapp
- - /etc/rt-sdk-ara240/proxy_config.yam - config for proxy
-
-Notes:
- - The 'uv' package manager is a fast all-in-one Python package and project manager written in Rust which makes it easy to work with virtual env's to avoid Python package version clashing which is essential
- - on bootup make sure you wait for the console messages indicating the Proxy is launched before using it as it can take a couple of minutes
- - the binary tools and libs are all static linked for compatibility
- - the GStreamer libs require GStreamer 1.26 or newer and is dynamic linked
-
-Verification steps:
- 1. show chip_info
-{{{#!bash
-chip_info.sh
-}}}
- 1. verify service
-{{{#!bash
-# show service status
-systemctl status rt-sdk-ara2.service --no-pager -l
-# view detailed service logs for the current boot and follow the journal
-journalctl --boot --follow --unit rt-sdk-ara2.service
-# verify proxy is running (critical)
-ps -eaf | grep proxy_ara240
-}}}
-
-Examples:
- - Download pre-compiled models for testing:
-  - The fetch_models script from the ara2-rt will fetch models from !HuggingFace.
-{{{#!bash
-# list models available for nxp/ara
-fetch_models --list
-# install YOLOv8
-fetch_models --repo-id nxp/YOLOv8 # 746MB (711MiB)
-}}}
-  - the 'fetch_models' script is a python wrapper that uses uvx and the fetch-models python wheel (/usr/share/python-wheels/fetch_models-1.0.0-py3-none-any.whl) to fetch and install models from !HuggingFace HUB
-  - the models will be installed in /usr/share/cnn (Convolutional Neural Network) and /usr/share/llm (Large Language Model)
-  - NXP has Ara2 optimized models at https://huggingface.co/nxp
- - Run performance benchmark (uses nnapp)
-{{{#!bash
-run_model_perf.sh
-}}} 
-  - the 'run_model_perf.sh' script makes it easy to list and show model categories and models and is a wrapper around the nnapp app which has a lot of options and a config file
- - monitor real-time NPU metrics including utilization, temperature, DRAM usage and device state (interactively during benchmarking or model execution)
-{{{#!bash
-ara2_metrics.sh
-}}}
-
-[=#gstreamer]
-=== GStreamer plugins
-The Ara runtime provides an OpenSource GStreamer plugin for detection models:
- - [https://github.com/nxp-imx-support/gstreamer-plugins-ara240 dvInf]
-
-The plugin can sink 32bit pixel samples (ie format=BGRx using 4 bytes per pixel, blue, green, red, and a pading byte as a structural spacer)
-
-The model is specified via the 'model' property. If using yolov8x for example you would specify the path to the yolov8x.dvm
-
-For detection models the element frame data will contain a buffer with number of bytes (32bit) followed by a series of detection structures containing the bounding box, confidence level, and COCO class ID of the object detected.
-
-The units for the bounding box are relative to the models size and will need to be scaled back to your original image size. For example the YOLO models operate on 640x640 pixel data. You can pass something larger in and it will essentially tile but its unclear if there is an advantage of doing that.
-
-While the gstreamer plugin source provided is provided [https://github.com/nxp-imx-support/gstreamer-plugins-ara240 here] it is included in the Ara runtime pre-compiled for convenience linked against stdlibc (libc.so.6) and libgstreamer-1.0.so.0 and compatible with GStreamer 1.26 or newer.
-
-Install GStreamer:
-{{{#!bash
-apt-get update && apt install -y \
-   gstreamer1.0-x \
-   gstreamer1.0-tools \
-   gstreamer1.0-plugins-base \
-   gstreamer1.0-plugins-good \
-   gstreamer1.0-plugins-bad \
-   gstreamer1.0-plugins-ugly \
-   gstreamer1.0-libav \
-   v4l-utils
-}}}
- - this adds about 500MiB of disk space
-
-Specify Plugin path:
-{{{#!bash
-# export now to current shell
-export GST_PLUGIN_PATH=/usr/lib/gstreamer-1.0/
-# put in .bashrc so it happens for any new bash shell
-echo "export GST_PLUGIN_PATH=/usr/lib/gstreamer-1.0/" >> ~/.bashrc
-}}}
- - this tells GStreamer to look for plugins in the non-standard location of the ARA gstreamer plugins
-
-At this point you can inspect the dvInf element:
-{{{#!bash
-gst-inspect-1.0 dvInf
-}}}
-
-Examples:
- * gst-launch pipeline prototyping:
-  - enabling debug level 6 on dvPost will show the number of object detections in its debug output but if you want to do anything with that data you need to write an application that can decode frame buffers. Still this is useful for prototyping:
-   * perform detection on a v4l2 video device like a webcam:
-{{{#!bash
-DEV=/dev/video_webcam
-MODEL=/usr/share/cnn/detection/yolov8n/model.dvm
-GST_DEBUG="dvInf:6" \
-gst-launch-1.0 -v \
-  v4l2src device=$DEV ! \
-  video/x-raw,width=640,height=480,framerate=30/1 ! \
-  videoconvert ! video/x-raw,format=BGRx ! \
-  dvInf model=$MODEL orig-width=640 orig-height=480 stream=0 \
-        sock=/var/run/proxy.sock use-shm=true shm-path=/dev/shm/ara_shm ! \
-  fakesink sync=false | grep Detected
-}}}
-    - see wiki:linux/persistent_device_naming#video for details about making video devices have persistent device names
-   * perform a detection on an image:
-{{{#!bash
-URI=file:///$PWD/traffic.png
-MODEL=/usr/share/cnn/detection/yolov8n/model.dvm
-GST_DEBUG="dvInf:6" \
-gst-launch-1.0 -v \
-  filesrc location=traffic.png ! \
-  pngdec ! imagefreeze num-buffers=10 ! \
-  videoscale ! videoconvert ! video/x-raw,format=BGRx,width=640,height=480 ! \
-  dvInf model=$MODEL orig-width=640 orig-height=480 stream=0 \
-        sock=/var/run/proxy.sock use-shm=true shm-path=/dev/shm/ara_shm ! \
-  fakesink sync=false | grep Detected
-}}}
-
-For a more complete example see below
-
-
-[=#eiq-aaf-connector]
-== eIQ AAF Connector for LLM inference
-The eIQ AAF Connector (edge Intelligence Ara Application Framework)
-is a REST-based server that enables LLM inference on NXP i.MX processors with the ARA-240 DNPU. The API implemented is the de-facto API standard created by OpenAI for ChatGPT. It provides a simple Chat Completions-based HTTP interface for serving models to client applications.
-
-Requirements:
- - python 3.13 (we will install in a virtual env)
- - uv - used for the user-specific Python virtual environment
- - Optimum Ara framework for running Large Language Models (LLMs) and Vision-Language Models (VLMs) on Ara240 (part of rt-sdk)
- - OpenCV (dependency of the QwenVL engine)
- - Models
-
-Source:
- - https://github.com/nxp-imx-support/eiq-aaf-connector
-
-For ease of use Gateworks provides a pre-built deb package of eiq-aaf-connector v2.1 built from the NXP IMX Yocto BSP which you can install with:
-{{{#!bash
-# fetch
-wget https://dev.gateworks.com/ara/eiq-aaf-connector_2.1-r0_arm64.deb
-# extract data (but don't install)
-dpkg-deb --vextract eiq-aaf-connector_2.1-r0_arm64.deb /
-# run the install script
-/usr/share/eiq/aaf-connector/install.sh
-# fetch LLM models (installed to /usr/share/llm)
-fetch_models --repo-id nxp/Qwen2.5-7B-Instruct-Ara240 # 7.7GiB LLM
-fetch_models --repo-id nxp/Qwen2.5-Coder-1.5B-Ara240 # 1.67GiB LLM
-fetch_models --repo-id nxp/Qwen2.5-VL-7B-Instruct-Ara240 # 12GB VLM
-}}}
-
-files:
- - /usr/share/eiq/aaf-connector/install.sh (install script)
- - /usr/share/python-wheels/eiq_aaf_connector-2.1-py3-none-any.whl (python wheel)
- - /usr/share/eiq/aaf-connector/server_config.json (config file)
- - /etc/systemd/system/eiq-aaf-connector.service (created from install script)
-
-The install script creates a systemd eiq-aaf-connector.service:
-{{{#!bash
-# Enable service on boot
-systemctl enable eiq-aaf-connector.service
-# Start the service now (or reboot)
-systemctl start eiq-aaf-connector.service
-}}}
-
-Notes:
- - By default the connector will listen on 127.0.0.1:8000. If you wish the service to be accessible externally set the host to '0.0.0.0' instead:
-{{{#!bash
-sed -i 's|--host 127.0.0.1|--host 0.0.0.0|g' /etc/systemd/system/eiq-aaf-connector.service
-}}}
- - the default config file has configuration for all of the above Ara models but they are not 'enabled' by default. You must only enable 1 model at a time and doing so loads the model onto the Ara when the servoce starts. To enable a model change the appropriate 'enabled' property to 'true' in /etc/systemd/system/eiq-aaf-connector.service and restart the service
- - it takes several minutes for the service to actually be ready for connections as it must process the models (monitor with 'journalctl -u eiq-aaf-connector.service --no-pager -f' and test that its ready for listening with 'ss -tulpn | grep :8000').
- - the connector self-hosts API documentation at http://<serverip>:8000/docs (available externally if configured for a host of 0.0.0.0)
-
-Example Usage:
- - verify connector running
-{{{#!bash
-# show service status
-systemctl status eiq-aaf-connector.service --no-pager -l
-# view detailed service logs for the current boot and follow the journal
-journalctl --boot --follow --unit eiq-aaf-connector.service
-# verify process exists
-ps -ef | grep aaf-connector
-# verify port open
-ss -tulpn | grep :8000 # show IP:PORT server is listening on
-}}}
- - view API docs and interact with server (requires changing the host to '0.0.0.0' in the !ExecStart config for /etc/systemd/system/eiq-aaf-connector.service by opening !http://<serverip>:8000/docs
- - use API via curl/jq
-{{{#!bash
-# make sure curl and jq are installed (jq allows easy interaction with json data)
-apt install -y curl jq
-# list of models
-curl -X 'GET' \
-  'http://127.0.0.1:8000/v1/models' \
-  -H 'accept: application/json' | jq
-# get info about a specific model (Qwen2.5-7B-Instruct)
-curl -X 'GET' \
-  'http://127.0.0.1:8000/params/Qwen2.5-7B-Instruct' \
-  -H 'accept: application/json' | jq
-# send a LLM query
-curl -X POST http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
-  "model": "Qwen2.5-7B-Instruct",
-  "messages": [
-    {"role": "system", "content": "You are a helpful assistant running on NXP i.MX hardware."},
-    {"role": "user", "content": "Explain what an NPU is in one sentence."}
-  ],
-  "max_tokens": 50
-}' | jq
-}}}
- - run connector by hand (useful for troubleshooting or monitoring)
-{{{#!bash
-systemctl stop eiq-aaf-connector.service
-source "/usr/share/eiq/aaf-connector/venv/bin/activate"
-connector --host 0.0.0.0 --port 8000 # will run until stopped
-deactivate
-}}}
-
-
-[=#examples]
-== Examples
-Here are some Ara example applications put together by Gateworks
-
-=== dvapi stats
-This is an ANSI c app that provides an example of using the dvapi to connect to the proxy and obtain NPU endpoint stats such as temperature, clocks and usage. Basically it's a re-implementation of the closed source /usr/share/rt-sdk-ara240/scripts/ara2_metrics_bin/hw_metrics.out.
-
-ara_status.c:
-{{{#!c
-#include <stdio.h>
-#include <stdlib.h>
-#include "dvapi.h"
-
-int main() {
-    dv_session_t *session = NULL;
-    dv_endpoint_t *ep_list = NULL;
-    int ep_count = 0;
-    dv_status_code_t status;
-    const char *socket_path = "/run/proxy.sock"; 
-
-    // 1. Establish session
-    status = dv_session_create_via_unix_socket(socket_path, &session);
-    if (status != DV_SUCCESS) {
-        fprintf(stderr, "Failed to connect: %s\n", dv_stringify_status_code(status));
-        return 1;
-    }
-
-    // 2. Get list of NPU endpoints
-    dv_endpoint_get_list(session, &ep_list, &ep_count);
-
-    for (int i = 0; i < ep_count; i++) {
-        dv_endpoint_t *ep = &ep_list[i];
-        dv_endpoint_statistics_t *stats = NULL;
-        int s_count = 0;
-        bool is_busy = false;
-
-        // 3. Retrieve status and statistics
-        dv_get_endpoint_busyness(session, ep, &is_busy);
-        status = dv_endpoint_get_statistics(session, ep, &stats, &s_count);
-
-        if (status == DV_SUCCESS && s_count > 0) {
-            // DRAM Calculations (Bytes to GB)
-            double used_gb = (double)stats->ep_dram_stats.ep_total_dram_occupancy_size / 1073741824.0;
-            double total_gb = (double)stats->ep_dram_stats.ep_total_dram_size / 1073741824.0;
-            double dram_pct = (total_gb > 0) ? (used_gb / total_gb) * 100.0 : 0.0;
-
-            // NPU Utilization (Queue occupancy)
-            double npu_load = 0.0;
-            if (stats->ep_infq_stats && stats->ep_infq_stats->length > 0) {
-                npu_load = ((double)stats->ep_infq_stats->occupancy_count / stats->ep_infq_stats->length) * 100.0;
-            }
-
-            printf("--- NPU Endpoint %d Statistics ---\n", i);
-            printf("Busy State:       %s\n", is_busy ? "TRUE" : "FALSE");
-            printf("NPU Utilization:  %.1f%%\n", npu_load);
-            printf("Temperature:      %.1f C\n", stats->ep_temp);
-            printf("NNP Clock:        %d MHz\n", stats->ep_nnp_clk);
-            printf("SBP Clock:        %d MHz\n", stats->ep_sbp_clk);
-            printf("DRAM Clock:       %d MHz\n", stats->ep_dram_clk);
-            
-            // Format: DRAM Usage: 8.2GB/16.0GB (51.3%)
-            printf("DRAM Usage:       %.1fGB/%.1fGB (%.1f%%)\n", used_gb, total_gb, dram_pct);
-            printf("\n");
-
-            dv_endpoint_free_statistics(stats, s_count);
-        }
-    }
-
-    // 4. Cleanup
-    dv_endpoint_free_group(ep_list);
-    dv_session_close(session);
-    return 0;
-}
-}}}
-
-Compile:
-{{{#!bash
-apt update && apt install build-essential
-gcc ara_status.c -I/usr/include/sdk_ara/ -L/usr/lib/ -laraclient_aarch64 -o ara_status
-}}}
-
-Execution:
-{{{#!bash
-# ./ara_status
---- NPU Endpoint 0 Statistics ---
-Busy State:       FALSE
-NPU Utilization:  0.0%
-Temperature:      56.0 C
-NNP Clock:        900 MHz
-SBP Clock:        355 MHz
-DRAM Clock:       1066 MHz
-DRAM Usage:       10.0GB/16.0GB (62.5%)
-}}}
-
-
-=== Image Detection with boxying via Python
-Python is incredibly useful for accessing GStreamer and handling the ARA detection frame data and imagemagick provides excellent tools for converting and drawing on images. We use PyGObject which is a Python package that provides bindings for libraries based on GObject Introspection such as GTK, !WebKit, and GStreamer. It allows you to use C-based frameworks in python.
-
-Steps:
- 1. We need to install the C libs for GStreamer and build utilities:
-{{{#!bash
-apt-get install -y \
-  libcairo2-dev \
-  libgirepository-2.0-dev \
-  python3-dev \
-  python3-gst-1.0 \
-  cmake pkg-config
-# we are also going to need to install gstreamer and its dev packages
-apt-get install -y \
-  libgstreamer1.0-dev \
-  libgstreamer-plugins-base1.0-dev \
-  libgstreamer-plugins-bad1.0-dev \
-  gstreamer1.0-plugins-base \
-  gstreamer1.0-plugins-good \
-  gstreamer1.0-plugins-bad \
-  gstreamer1.0-plugins-ugly \
-  gstreamer1.0-libav \
-  gstreamer1.0-tools
-}}}
- 1. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:
-{{{#!bash
-# create a dir for the venv
-mkdir image-detect
-cd image-detect
-# create a venv (.venv)
-uv venv
-# install our scripts dependencies
-uv pip install pygobject
-}}}
- 1. (optional) fetch some images for detection
-{{{#!bash
-# fetch a coco validation image; it contains a dog on a bench and the dog is at 208,147 to 293,289
-wget http://images.cocodataset.org/val2017/000000546829.jpg -O dog.jpg
-# use ffmpeg to grab a frame from within an MP4
-apt install -y ffmpeg
-ffmpeg -i /usr/share/ara2-vision-examples/sample_videos/video_0.mp4 -f null - # shows how lon git is (time=00:00:15.50)
-ffmpeg -i /usr/share/ara2-vision-examples/sample_videos/video_0.mp4 -ss 00:00:5 -frames:v 1 traffic.png
-}}}
- 1. fetch the script
-{{{#!bash
-wget https://dev.gateworks.com/ara/examples/image_detect.py
-}}}
- 1. run the script (image_detect.py <source-image> <destination-image> [model-path])
-{{{#!bash
-uv run image_detect.py dog.jpg coco_detections.jpg
-}}}
-   - Note that without shm the pipeline needs to copy the raw image bytes over a local network-style socket connection. By mounting a dedicated memory path to /dev/shm you can eliminate that transfer (zero-copy): dvPre dumps the processed directly into a designated block of system RAM and dvInf uses a pointer to it
-   - you would think that if your original image was 1080x1920 and you resized it to the model size of 640x640 that if you tell dvPost the orig-width=1080 orig-height=1920 that it would scale the bounding boxes properly however in practice it seems it does not unless your image has the same aspect ratio of the model. mapping it as above (telling dvPost that the image is 640x640 and scaling ourselves) resolves this
-   - images:
-
-[[Image(dog.jpg,400px)]]
-[[Image(dog_detect.jpg,400px)]]
-
-[[Image(traffic.jpg,400px)]]
-[[Image(traffic_detect_yolo8n.jpg,400px)]]
-[[Image(traffic_detect_yolo8x.jpg,400px)]]
-
-
-=== Video Detection Webapp via Python
-Python is incredibly useful for accessing GStreamer and handling the ARA detection frame data and building webapps. The script using PyGObject which is a Python package that provides bindings for libraries based on GObject Introspection such as GTK, !WebKit, and GStreamer. It allows you to use C-based frameworks in python. We need to install the C libs for GStreamer for this
-
-Steps:
- 1. We need to install the C libs for GStreamer and build utilities:
-{{{#!bash
-apt-get install -y \
-  libcairo2-dev \
-  libgirepository-2.0-dev \
-  python3-dev \
-  python3-gst-1.0 \
-  cmake pkg-config
-# we are also going to need to install GStreamer and its dev packages
-apt-get install -y \
-  libgstreamer1.0-dev \
-  libgstreamer-plugins-base1.0-dev \
-  libgstreamer-plugins-bad1.0-dev \
-  gstreamer1.0-plugins-base \
-  gstreamer1.0-plugins-good \
-  gstreamer1.0-plugins-bad \
-  gstreamer1.0-plugins-ugly \
-  gstreamer1.0-libav \
-  gstreamer1.0-tools
-}}}
- 1. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:
-{{{#!bash
-# create a dir for the venv
-mkdir vision-webapp
-cd vision-webapp
-# create a venv (.venv)
-uv venv
-# install our scripts dependencies
-uv pip install pygobject opencv-python-headless flask
-}}}
- 1. fetch the script
-{{{#!bash
-wget https://dev.gateworks.com/ara/examples/vision-webapp.py
-}}}
- 1. run the script (vison-webapp.py [--port <portno>] [--camera <camera-dev>] [--mp4 <mp4-dir>]
-{{{#!bash
-uv run vision-webapp.py --camera /dev/video_webcam --mp4 /usr/share/media/sample_videos/
-}}}
-  - you can provide a webcam device path to enable streaming from a webcam and/or an mp4 directory to enable processing those. A dropdown will allow you to select the input stream and the model and the browser window will show you detections and statistics 
-
-[[Image(vision-webapp.jpg,400px)]]
-
-
-=== command-line python eIQ chatbot (chat.py)
-This is a command-line chatbot written in python using the eIQ AAF Connector
- 1. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:
-{{{#!bash
-# create a dir for the venv
-mkdir chat
-cd chat
-# create a venv (.venv)
-uv venv
-# install our scripts dependencies
-uv pip install -q requests
-}}}
- 1. fetch the script
-{{{#!bash
-wget https://dev.gateworks.com/ara/examples/chat.py
-}}}
- 1. run the script
-{{{#!bash
-uv run chat.py
-}}}
-
-Example session:
-{{{#!bash
---- i.MX LLM Session (Model: Qwen2.5-7B-Instruct) ---
-Type 'exit' to stop.
-
-You: Why is the sky blue
-AI: The sky appears blue because of a phenomenon called Rayleigh scattering. When sunlight enters the Earth's atmosphere, it collides with molecules and small particles in the air. Sunlight is made up of different colors, each of which has a different wavelength. Blue light has a shorter wavelength and is scattered more than other colors by the gases and particles in the atmosphere. This scattering makes the sky appear blue to our eyes.
-
-During sunrise and sunset, the sky can appear red or orange because the light has to travel through more of the Earth's atmosphere. This longer path means that more blue and green light is scattered out of the beam, leaving the red and orange wavelengths to dominate the light that reaches our eyes.
-
-So, the blue color of the sky is primarily due to the way shorter wavelength light is scattered by the Earth's atmosphere.
-
---- Stats ---
-Time taken: 29.18 seconds
-Throughput: 5.04 tokens/sec
--------------
-
-You: exit
-}}}
-
-
-=== Web based python eIQ chatbot (webchat.py)
-This is a web based chatbot in python using eIQ AAF Connector
-
- 1. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:
-{{{#!bash
-# create a dir for the venv
-mkdir webchat
-cd webchat
-# create a venv (.venv)
-uv venv
-# install our scripts dependencies
-uv pip install -q fastapi psutil uvicorn
-}}}
- 1. fetch the script
-{{{#!bash
-wget https://dev.gateworks.com/ara/examples/webchat.py
-}}}
- 1. run the script
-{{{#!bash
-uv run webchat.py
-}}}
- 1. Open a web browser to your boards IP address port 8080: http://<ipaddr>:8080
-
-Notes:
- * By default this will listen for HTTP requests on port 8080
- * On startup it will enable the model (specified in the script) and restart the eIQ server if needed - waiting for the model to load may take several minutes
-
-
-[=#vlm]
-=== Web based python VLM eIQ example (webvlm.py)
-The eIQ AAF Connector can be used to analyze video and images.
-
-Here is an example of a headless web-app based off NXP's [https://github.com/nxp-imx-support/vlm-edge-studio/tree/main/src vlm-edge-studio example] using:
- - Qwen2.5-VL-7B-Instruct-Ara240
- - eIQ AAF Connector
-
-Requirements:
- - Ara runtime
- - eIQ AAF Connector
- - Qwen2.5-VL-7B-Instruct-Ara240 model
-
-Steps:
- 1. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:
-{{{#!bash
-# create a dir for the venv
-mkdir webvlm
-cd webvlm
-# create a venv (.venv)
-uv venv
-# install our scripts dependencies
-uv pip install -q httpx uvicorn fastapi argparse
-}}}
- 1. fetch the script
-{{{#!bash
-wget https://dev.gateworks.com/ara/examples/webvlm.py
-}}}
- 1. run the script
-{{{#!bash
-uv run webvlm.py --video-dir /usr/share/media/sample_videos/
-}}}
- 1. Open a web browser to your boards IP address port 8080: http://<ipaddr>:8080
-
-Notes:
- * By default this will listen for HTTP requests on port 8080
- * On startup it will enable the model (specified in the script) and restart the eIQ server if needed - waiting for the model to load may take several minutes
-
-
-[[Image(vlm-webapp.jpg,400px)]]
-
-
-== Troubleshooting
-
-Please note software support should be routed through NXP, who produces the Ara240 DNPU Chip and Software SDK. 
-
-[https://community.nxp.com]
 
 
@@ -1105 +474 @@
 INFO:     Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
 }}}
+
+
+= NXP Ara240 DNPU AI Accelerator Quick Start
+Below are details regarding installing the ARA2 runtime software requirements manually. If you are using the Gateworks i.MX95 Catalina AI Devkit this software is already installed. Please see [#ai-devkit above for details]
+ 
+[=#ara2-runtime]
+== Ara Runtime
+NXP has a runtime library for the Ara240 which consists of some statically built libraries as well as a dynamic linked GStreamer plugin.
+
+The Ara runtime includes a couple of Python Wheels. A Python Wheel is a standard built-package format for distributing Python libraries. It is essentially a ZIP-format archive with a .whl extension that contains all the files needed for a package to run immediately after being. It's fairly standard when using Python to run into package version incompatibilities which is why user based Python virtual environments are used.
+
+The Ara runtime provides a complete runtime environment for AI/ML acceleration using the Ara240 NPU on for aarch64. This package includes:
+ * Runtime libraries for Ara240 NPU integration
+ * Python bindings (DVAPI) for custom inference applications
+ * Optimum-Ara framework for LLMs and VLMs
+ * GStreamer plugin for Real-Time Detection Object Applications
+ * Helper scripts for monitoring, benchmarking, and model management
+ * Systemd service for automatic hardware initialization
+
+Installation on a Gateworks board with Ubuntu based OS:
+ - Download and extract the self-extracting binary from NXP:
+{{{#!bash
+VER=imx-nxp-ara2-2.1.1-063d56c
+wget https://www.nxp.com/lgfiles/NMG/MAD/YOCTO/$VER.bin
+sh $VER.bin
+}}}
+ - take care of postinst steps
+  - miscellaneous
+{{{#!bash
+# create dirs (used for models)
+mkdir -pv /usr/share/{cnn,llm}
+}}}
+  - configure swap (necessary if using VLM)
+{{{#!bash
+/usr/bin/enable_swap 2
+}}}
+  - install uv package manager for Python virtualization and packaging for local user (which is installed to ~/.local/bin so we create symlinks to /usr/bin)
+{{{#!bash
+apt update && apt install -y curl
+curl -LsSf https://astral.sh/uv/install.sh | sh
+ln -s /root/.local/bin/uv /usr/bin/uv
+ln -s /root/.local/bin/uvx /usr/bin/uvx
+}}}
+  - build driver
+{{{#!bash
+apt update && apt install -y build-essential git bc file flex bison
+git clone https://github.com/nxp-imx-support/uiodma-driver
+( cd uiodma-driver/uiodma; make )
+# install it where the rt service expects to find it (over the top of the non-compatible one)
+cp uiodma-driver/uiodma/uiodma.ko /usr/share/rt-sdk-ara240/driver/
+}}}
+  - enable service:
+{{{#!bash
+# enable service
+systemctl enable rt-sdk-ara2.service
+# start service now (unless you reboot)
+systemctl start rt-sdk-ara2.service
+}}}
+  - use 'fetch_models' to pre-compiled models for testing via the fetch_models script which will fetch models from !HuggingFace.
+{{{#!bash
+# list models available for nxp/ara
+fetch_models --list
+# install YOLOv8
+fetch_models --repo-id nxp/YOLOv8 # 746MB (711MiB)
+}}}
+   - the script is a python wrapper that uses uvx and the fetch-models python wheel (/usr/share/python-wheels/fetch_models-1.0.0-py3-none-any.whl) to fetch and install models from !HuggingFace HUB
+   - the models will be installed in either /usr/share/cnn (Convolutional Neural Network) and /usr/share/llm (Large Language Model)
+   - NXP has Ara2 optimized models at https://huggingface.co/nxp
+   - the script has a hard coded list of models available and where to install them locally. You can use 'python -m zipfile -e /usr/share/python-wheels/fetch_models-1.0.0-py3-none-any.whl ./fetch_models' to see what it's doing
+
+Notable Files:
+ - /usr/lib/
+  - libaraclient_aarch64.so - base library for interfacing with ara2
+  - libara_vision_inference.so - inference lib that builds on libaraclient
+ - /usr/lib/gstreamer-1.0
+  - libgstdvInf.so - GStreamer plugin
+ - /usr/share/rt-sdk-ara240 (symlink to a version independent dir at same location)
+  - hw_utils/boot_img - firmware files
+  - hw_utils/ddr_config - ddr binaries
+  - hw_utils/bins/ - the hw utils for bringup/programming
+  - optimum-ara/ - extension of the Hugging Face library that integrates with Ara240 DNPU
+  - scripts - various wrappers around the tools etc
+  - nnapp - tool for benchmarking models
+  - config - various example yaml config files used for proxy/nnapp
+  - include/dvapi.py - python bindings to dvapi
+  - driver/uiodma.ko - driver (where the setup script expects to find it)
+ - /usr/share/python-wheels - python wheels for fetch_models and optimum_ara
+ - /usr/shar/doc/rt-sdk-ara2 - license info
+ - /usr/include/sdk_ara - headers for C libs
+ - /usr/bin - various scripts
+ - /etc/udev/rules.d/99-ara2.rules - udev rule which makes the PCI ID dependent on the systemd service
+ - /etc/systemd/system/rt-sdk-ara2.service - systemd service that handles the various hw util config
+ - /etc/rt-sdk-ara240/cnn_config.yaml - config for nnapp
+ - /etc/rt-sdk-ara240/proxy_config.yam - config for proxy
+
+Notes:
+ - The 'uv' package manager is a fast all-in-one Python package and project manager written in Rust which makes it easy to work with virtual env's to avoid Python package version clashing which is essential
+ - on bootup make sure you wait for the console messages indicating the Proxy is launched before using it as it can take a couple of minutes
+ - the binary tools and libs are all static linked for compatibility
+ - the GStreamer libs require GStreamer 1.26 or newer and is dynamic linked
+
+Verification steps:
+ 1. show chip_info
+{{{#!bash
+chip_info.sh
+}}}
+ 1. verify service
+{{{#!bash
+# show service status
+systemctl status rt-sdk-ara2.service --no-pager -l
+# view detailed service logs for the current boot and follow the journal
+journalctl --boot --follow --unit rt-sdk-ara2.service
+# verify proxy is running (critical)
+ps -eaf | grep proxy_ara240
+}}}
+
+Examples:
+ - Download pre-compiled models for testing:
+  - The fetch_models script from the ara2-rt will fetch models from !HuggingFace.
+{{{#!bash
+# list models available for nxp/ara
+fetch_models --list
+# install YOLOv8
+fetch_models --repo-id nxp/YOLOv8 # 746MB (711MiB)
+}}}
+  - the 'fetch_models' script is a python wrapper that uses uvx and the fetch-models python wheel (/usr/share/python-wheels/fetch_models-1.0.0-py3-none-any.whl) to fetch and install models from !HuggingFace HUB
+  - the models will be installed in /usr/share/cnn (Convolutional Neural Network) and /usr/share/llm (Large Language Model)
+  - NXP has Ara2 optimized models at https://huggingface.co/nxp
+ - Run performance benchmark (uses nnapp)
+{{{#!bash
+run_model_perf.sh
+}}} 
+  - the 'run_model_perf.sh' script makes it easy to list and show model categories and models and is a wrapper around the nnapp app which has a lot of options and a config file
+ - monitor real-time NPU metrics including utilization, temperature, DRAM usage and device state (interactively during benchmarking or model execution)
+{{{#!bash
+ara2_metrics.sh
+}}}
+
+[=#gstreamer]
+=== GStreamer plugins
+The Ara runtime provides an OpenSource GStreamer plugin for detection models:
+ - [https://github.com/nxp-imx-support/gstreamer-plugins-ara240 dvInf]
+
+The plugin can sink 32bit pixel samples (ie format=BGRx using 4 bytes per pixel, blue, green, red, and a pading byte as a structural spacer)
+
+The model is specified via the 'model' property. If using yolov8x for example you would specify the path to the yolov8x.dvm
+
+For detection models the element frame data will contain a buffer with number of bytes (32bit) followed by a series of detection structures containing the bounding box, confidence level, and COCO class ID of the object detected.
+
+The units for the bounding box are relative to the models size and will need to be scaled back to your original image size. For example the YOLO models operate on 640x640 pixel data. You can pass something larger in and it will essentially tile but its unclear if there is an advantage of doing that.
+
+While the gstreamer plugin source provided is provided [https://github.com/nxp-imx-support/gstreamer-plugins-ara240 here] it is included in the Ara runtime pre-compiled for convenience linked against stdlibc (libc.so.6) and libgstreamer-1.0.so.0 and compatible with GStreamer 1.26 or newer.
+
+Install GStreamer:
+{{{#!bash
+apt-get update && apt install -y \
+   gstreamer1.0-x \
+   gstreamer1.0-tools \
+   gstreamer1.0-plugins-base \
+   gstreamer1.0-plugins-good \
+   gstreamer1.0-plugins-bad \
+   gstreamer1.0-plugins-ugly \
+   gstreamer1.0-libav \
+   v4l-utils
+}}}
+ - this adds about 500MiB of disk space
+
+Specify Plugin path:
+{{{#!bash
+# export now to current shell
+export GST_PLUGIN_PATH=/usr/lib/gstreamer-1.0/
+# put in .bashrc so it happens for any new bash shell
+echo "export GST_PLUGIN_PATH=/usr/lib/gstreamer-1.0/" >> ~/.bashrc
+}}}
+ - this tells GStreamer to look for plugins in the non-standard location of the ARA gstreamer plugins
+
+At this point you can inspect the dvInf element:
+{{{#!bash
+gst-inspect-1.0 dvInf
+}}}
+
+Examples:
+ * gst-launch pipeline prototyping:
+  - enabling debug level 6 on dvPost will show the number of object detections in its debug output but if you want to do anything with that data you need to write an application that can decode frame buffers. Still this is useful for prototyping:
+   * perform detection on a v4l2 video device like a webcam:
+{{{#!bash
+DEV=/dev/video_webcam
+MODEL=/usr/share/cnn/detection/yolov8n/model.dvm
+GST_DEBUG="dvInf:6" \
+gst-launch-1.0 -v \
+  v4l2src device=$DEV ! \
+  video/x-raw,width=640,height=480,framerate=30/1 ! \
+  videoconvert ! video/x-raw,format=BGRx ! \
+  dvInf model=$MODEL orig-width=640 orig-height=480 stream=0 \
+        sock=/var/run/proxy.sock use-shm=true shm-path=/dev/shm/ara_shm ! \
+  fakesink sync=false | grep Detected
+}}}
+    - see wiki:linux/persistent_device_naming#video for details about making video devices have persistent device names
+   * perform a detection on an image:
+{{{#!bash
+URI=file:///$PWD/traffic.png
+MODEL=/usr/share/cnn/detection/yolov8n/model.dvm
+GST_DEBUG="dvInf:6" \
+gst-launch-1.0 -v \
+  filesrc location=traffic.png ! \
+  pngdec ! imagefreeze num-buffers=10 ! \
+  videoscale ! videoconvert ! video/x-raw,format=BGRx,width=640,height=480 ! \
+  dvInf model=$MODEL orig-width=640 orig-height=480 stream=0 \
+        sock=/var/run/proxy.sock use-shm=true shm-path=/dev/shm/ara_shm ! \
+  fakesink sync=false | grep Detected
+}}}
+
+For a more complete example see below
+
+
+[=#eiq-aaf-connector]
+== eIQ AAF Connector for LLM inference
+The eIQ AAF Connector (edge Intelligence Ara Application Framework)
+is a REST-based server that enables LLM inference on NXP i.MX processors with the ARA-240 DNPU. The API implemented is the de-facto API standard created by OpenAI for ChatGPT. It provides a simple Chat Completions-based HTTP interface for serving models to client applications.
+
+Requirements:
+ - python 3.13 (we will install in a virtual env)
+ - uv - used for the user-specific Python virtual environment
+ - Optimum Ara framework for running Large Language Models (LLMs) and Vision-Language Models (VLMs) on Ara240 (part of rt-sdk)
+ - OpenCV (dependency of the QwenVL engine)
+ - Models
+
+Source:
+ - https://github.com/nxp-imx-support/eiq-aaf-connector
+
+For ease of use Gateworks provides a pre-built deb package of eiq-aaf-connector v2.1 built from the NXP IMX Yocto BSP which you can install with:
+{{{#!bash
+# fetch
+wget https://dev.gateworks.com/ara/eiq-aaf-connector_2.1-r0_arm64.deb
+# extract data (but don't install)
+dpkg-deb --vextract eiq-aaf-connector_2.1-r0_arm64.deb /
+# run the install script
+/usr/share/eiq/aaf-connector/install.sh
+# fetch LLM models (installed to /usr/share/llm)
+fetch_models --repo-id nxp/Qwen2.5-7B-Instruct-Ara240 # 7.7GiB LLM
+fetch_models --repo-id nxp/Qwen2.5-Coder-1.5B-Ara240 # 1.67GiB LLM
+fetch_models --repo-id nxp/Qwen2.5-VL-7B-Instruct-Ara240 # 12GB VLM
+}}}
+
+files:
+ - /usr/share/eiq/aaf-connector/install.sh (install script)
+ - /usr/share/python-wheels/eiq_aaf_connector-2.1-py3-none-any.whl (python wheel)
+ - /usr/share/eiq/aaf-connector/server_config.json (config file)
+ - /etc/systemd/system/eiq-aaf-connector.service (created from install script)
+
+The install script creates a systemd eiq-aaf-connector.service:
+{{{#!bash
+# Enable service on boot
+systemctl enable eiq-aaf-connector.service
+# Start the service now (or reboot)
+systemctl start eiq-aaf-connector.service
+}}}
+
+Notes:
+ - By default the connector will listen on 127.0.0.1:8000. If you wish the service to be accessible externally set the host to '0.0.0.0' instead:
+{{{#!bash
+sed -i 's|--host 127.0.0.1|--host 0.0.0.0|g' /etc/systemd/system/eiq-aaf-connector.service
+}}}
+ - the default config file has configuration for all of the above Ara models but they are not 'enabled' by default. You must only enable 1 model at a time and doing so loads the model onto the Ara when the servoce starts. To enable a model change the appropriate 'enabled' property to 'true' in /etc/systemd/system/eiq-aaf-connector.service and restart the service
+ - it takes several minutes for the service to actually be ready for connections as it must process the models (monitor with 'journalctl -u eiq-aaf-connector.service --no-pager -f' and test that its ready for listening with 'ss -tulpn | grep :8000').
+ - the connector self-hosts API documentation at http://<serverip>:8000/docs (available externally if configured for a host of 0.0.0.0)
+
+Example Usage:
+ - verify connector running
+{{{#!bash
+# show service status
+systemctl status eiq-aaf-connector.service --no-pager -l
+# view detailed service logs for the current boot and follow the journal
+journalctl --boot --follow --unit eiq-aaf-connector.service
+# verify process exists
+ps -ef | grep aaf-connector
+# verify port open
+ss -tulpn | grep :8000 # show IP:PORT server is listening on
+}}}
+ - view API docs and interact with server (requires changing the host to '0.0.0.0' in the !ExecStart config for /etc/systemd/system/eiq-aaf-connector.service by opening !http://<serverip>:8000/docs
+ - use API via curl/jq
+{{{#!bash
+# make sure curl and jq are installed (jq allows easy interaction with json data)
+apt install -y curl jq
+# list of models
+curl -X 'GET' \
+  'http://127.0.0.1:8000/v1/models' \
+  -H 'accept: application/json' | jq
+# get info about a specific model (Qwen2.5-7B-Instruct)
+curl -X 'GET' \
+  'http://127.0.0.1:8000/params/Qwen2.5-7B-Instruct' \
+  -H 'accept: application/json' | jq
+# send a LLM query
+curl -X POST http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
+  "model": "Qwen2.5-7B-Instruct",
+  "messages": [
+    {"role": "system", "content": "You are a helpful assistant running on NXP i.MX hardware."},
+    {"role": "user", "content": "Explain what an NPU is in one sentence."}
+  ],
+  "max_tokens": 50
+}' | jq
+}}}
+ - run connector by hand (useful for troubleshooting or monitoring)
+{{{#!bash
+systemctl stop eiq-aaf-connector.service
+source "/usr/share/eiq/aaf-connector/venv/bin/activate"
+connector --host 0.0.0.0 --port 8000 # will run until stopped
+deactivate
+}}}
+
+
+[=#examples]
+== Examples
+Here are some Ara example applications put together by Gateworks
+
+=== dvapi stats
+This is an ANSI c app that provides an example of using the dvapi to connect to the proxy and obtain NPU endpoint stats such as temperature, clocks and usage. Basically it's a re-implementation of the closed source /usr/share/rt-sdk-ara240/scripts/ara2_metrics_bin/hw_metrics.out.
+
+ara_status.c:
+{{{#!c
+#include <stdio.h>
+#include <stdlib.h>
+#include "dvapi.h"
+
+int main() {
+    dv_session_t *session = NULL;
+    dv_endpoint_t *ep_list = NULL;
+    int ep_count = 0;
+    dv_status_code_t status;
+    const char *socket_path = "/run/proxy.sock"; 
+
+    // 1. Establish session
+    status = dv_session_create_via_unix_socket(socket_path, &session);
+    if (status != DV_SUCCESS) {
+        fprintf(stderr, "Failed to connect: %s\n", dv_stringify_status_code(status));
+        return 1;
+    }
+
+    // 2. Get list of NPU endpoints
+    dv_endpoint_get_list(session, &ep_list, &ep_count);
+
+    for (int i = 0; i < ep_count; i++) {
+        dv_endpoint_t *ep = &ep_list[i];
+        dv_endpoint_statistics_t *stats = NULL;
+        int s_count = 0;
+        bool is_busy = false;
+
+        // 3. Retrieve status and statistics
+        dv_get_endpoint_busyness(session, ep, &is_busy);
+        status = dv_endpoint_get_statistics(session, ep, &stats, &s_count);
+
+        if (status == DV_SUCCESS && s_count > 0) {
+            // DRAM Calculations (Bytes to GB)
+            double used_gb = (double)stats->ep_dram_stats.ep_total_dram_occupancy_size / 1073741824.0;
+            double total_gb = (double)stats->ep_dram_stats.ep_total_dram_size / 1073741824.0;
+            double dram_pct = (total_gb > 0) ? (used_gb / total_gb) * 100.0 : 0.0;
+
+            // NPU Utilization (Queue occupancy)
+            double npu_load = 0.0;
+            if (stats->ep_infq_stats && stats->ep_infq_stats->length > 0) {
+                npu_load = ((double)stats->ep_infq_stats->occupancy_count / stats->ep_infq_stats->length) * 100.0;
+            }
+
+            printf("--- NPU Endpoint %d Statistics ---\n", i);
+            printf("Busy State:       %s\n", is_busy ? "TRUE" : "FALSE");
+            printf("NPU Utilization:  %.1f%%\n", npu_load);
+            printf("Temperature:      %.1f C\n", stats->ep_temp);
+            printf("NNP Clock:        %d MHz\n", stats->ep_nnp_clk);
+            printf("SBP Clock:        %d MHz\n", stats->ep_sbp_clk);
+            printf("DRAM Clock:       %d MHz\n", stats->ep_dram_clk);
+            
+            // Format: DRAM Usage: 8.2GB/16.0GB (51.3%)
+            printf("DRAM Usage:       %.1fGB/%.1fGB (%.1f%%)\n", used_gb, total_gb, dram_pct);
+            printf("\n");
+
+            dv_endpoint_free_statistics(stats, s_count);
+        }
+    }
+
+    // 4. Cleanup
+    dv_endpoint_free_group(ep_list);
+    dv_session_close(session);
+    return 0;
+}
+}}}
+
+Compile:
+{{{#!bash
+apt update && apt install build-essential
+gcc ara_status.c -I/usr/include/sdk_ara/ -L/usr/lib/ -laraclient_aarch64 -o ara_status
+}}}
+
+Execution:
+{{{#!bash
+# ./ara_status
+--- NPU Endpoint 0 Statistics ---
+Busy State:       FALSE
+NPU Utilization:  0.0%
+Temperature:      56.0 C
+NNP Clock:        900 MHz
+SBP Clock:        355 MHz
+DRAM Clock:       1066 MHz
+DRAM Usage:       10.0GB/16.0GB (62.5%)
+}}}
+
+
+=== Image Detection with boxying via Python
+Python is incredibly useful for accessing GStreamer and handling the ARA detection frame data and imagemagick provides excellent tools for converting and drawing on images. We use PyGObject which is a Python package that provides bindings for libraries based on GObject Introspection such as GTK, !WebKit, and GStreamer. It allows you to use C-based frameworks in python.
+
+Steps:
+ 1. We need to install the C libs for GStreamer and build utilities:
+{{{#!bash
+apt-get install -y \
+  libcairo2-dev \
+  libgirepository-2.0-dev \
+  python3-dev \
+  python3-gst-1.0 \
+  cmake pkg-config
+# we are also going to need to install gstreamer and its dev packages
+apt-get install -y \
+  libgstreamer1.0-dev \
+  libgstreamer-plugins-base1.0-dev \
+  libgstreamer-plugins-bad1.0-dev \
+  gstreamer1.0-plugins-base \
+  gstreamer1.0-plugins-good \
+  gstreamer1.0-plugins-bad \
+  gstreamer1.0-plugins-ugly \
+  gstreamer1.0-libav \
+  gstreamer1.0-tools
+}}}
+ 1. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:
+{{{#!bash
+# create a dir for the venv
+mkdir image-detect
+cd image-detect
+# create a venv (.venv)
+uv venv
+# install our scripts dependencies
+uv pip install pygobject
+}}}
+ 1. (optional) fetch some images for detection
+{{{#!bash
+# fetch a coco validation image; it contains a dog on a bench and the dog is at 208,147 to 293,289
+wget http://images.cocodataset.org/val2017/000000546829.jpg -O dog.jpg
+# use ffmpeg to grab a frame from within an MP4
+apt install -y ffmpeg
+ffmpeg -i /usr/share/ara2-vision-examples/sample_videos/video_0.mp4 -f null - # shows how lon git is (time=00:00:15.50)
+ffmpeg -i /usr/share/ara2-vision-examples/sample_videos/video_0.mp4 -ss 00:00:5 -frames:v 1 traffic.png
+}}}
+ 1. fetch the script
+{{{#!bash
+wget https://dev.gateworks.com/ara/examples/image_detect.py
+}}}
+ 1. run the script (image_detect.py <source-image> <destination-image> [model-path])
+{{{#!bash
+uv run image_detect.py dog.jpg coco_detections.jpg
+}}}
+   - Note that without shm the pipeline needs to copy the raw image bytes over a local network-style socket connection. By mounting a dedicated memory path to /dev/shm you can eliminate that transfer (zero-copy): dvPre dumps the processed directly into a designated block of system RAM and dvInf uses a pointer to it
+   - you would think that if your original image was 1080x1920 and you resized it to the model size of 640x640 that if you tell dvPost the orig-width=1080 orig-height=1920 that it would scale the bounding boxes properly however in practice it seems it does not unless your image has the same aspect ratio of the model. mapping it as above (telling dvPost that the image is 640x640 and scaling ourselves) resolves this
+   - images:
+
+[[Image(dog.jpg,400px)]]
+[[Image(dog_detect.jpg,400px)]]
+
+[[Image(traffic.jpg,400px)]]
+[[Image(traffic_detect_yolo8n.jpg,400px)]]
+[[Image(traffic_detect_yolo8x.jpg,400px)]]
+
+
+=== Video Detection Webapp via Python
+Python is incredibly useful for accessing GStreamer and handling the ARA detection frame data and building webapps. The script using PyGObject which is a Python package that provides bindings for libraries based on GObject Introspection such as GTK, !WebKit, and GStreamer. It allows you to use C-based frameworks in python. We need to install the C libs for GStreamer for this
+
+Steps:
+ 1. We need to install the C libs for GStreamer and build utilities:
+{{{#!bash
+apt-get install -y \
+  libcairo2-dev \
+  libgirepository-2.0-dev \
+  python3-dev \
+  python3-gst-1.0 \
+  cmake pkg-config
+# we are also going to need to install GStreamer and its dev packages
+apt-get install -y \
+  libgstreamer1.0-dev \
+  libgstreamer-plugins-base1.0-dev \
+  libgstreamer-plugins-bad1.0-dev \
+  gstreamer1.0-plugins-base \
+  gstreamer1.0-plugins-good \
+  gstreamer1.0-plugins-bad \
+  gstreamer1.0-plugins-ugly \
+  gstreamer1.0-libav \
+  gstreamer1.0-tools
+}}}
+ 1. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:
+{{{#!bash
+# create a dir for the venv
+mkdir vision-webapp
+cd vision-webapp
+# create a venv (.venv)
+uv venv
+# install our scripts dependencies
+uv pip install pygobject opencv-python-headless flask
+}}}
+ 1. fetch the script
+{{{#!bash
+wget https://dev.gateworks.com/ara/examples/vision-webapp.py
+}}}
+ 1. run the script (vison-webapp.py [--port <portno>] [--camera <camera-dev>] [--mp4 <mp4-dir>]
+{{{#!bash
+uv run vision-webapp.py --camera /dev/video_webcam --mp4 /usr/share/media/sample_videos/
+}}}
+  - you can provide a webcam device path to enable streaming from a webcam and/or an mp4 directory to enable processing those. A dropdown will allow you to select the input stream and the model and the browser window will show you detections and statistics 
+
+[[Image(vision-webapp.jpg,400px)]]
+
+
+=== command-line python eIQ chatbot (chat.py)
+This is a command-line chatbot written in python using the eIQ AAF Connector
+ 1. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:
+{{{#!bash
+# create a dir for the venv
+mkdir chat
+cd chat
+# create a venv (.venv)
+uv venv
+# install our scripts dependencies
+uv pip install -q requests
+}}}
+ 1. fetch the script
+{{{#!bash
+wget https://dev.gateworks.com/ara/examples/chat.py
+}}}
+ 1. run the script
+{{{#!bash
+uv run chat.py
+}}}
+
+Example session:
+{{{#!bash
+--- i.MX LLM Session (Model: Qwen2.5-7B-Instruct) ---
+Type 'exit' to stop.
+
+You: Why is the sky blue
+AI: The sky appears blue because of a phenomenon called Rayleigh scattering. When sunlight enters the Earth's atmosphere, it collides with molecules and small particles in the air. Sunlight is made up of different colors, each of which has a different wavelength. Blue light has a shorter wavelength and is scattered more than other colors by the gases and particles in the atmosphere. This scattering makes the sky appear blue to our eyes.
+
+During sunrise and sunset, the sky can appear red or orange because the light has to travel through more of the Earth's atmosphere. This longer path means that more blue and green light is scattered out of the beam, leaving the red and orange wavelengths to dominate the light that reaches our eyes.
+
+So, the blue color of the sky is primarily due to the way shorter wavelength light is scattered by the Earth's atmosphere.
+
+--- Stats ---
+Time taken: 29.18 seconds
+Throughput: 5.04 tokens/sec
+-------------
+
+You: exit
+}}}
+
+
+=== Web based python eIQ chatbot (webchat.py)
+This is a web based chatbot in python using eIQ AAF Connector
+
+ 1. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:
+{{{#!bash
+# create a dir for the venv
+mkdir webchat
+cd webchat
+# create a venv (.venv)
+uv venv
+# install our scripts dependencies
+uv pip install -q fastapi psutil uvicorn
+}}}
+ 1. fetch the script
+{{{#!bash
+wget https://dev.gateworks.com/ara/examples/webchat.py
+}}}
+ 1. run the script
+{{{#!bash
+uv run webchat.py
+}}}
+ 1. Open a web browser to your boards IP address port 8080: http://<ipaddr>:8080
+
+Notes:
+ * By default this will listen for HTTP requests on port 8080
+ * On startup it will enable the model (specified in the script) and restart the eIQ server if needed - waiting for the model to load may take several minutes
+
+
+[=#vlm]
+=== Web based python VLM eIQ example (webvlm.py)
+The eIQ AAF Connector can be used to analyze video and images.
+
+Here is an example of a headless web-app based off NXP's [https://github.com/nxp-imx-support/vlm-edge-studio/tree/main/src vlm-edge-studio example] using:
+ - Qwen2.5-VL-7B-Instruct-Ara240
+ - eIQ AAF Connector
+
+Requirements:
+ - Ara runtime
+ - eIQ AAF Connector
+ - Qwen2.5-VL-7B-Instruct-Ara240 model
+
+Steps:
+ 1. create a python virtual env (always a good idea to keep python dependencies containerized) and install python libs we need:
+{{{#!bash
+# create a dir for the venv
+mkdir webvlm
+cd webvlm
+# create a venv (.venv)
+uv venv
+# install our scripts dependencies
+uv pip install -q httpx uvicorn fastapi argparse
+}}}
+ 1. fetch the script
+{{{#!bash
+wget https://dev.gateworks.com/ara/examples/webvlm.py
+}}}
+ 1. run the script
+{{{#!bash
+uv run webvlm.py --video-dir /usr/share/media/sample_videos/
+}}}
+ 1. Open a web browser to your boards IP address port 8080: http://<ipaddr>:8080
+
+Notes:
+ * By default this will listen for HTTP requests on port 8080
+ * On startup it will enable the model (specified in the script) and restart the eIQ server if needed - waiting for the model to load may take several minutes
+
+
+[[Image(vlm-webapp.jpg,400px)]]
+
+
+== Troubleshooting
+
+Please note software support should be routed through NXP, who produces the Ara240 DNPU Chip and Software SDK. 
+
+[https://community.nxp.com]
+