Changes between Version 5 and Version 6 of venice/npu

Timestamp:

07/26/2024 07:02:22 PM (3 months ago)

Author:

Blake Stewart

Comment:

--

venice/npu

@@ -5 +5 @@
  * Any GW71xx, GW72xx and GW73xx using a GW702x SOM module will use the i.MX8M Plus processor
 
-The NPU operatines up to 2.25 TOPS. 
-
-[[Image(https://i.imgur.com/Jw1JTHp.png)]]
-
-The easiest way to get started with the NPU is to use a image from the NXP BSP. This image contains the necessary libraries and kernel to interface the NPU without much configuration. You can either [[https://www.nxp.com/docs/en/user-guide/IMX_YOCTO_PROJECT_USERS_GUIDE.pdf | follow the guide to build their image]] or [[https://www.nxp.com/design/design-center/software/embedded-software/i-mx-software/embedded-linux-for-i-mx-applications-processors:IMXLINUX | download a pre-built one]] (recommended).
+The NPU operatines up to 2.25 TOPS. Out of the box, this makes Gateworks boards with NPU capabilities powerful for AI applications on the edge.
+
+[[Image(https://trac.gateworks.com/raw-attachment/wiki/venice/npu/gw74xx_npu_benchmark.png)]]
+
+The easiest way to get started with the NPU is to use a image from the NXP BSP. This image contains the necessary libraries and kernel to interface the NPU with TensorFlow without much configuration. You can either [[https://www.nxp.com/docs/en/user-guide/IMX_YOCTO_PROJECT_USERS_GUIDE.pdf | follow the guide to build their image]] or [[https://www.nxp.com/design/design-center/software/embedded-software/i-mx-software/embedded-linux-for-i-mx-applications-processors:IMXLINUX | download a pre-built one]] (recommended).
 
 This guide assumes you have:
@@ -16 +16 @@
 - A >= 16GB flash drive, SD card, or other removable block storage to install a Rescue Image, NXP Image, and updated device trees (DTBs) onto the board.
 
+The steps are as generalized as possible to not depend on the boards available RAM to load an image, or the low speeds of JTAG uploading, as the .wic from NXP is >8GB. We will use a ramdisk to boot a "rescue image" fully in RAM, then use dd to write from the removable multimedia (flash drive) to the onboard eMMC (/dev/mmcblk2). 
+
 == Getting Started with the NPU
 === 1. Download the Gateworks Venice Rescue Image to removable multimedia.
@@ -45 +47 @@
 
 === 3. Patch & Build patch Venice DTBs from the Kernel source.
-Due to small inconsistencies between the NXP and Gateworks devicetrees for bleeding-edge peripherals, a patch is required until mainline compatibility is reached.
+Due to small inconsistencies between the NXP and Gateworks devicetrees for bleeding-edge peripherals, a patch is required until mainline compatibility is reached. The below script gets the patches from the attachments at the bottom of this page.
 
 {{{
 git clone https://github.com/nxp-imx/linux-imx -b lf-6.6.y
 cd linux-imx
-wget <patches>
+wget https://trac.gateworks.com/raw-attachment/wiki/venice/npu/0001-arm64-dts-imx8mp-venice-fix-USB_OC-pinmux.patch
+wget https://trac.gateworks.com/raw-attachment/wiki/venice/npu/0002-arm64-dts-imx8mm-venice-gw700x-remove-ddrc.patch
+wget https://trac.gateworks.com/raw-attachment/wiki/venice/npu/0003-arm64-dts-freescale-add-Gateworks-venice-board-dtbs.patch
+wget https://trac.gateworks.com/raw-attachment/wiki/venice/npu/0004-arm64-dts-imx8mp-venice-gw74xx-enable-gpu-nodes.patch
 patch -p1 < 0001-arm64-dts-imx8mp-venice-fix-USB_OC-pinmux.patch
 patch -p1 < 0002-arm64-dts-imx8mm-venice-gw700x-remove-ddrc.patch
@@ -177 +182 @@
 Without considering the warmup times, this is a >**98% speedup**! For every CPU frame, the NPU can process 53.
 
-[[Image(https://i.imgur.com/Jw1JTHp.png)]]
+[[Image(https://trac.gateworks.com/raw-attachment/wiki/venice/npu/gw74xx_npu_benchmark.png)]]
 
 === GStreamer Example
@@ -195 +200 @@
 If everything works properly, you should instantly see your video input streamed to your desktop host. After a few seconds of warming up, the bounding boxes from the [[https://nnstreamer.github.io/gst/nnstreamer/README.html | TensorFlow filter]] will be overlaid on the video. The stream properties can be changed for different resolutions and framerates; see [[https://trac.gateworks.com/wiki/Yocto/gstreamer/streaming | gstreamer/streaming]].
 
-[[Image(https://i.imgur.com/7KK4Wo8.png)]]
+[[Image(https://trac.gateworks.com/raw-attachment/wiki/venice/npu/imx8mp_border.png)]]