Changes between Version 14 and Version 15 of provisioning

Timestamp:

02/13/2021 02:03:33 AM (3 years ago)

Author:

Tim Harvey

Comment:

added sections regarding flash programming methods and provisioning from linux

provisioning

@@ -2 +2 @@
 
 = Provisioning boards =
-We refer to the act of duplicating a firmware image across multiple boards as **Provisioning**.
-
-The easiest way to provision boards or removable storage devices is to build the particular BSP you are interested in, and use its tools to create a JTAG image suitable for programming with the Gateworks JTAG dongle (for NAND flash boards) or to create removable storage devices (for NAND-less boards). See [wiki:linux/ubi linux/ubi].
- * For Ventana click [wiki:/ventana/bootloader#BuildingfromSource here] for instructions on building custom a U-boot environment.  
-
-If however you wish to customize a board's configuration in some way that you have not configured into the build system you will want to boot a board, make your customizations, then pull those customizations off and use them to provision further boards. This is what is presented in detail on this page.
-
-
-== SPI / NOR FLASH based boards (Laguna) ==
-Products that use NOR and/or SPI flash have the ability to be uploaded to a host PC via the Gateworks GW16042 JTAG dongle and jtag_usb application.
-
-For these products simply configuring a board the way you want it at runtime then uploading the flash to a file provides you with a firmware image that can then be programmed onto other boards.
-
-Please read more about JTAG upload here: [wiki:jtag_instructions JTAG Instructions]
-
+We refer to the act of installing a firmware image across multiple boards as **Provisioning**.
+
+Provisioning always starts with creating an image from a build system and this varies greatly depending on your target system. Ultimately you end up building binary images that need to be programmed to the boot device at various locations. For example, boot firmware, firmware blobs, filesystem data etc.
+
+You could stitch all these various images together into a 'disk image' or 'device image' and flash it as a single object however doing so on a particularly large flash device, say an 8GiB eMMC can take a long time depending on how much data you need to write.
+
+== FLASH programming methods and performance
+The speed at which you can physically write to a FLASH device depends on the method you are using to transfer and write the data as well as the performance of the SoC's access to the device.
+
+You have a few options available to write to the FLASH boot device:
+ - Gateworks JTAG programmer (extremely low transfer data rate)
+ - U-Boot (much higher data rate depending on driver capability in U-Boot)
+ - Linux (best data rate)
+
+Both the transfer and write data rate are important. JTAG offers an extremely low transfer data rate but is an extremely useful method for getting <32MiB boot firmware and/or provisioning firmware on a board in an extremely reliable fashion. The board does not need to have functioning or compatible firmware to begin with.
+
+Note that the performance varies greatly across product family as well. The Venice IMX8MM product family for example has much higher MMC write performance than the older Newport CN803X product family. These considerations as well as your image contents and size need to be taken into account when you decide upon your provisioning method.
+
+U-Boot offers ok transfer rates and write performance (depending on driver support in U-Boot and noting that U-Boot is not an OS) but can't be used if the firmware on the board if corrupt, blank, or if the tools and features you need for provisioning do not exist in U-Boot. Note that because Gateworks ships all boards with U-Boot and a Linux based OS you can be guaranteed to boot to a U-Boot prompt with a reliable set of tools and features.
+
+Linux offers the best transfer rate and write performance with the notion that it is an interrupt capable full operating system but also requires that you are able to boot to Linux or a Linux that has the tools and features needed for your provisioning needs. Note that Because Gateworks ships all boards with U-Boot you at least know you can start at that point to load a version of Linux supporting whatever provisioning tools you need even if the default Linux OS may not have what you need.
+
+Summary of Pros and Cons of FLASH programming methods:
+- JTAG:
+ - Pro: no dependence on booting firmware already on the boot device (FLASH can be corrupt/blank)
+ - Con: slow transfer rate
+- U-Boot:
+ - Pro: minimal dependence on boot firmware
+ - Cons:
+  - must have U-Boot pre-programmed on board
+  - performance maybe sub-par compared to Linux
+  - possible unfamiliarity of U-Boot commands and features
+  - possible lack of features and flexibility
+- Linux:
+ - Cons:
+  - more dependence and work in setting up a provisioning system
+ - Pros:
+  - best in class transfer and write performance
+  - vast variety and familiarity of tools
+
+
+== Using compressed disk images
+While Gateworks often distributes single binary compressed firmware images for options such as OpenWrt and Ubuntu compatible images, we take care to keep these images small so that the time to flash them is minimal in order to be able to easily flash them with U-Boot. This very well may not be obtainable in your situation especially if you end up wanting multiple partitions with large spaces between them.
+
+The compressed disk image method involves:
+- creating an uncompressed image that includes things like your boot firmware, partition table, and partition images
+- stitching all these together putting each binary image at its correct offset using a tool like {{{dd}}}
+- compressing this with gzip
+- optionally growing your partition and filesystem to fit the extents of the device at runtime on the first boot
+
+Pros of compressed disk image method:
+- single file compressed image distribution
+- easy to install via U-Boot (obtaining the image via tftp or removable storage)
+
+Cons of compressed disk image method:
+- the compressed image needs to fit into available RAM (or a complicated method of splitting it be used)
+- can be slow if your uncompressed image is large (ie over 100's MiB's)
+- may require resizing the partition(s) and filesystem(s) per your needs
+
+Note that in order to keep the time required to install the Gateworks Ubuntu rootfs images for example, we create a filesystem just large enough to fit the rootfs meaning (appx 1.6GiB) meaning that is all we have to write. Upon first boot a script runs that resizes the rootfs partition and filesystem to fit the extents of the device. This not only minimizes the amount of data that must be written but allows the disk image to fit into large devices and provide the user with expansion space without any further actions needed.
+
+Some performance examples:
+- Gateworks Venice GW7301-01 IMX8MM installing focal-venice.img.gz
+ - Note that IMX8MM supports HS400 eMMC data rates
+ - U-Boot:
+  - 14 seconds to transfer 460MiB compressed image via tftp to RAM at GbE
+  - 45 seconds to uncompress and write the 1.6GiB of data to eMMC at HS400 speeds
+- Gateworks Newport GW6404 CN8031 installing focal-newport.img.gz
+ - Note that CN803x supports a max of 50MHz MMC data rates
+ - U-Boot:
+  - 58 seconds to transfer 460MiB compressed image via tftp to RAM at GbE
+  - 2 minutes to uncompress and write the 1.6GiB of data to eMMC at 50Mhz speeds
+
+To present an example of where this method might not scale well consider a scenario where you multiple partitions defined across an 8GiB device. Using this method you would need to write almost the entire 8GiB which would take appx 10 minutes using this method. If your board has a 64GiB eMMC device you can see how this can get out of hand fairly quickly.
+
+
+== Hybrid approaches
+There are hybrid approaches you may consider as well. You could choose to create a JTAG'able image that contained custom firmware that automatically boots into a provisioning system that uses Linux for example. This JTAG'able image could be small enough to flash quickly over JTAG and you could even elect to have Gateworks pre-program your image on your boards if you have a Gateworks special or custom build. In this situation for example your image could pull a script from the network to complete your provisioning so that the image programmed on your boards can stay consistent but provide you flexibility in modifying the instructions for provisioning later on. This is the most flexible approach.
+
+Another example of a hyrid approach is to use U-Boot to provision images based on a script of U-Boot commands such that you can program various portions of your disk image at a time. For example if you had a scenario where you have multiple filesystems spread across a large eMMC device you could fetch and program the boot-firmware, partition table, and each filesystem independently keeping the data actually written to your FLASH device as small as possible. While this approach could be done in U-Boot alone, there are likely still multiple advantages to booting a Linux ramdisk image to provide greater flexibility and/or familiarity with tools.
+
+
+== Provisioning from a Linux ramdisk environment
+By far the most flexible solution is being able to provision your boards live with a Linux environment booting to a ramdisk providing all the tools you need for your custom environment.
+
+The most successful provisioning environments Gateworks has seen in the past are those that allow you to control your provisioning environment yourself in-dependent of what firmware may be pre-installed on your boards. While having a Gateworks special or Gateworks custom board allows you to dictate what firmware you want programmed on your boards by keeping that firmware as simple as possible you have an advantage of either not needing custom firmware or not having to change that firmware as new provisioning needs arise. For example, 
+
+While this is the most flexible it is also the most complicate to setup.
+
+A very easy way to build a minimal ramdisk Linux based OS is to use buildroot. All you need is basic kernel support for the target board and a minimal set of tools:
+- minimal root filesystem (only tools you need for your provisioning needs)
+- minimal kernel config (only drivers/features you need for your
+provisioning needs)
+- ramdisk (build artifact will be a 'Image' which is a kernel + ramdisk)
+- script that performs the provisioning
+- init config that runs the script on boot (if you wanted it automated)
+
+The tools you need in your minimal rootfs:
+- partition editor (sgdisk and/or parted)
+- filesystem creation tools (ie e2fsprogs if using ext2/3/4)
+- u-boot-tools (fw_printenv/fw_setenv) if setting or altering uboot env
+- networking support to fetch your filesystem tarballs (which could also come from removable storage)
+- optionally your provisioning script (or you can transfer this via network)
+
+The pre-built buildroot kernel+ramdisk minimal images that Gateworks provides at http://dev.gateworks.com/buildroot/ can be used for such provisioning. The instructions on how to build such an image can be found on the [wiki:#buildroot] page.
+
+When performing this provisioning, just like in creating a disk image, you need to understand what partitions may be needed by boot firmware (for example Newport expects a FATFS partition that exists within its boot firmware) and what Linux device you want to provision (ie eMMC). When we create disk partitions below we also take care to create reserved partitions to cover the boot firmware. It is also important to understand what, if any, bootscript you want to install and where.
+
+Examples:
+ * provision a Venice board eMMC with a single ext4 root filesystem from a rootfs and kernel tarball:
+  1. Load and boot kernel+ramdisk buildroot image from TFTP server
+{{{#!bash
+tftpboot $kernel_addr_r Image && booti $kernel_addr_r - $fdtcontroladdr
+}}}
+  2. Partition /dev/mmcblk0 (emmc)
+{{{#!bash
+DEV=/dev/mmcblk0
+GUID_BASIC_DATA=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
+GUID_RESERVED=8DA63339-0007-60C0-C436-083AC8230908
+GUID_LINUXFS=0FC63DAF-8483-4772-8E79-3D69D8477DE4
+# use sgdisk to create partitions
+sgdisk -og $DEV
+sgdisk -n 1:66:32767 -c 1:"boot" -t 1:$GUID_RESERVED $DEV
+sgdisk -n 2:32768:0 -c 2:"root" -t 2:$GUID_LINUXFS $DEV
+# set the legacy 'boot' attribute on rootfs partition
+sgdisk -A 2:set:2 $DEV
+# print partition table
+sgdisk -p $DEV
+}}}
+   - Note the partitioning scheme chosen above creates protective partitions around the boot firmware which includes the SPL, ATF, and U-Boot (plus it's environment). This is both to protect any other partition tools from thinking there is free space there as well as provide you a partition to be able to easily update those portions of the boot firmware at a later date
+   - Note we set the rootfs partition bit 2 attribute to mark the partition as BIOS bootable in case you are using the U-Boot generic distro config to look for bootscripts on bootable partitions
+   - Note the 'end sector' for the rootfs partition above is 0 meaning it will size to the end of the device
+  3. create and populate the rootfs partition
+{{{#!bash
+# fetch tar archives from network
+udhcpc -i eth0 # bring up networking
+cd /tmp
+wget http://tharvey/tftpboot/focal-venice.tar.xz
+wget http://tharvey/tftpboot/linux-venice.tar.xz
+
+wget http://dev.gateworks.com/ubuntu/focal/focal-venice.tar.xz
+wget http://dev.gateworks.com/venice/kernel/linux-venice.tar.xz
+# create and populate rootfs (P2) from tarballs
+PART=${DEV}p2
+mkfs.ext4 -q -F -L rootfs $PART
+mount $PART /mnt
+tar -C /mnt -xf focal-venice.tar.xz --keep-directory-symlink
+tar -C /mnt -xf linux-venice.tar.xz --keep-directory-symlink
+umount /mnt
+}}}
+  4. create a bootscript if desired (note this requires understanding of how U-Boot generic distro bootconfig works; you could alternately simply put whatever you need directly into U-boot env below)
+{{{#!bash
+cat <<\EOF > /tmp/bootscript
+echo "Venice Boot Script"
+# determine root device using uuid
+part uuid ${devtype} ${devnum}:${distro_bootpart} uuid
+# bootargs
+setenv bootargs console=$console root=PARTUUID=${uuid} rootwait $bootargs
+# load and boot kernel
+load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} ${prefix}Image &&
+booti ${kernel_addr_r} - ${fdtcontroladdr}
+EOF
+mount $PART /mnt
+mkimage -A arm64 -T script -C none -d /tmp/bootscript /mnt/boot/boot.scr
+umount /mnt
+}}}
+  5. customize U-Boot env if desired (note this requires understanding where the U-Boot env is located on FLASH)
+{{{#!bash
+# customize U-Boot env
+cat << EOF > /tmp/fw_env.config
+# Device        Device  offset
+$DEV    0xff0000        0x8000
+$DEV    0xff8000        0x8000
+EOF
+# get current config
+fw_printenv --config /tmp/fw_env.config > /tmp/uboot.env
+# append any desired config changes to /tmp/uboot.env
+echo "bootdelay=1" >> /tmp/uboot.env
+# write new config (perform twice so redundant env gets created as well)
+fw_setenv --config /tmp/fw_env.config --script /tmp/uboot.env
+fw_setenv --config /tmp/fw_env.config --script /tmp/uboot.env
+}}}
+
+
+ * provision a Newport board eMMC with a single ext4 root filesystem from a rootfs and kernel tarball:
+  1. Load and boot kernel+ramdisk buildroot image from TFTP server
+{{{#!bash
+tftpboot $kernel_addr_r Image && booti $kernel_addr_r - $fdtcontroladdr
+}}}
+  2. Partition /dev/mmcblk0 (emmc)
+{{{#!bash
+DEV=/dev/mmcblk0
+GUID_BASIC_DATA=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
+GUID_RESERVED=8DA63339-0007-60C0-C436-083AC8230908
+GUID_LINUXFS=0FC63DAF-8483-4772-8E79-3D69D8477DE4
+# first use parted to create a GPT as sgdisk does not like the default MBR based partition table
+parted --script $DEV mklabel gpt
+# use sgdisk to create partitions
+sgdisk -og $DEV
+sgdisk -n 1:2048:26623 -c 1:"fatfs" -t 1:$GUID_BASIC_DATA $DEV
+sgdisk -n 2:28672:30719 -c 2:"atf" -t 2:$GUID_RESERVED $DEV
+sgdisk -n 3:30720:32767 -c 3:"uboot" -t 3:$GUID_RESERVED $DEV
+sgdisk -n 4:32768:0 -c 4:"root" -t 4:$GUID_LINUXFS $DEV
+# set the legacy 'boot' attribute on rootfs partition
+sgdisk -A 4:set:2 $DEV
+# print partition table
+sgdisk -p $DEV
+}}}
+   - Note the partitioning scheme chosen above creates protective partitions around various portions of the boot firmware, namely the BDK bootstub, the fatfs filesystem, the ATF, and U-Boot (plus it's environment). This is both to protect any other partition tools from thinking there is free space there as well as provide you a partition to be able to easily update those portions of the boot firmware at a later date
+   - Note we set the rootfs partition's bit 2 attribute to mark the partition as BIOS bootable in case you are using the U-Boot generic distro config to look for bootscripts on bootable partitions
+   - Note the 'end sector' for the rootfs partition above is 0 meaning it will size to the end of the device
+  3. create and populate the rootfs partition
+{{{#!bash
+# fetch tar archives from network
+udhcpc -i eth0 # bring up networking
+cd /tmp
+wget http://dev.gateworks.com/ubuntu/focal/focal-newport.tar.xz
+wget http://dev.gateworks.com/newport/kernel/linux-newport.tar.xz
+# create and populate rootfs (P4) from tarballs
+PART=${DEV}p4
+mkfs.ext4 -q -F -L rootfs $PART
+mount $PART /mnt
+tar -C /mnt -xf focal-newport.tar.xz --keep-directory-symlink
+tar -C /mnt -xf linux-newport.tar.xz --keep-directory-symlink
+umount /mnt
+}}}
+  4. create a bootscript if desired (note this requires understanding of how U-Boot generic distro bootconfig works; you could alternately simply put whatever you need directly into U-boot env below)
+{{{#!bash
+cat <<\EOF > /tmp/bootscript
+echo "Newport Boot Script"
+setenv bootargs ${bootargs} root=/dev/mmcblk${devnum}p${distro_bootpart} rootwait
+# disable USB autosuspend (CN81xx errata)
+setenv bootargs ${bootargs} usbcore.autosuspend=-1
+# disable KPTI (expected chip errata)
+setenv bootargs ${bootargs} kpti=0
+# add console
+setenv bootargs ${bootargs} console=${console}
+# load and boot kernel
+echo "Loading kernel Image from ${devtype} ${devnum}:${distro_bootpart} ${prefix}"
+load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} ${prefix}Image
+booti ${kernel_addr_r} - ${fdtcontroladdr}
+EOF
+mount $PART /mnt
+mkimage -A arm64 -T script -C none -d /tmp/bootscript /mnt/boot/newport.scr
+umount /mnt
+}}}
+  5. customize U-Boot env if desired (note this requires understanding where the U-Boot env is located on FLASH)
+{{{#!bash
+# customize U-Boot env
+cat << EOF > /tmp/fw_env.config
+# Device        Device  offset
+$DEV    0xff0000        0x8000
+$DEV    0xff8000        0x8000
+EOF
+# get current config
+fw_printenv --config /tmp/fw_env.config > /tmp/uboot.env
+# append any desired config changes to /tmp/uboot.env
+echo "bootdelay=1" >> /tmp/uboot.env
+# write new config (perform twice so redundant env gets created as well)
+fw_setenv --config /tmp/fw_env.config --script /tmp/uboot.env
+fw_setenv --config /tmp/fw_env.config --script /tmp/uboot.env
+}}}
 
 == Ventana NAND flash based boards ==
@@ -133 +380 @@
 Notes:
  * you can always elect to build your own bootloader with a custom config rather than pulling the env data off a board
+
+== SPI / NOR FLASH based boards (Laguna) ==
+Products that use NOR and/or SPI flash have the ability to be uploaded to a host PC via the Gateworks GW16042 JTAG dongle and jtag_usb application.
+
+For these products simply configuring a board the way you want it at runtime then uploading the flash to a file provides you with a firmware image that can then be programmed onto other boards.
+
+Please read more about JTAG upload here: [wiki:jtag_instructions JTAG Instructions]