Provisioning boards

We refer to the act of installing a firmware image across multiple boards as Provisioning.

Provisioning always starts with creating an image from a build system and this varies greatly depending on your target system. Ultimately you end up building binary images that need to be programmed to the boot device at various locations. For example, boot firmware, firmware blobs, filesystem data etc.

You could stitch all these various images together into a 'disk image' or 'device image' and flash it as a single object however doing so on a particularly large flash device, say an 8GiB eMMC can take a long time depending on how much data you need to write.

FLASH programming methods and performance

The speed at which you can physically write to a FLASH device depends on the method you are using to transfer and write the data as well as the performance of the SoC's access to the device.

You have a few options available to write to the FLASH boot device:

• Gateworks JTAG programmer (extremely low transfer data rate)
• U-Boot (much higher data rate depending on driver capability in U-Boot)
• Linux (best data rate)

Both the transfer and write data rate are important. JTAG offers an extremely low transfer data rate but is an extremely useful method for getting <32MiB boot firmware and/or provisioning firmware on a board in an extremely reliable fashion. The board does not need to have functioning or compatible firmware to begin with.

Note that the performance varies greatly across product family as well. The Venice IMX8MM product family for example has much higher MMC write performance than the older Newport CN803X product family. These considerations as well as your image contents and size need to be taken into account when you decide upon your provisioning method.

U-Boot offers ok transfer rates and write performance (depending on driver support in U-Boot and noting that U-Boot is not an OS) but can't be used if the firmware on the board if corrupt, blank, or if the tools and features you need for provisioning do not exist in U-Boot. Note that because Gateworks ships all boards with U-Boot and a Linux based OS you can be guaranteed to boot to a U-Boot prompt with a reliable set of tools and features.

Linux offers the best transfer rate and write performance with the notion that it is an interrupt capable full operating system but also requires that you are able to boot to Linux or a Linux that has the tools and features needed for your provisioning needs. Note that Because Gateworks ships all boards with U-Boot you at least know you can start at that point to load a version of Linux supporting whatever provisioning tools you need even if the default Linux OS may not have what you need.

Summary of Pros and Cons of FLASH programming methods:

• JTAG:
  • Pro: no dependence on booting firmware already on the boot device (FLASH can be corrupt/blank)
  • Con: slow transfer rate
• U-Boot:
  • Pro: minimal dependence on boot firmware
  • Cons:
    • must have U-Boot pre-programmed on board
    • performance maybe sub-par compared to Linux
    • possible unfamiliarity of U-Boot commands and features
    • possible lack of features and flexibility
• Linux:
  • Cons:
    • more dependence and work in setting up a provisioning system
  • Pros:
    • best in class transfer and write performance
    • vast variety and familiarity of tools

Using compressed disk images

While Gateworks often distributes single binary compressed firmware images for options such as OpenWrt and Ubuntu compatible images, we take care to keep these images small so that the time to flash them is minimal in order to be able to easily flash them with U-Boot. This very well may not be obtainable in your situation especially if you end up wanting multiple partitions with large spaces between them.

The compressed disk image method involves:

• creating an uncompressed image that includes things like your boot firmware, partition table, and partition images
• stitching all these together putting each binary image at its correct offset using a tool like dd
• compressing this with gzip
• optionally growing your partition and filesystem to fit the extents of the device at runtime on the first boot

Pros of compressed disk image method:

• single file compressed image distribution
• easy to install via U-Boot (obtaining the image via tftp or removable storage)

Cons of compressed disk image method:

• the compressed image needs to fit into available RAM (or a complicated method of splitting it be used)
• can be slow if your uncompressed image is large (ie over 100's MiB's)
• may require resizing the partition(s) and filesystem(s) per your needs

Note that in order to keep the time required to install the Gateworks Ubuntu rootfs images for example, we create a filesystem just large enough to fit the rootfs meaning (appx 1.6GiB) meaning that is all we have to write. Upon first boot a script runs that resizes the rootfs partition and filesystem to fit the extents of the device. This not only minimizes the amount of data that must be written but allows the disk image to fit into large devices and provide the user with expansion space without any further actions needed.

Some performance examples:

• Gateworks Venice GW7301-01 IMX8MM installing Ubuntu
  • Note that IMX8MM supports HS400 eMMC data rates
  • U-Boot:
    • 14 seconds to transfer 460MiB compressed image via tftp to RAM at GbE
    • 45 seconds to uncompress and write the 1.6GiB of data to eMMC at HS400 speeds
• Gateworks Newport GW6404 CN8031 installing focal-newport.img.gz
  • Note that CN803x supports a max of 50MHz MMC data rates
  • U-Boot:
    • 58 seconds to transfer 460MiB compressed image via tftp to RAM at GbE
    • 2 minutes to uncompress and write the 1.6GiB of data to eMMC at 50Mhz speeds

To present an example of where this method might not scale well consider a scenario where you multiple partitions defined across an 8GiB device. Using this method you would need to write almost the entire 8GiB which would take appx 10 minutes using this method. If your board has a 64GiB eMMC device you can see how this can get out of hand fairly quickly.

Hybrid approaches

There are hybrid approaches you may consider as well. You could choose to create a JTAG'able image that contained custom firmware that automatically boots into a provisioning system that uses Linux for example. This JTAG'able image could be small enough to flash quickly over JTAG and you could even elect to have Gateworks pre-program your image on your boards if you have a Gateworks special or custom build. In this situation for example your image could pull a script from the network to complete your provisioning so that the image programmed on your boards can stay consistent but provide you flexibility in modifying the instructions for provisioning later on. This is the most flexible approach.

Another example of a hyrid approach is to use U-Boot to provision images based on a script of U-Boot commands such that you can program various portions of your disk image at a time. For example if you had a scenario where you have multiple filesystems spread across a large eMMC device you could fetch and program the boot-firmware, partition table, and each filesystem independently keeping the data actually written to your FLASH device as small as possible. While this approach could be done in U-Boot alone, there are likely still multiple advantages to booting a Linux ramdisk image to provide greater flexibility and/or familiarity with tools.

Provisioning from a Linux ramdisk environment

By far the most flexible solution is being able to provision your boards live with a Linux environment booting to a ramdisk providing all the tools you need for your custom environment.

The most successful provisioning environments Gateworks has seen in the past are those that allow you to control your provisioning environment yourself in-dependent of what firmware may be pre-installed on your boards. While having a Gateworks special or Gateworks custom board allows you to dictate what firmware you want programmed on your boards by keeping that firmware as simple as possible you have an advantage of either not needing custom firmware or not having to change that firmware as new provisioning needs arise. For example,

While this is the most flexible it is also the most complicate to setup.

A very easy way to build a minimal ramdisk Linux based OS is to use buildroot. All you need is basic kernel support for the target board and a minimal set of tools:

• minimal root filesystem (only tools you need for your provisioning needs)
• minimal kernel config (only drivers/features you need for your

provisioning needs)

• ramdisk (build artifact will be a 'Image' which is a kernel + ramdisk)
• script that performs the provisioning
• init config that runs the script on boot (if you wanted it automated)

The tools you need in your minimal rootfs:

• partition editor (sgdisk and/or parted)
• filesystem creation tools (ie e2fsprogs if using ext2/3/4)
• u-boot-tools (fw_printenv/fw_setenv/mkenvimage) if setting or altering uboot env
• networking support to fetch your filesystem tarballs (which could also come from removable storage)
• optionally your provisioning script (or you can transfer this via network)

The pre-built buildroot kernel+ramdisk minimal images that Gateworks provides at ​http://dev.gateworks.com/buildroot/ can be used for such provisioning. The instructions on how to build such an image can be found on the #buildroot page.

When performing this provisioning, just like in creating a disk image, you need to understand what partitions may be needed by boot firmware (for example Newport expects a FATFS partition that exists within its boot firmware) and what Linux device you want to provision (ie eMMC). When we create disk partitions below we also take care to create reserved partitions to cover the boot firmware. It is also important to understand what, if any, bootscript you want to install and where.

Examples:

• provision a Venice board eMMC with a single ext4 root filesystem from a rootfs and kernel tarball:
  1. Load and boot kernel+ramdisk buildroot image from TFTP server

     tftpboot $kernel_addr_r Image && booti $kernel_addr_r - $fdtcontroladdr
     

  2. Partition /dev/mmcblk0 (emmc)

     DEV=/dev/mmcblk0
     GUID_BASIC_DATA=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
     GUID_RESERVED=8DA63339-0007-60C0-C436-083AC8230908
     GUID_LINUXFS=0FC63DAF-8483-4772-8E79-3D69D8477DE4
     # use sgdisk to create partitions
     sgdisk -og $DEV
     sgdisk -n 1:66:32767 -c 1:"boot" -t 1:$GUID_RESERVED $DEV
     sgdisk -n 2:32768:0 -c 2:"root" -t 2:$GUID_LINUXFS $DEV
     # set the legacy 'boot' attribute on rootfs partition
     sgdisk -A 2:set:2 $DEV
     # print partition table
     sgdisk -p $DEV
     

     • Note the partitioning scheme chosen above creates protective partitions around the boot firmware which includes the SPL, ATF, and U-Boot (plus it's environment). This is both to protect any other partition tools from thinking there is free space there as well as provide you a partition to be able to easily update those portions of the boot firmware at a later date
     • Note we set the rootfs partition bit 2 attribute to mark the partition as BIOS bootable in case you are using the U-Boot generic distro config to look for bootscripts on bootable partitions
     • Note the 'end sector' for the rootfs partition above is 0 meaning it will size to the end of the device
  3. create and populate the rootfs partition

     # fetch tar archives from network
     udhcpc -i eth0 # bring up networking
     cd /tmp
     wget http://tharvey/tftpboot/focal-venice.tar.xz
     wget http://tharvey/tftpboot/linux-venice.tar.xz
     
     wget http://dev.gateworks.com/ubuntu/focal/focal-venice.tar.xz
     wget http://dev.gateworks.com/venice/kernel/linux-venice.tar.xz
     # create and populate rootfs (P2) from tarballs
     PART=${DEV}p2
     mkfs.ext4 -q -F -L rootfs $PART
     mount $PART /mnt
     tar -C /mnt -xf focal-venice.tar.xz --keep-directory-symlink
     tar -C /mnt -xf linux-venice.tar.xz --keep-directory-symlink
     umount /mnt
     

  4. create a bootscript if desired (note this requires understanding of how U-Boot generic distro bootconfig works; you could alternately simply put whatever you need directly into U-boot env below)

     cat <<\EOF > /tmp/bootscript
     echo "Venice Boot Script"
     # determine root device using uuid
     part uuid ${devtype} ${devnum}:${distro_bootpart} uuid
     # bootargs
     setenv bootargs console=$console root=PARTUUID=${uuid} rootwait $bootargs
     # load and boot kernel
     load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} ${prefix}Image &&
     booti ${kernel_addr_r} - ${fdtcontroladdr}
     EOF
     mount $PART /mnt
     mkimage -A arm64 -T script -C none -d /tmp/bootscript /mnt/boot/boot.scr
     umount /mnt
     

  5. customize U-Boot env if desired (note this requires understanding where the U-Boot env is located on FLASH)

     # customize U-Boot env
     cat << EOF > /tmp/fw_env.config
     # Device        Device  offset
     $DEV    0xff0000        0x8000
     $DEV    0xff8000        0x8000
     EOF
     # get current config
     fw_printenv --config /tmp/fw_env.config > /tmp/uboot.env
     # append any desired config changes to /tmp/uboot.env
     echo "bootdelay=1" >> /tmp/uboot.env
     # write new config (perform twice so redundant env gets created as well)
     fw_setenv --config /tmp/fw_env.config --script /tmp/uboot.env
     fw_setenv --config /tmp/fw_env.config --script /tmp/uboot.env
     

• provision a Newport board eMMC with a single ext4 root filesystem from a rootfs and kernel tarball:
  1. Load and boot kernel+ramdisk buildroot image from TFTP server

     tftpboot $kernel_addr_r Image && booti $kernel_addr_r - $fdtcontroladdr
     

  2. Partition /dev/mmcblk0 (emmc)

     DEV=/dev/mmcblk0
     GUID_BASIC_DATA=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
     GUID_RESERVED=8DA63339-0007-60C0-C436-083AC8230908
     GUID_LINUXFS=0FC63DAF-8483-4772-8E79-3D69D8477DE4
     # first use parted to create a GPT as sgdisk does not like the default MBR based partition table
     parted --script $DEV mklabel gpt
     # use sgdisk to create partitions
     sgdisk -og $DEV
     sgdisk -n 1:2048:26623 -c 1:"fatfs" -t 1:$GUID_BASIC_DATA $DEV
     sgdisk -n 2:28672:30719 -c 2:"atf" -t 2:$GUID_RESERVED $DEV
     sgdisk -n 3:30720:32767 -c 3:"uboot" -t 3:$GUID_RESERVED $DEV
     sgdisk -n 4:32768:0 -c 4:"root" -t 4:$GUID_LINUXFS $DEV
     # set the legacy 'boot' attribute on rootfs partition
     sgdisk -A 4:set:2 $DEV
     # print partition table
     sgdisk -p $DEV
     

     • Note the partitioning scheme chosen above creates protective partitions around various portions of the boot firmware, namely the BDK bootstub, the fatfs filesystem, the ATF, and U-Boot (plus it's environment). This is both to protect any other partition tools from thinking there is free space there as well as provide you a partition to be able to easily update those portions of the boot firmware at a later date
     • Note we set the rootfs partition's bit 2 attribute to mark the partition as BIOS bootable in case you are using the U-Boot generic distro config to look for bootscripts on bootable partitions
     • Note the 'end sector' for the rootfs partition above is 0 meaning it will size to the end of the device
  3. create and populate the rootfs partition

     # fetch tar archives from network
     udhcpc -i eth0 # bring up networking
     cd /tmp
     wget http://dev.gateworks.com/ubuntu/focal/focal-newport.tar.xz
     wget http://dev.gateworks.com/newport/kernel/linux-newport.tar.xz
     # create and populate rootfs (P4) from tarballs
     PART=${DEV}p4
     mkfs.ext4 -q -F -L rootfs $PART
     mount $PART /mnt
     tar -C /mnt -xf focal-newport.tar.xz --keep-directory-symlink
     tar -C /mnt -xf linux-newport.tar.xz --keep-directory-symlink
     umount /mnt
     

  4. create a bootscript if desired (note this requires understanding of how U-Boot generic distro bootconfig works; you could alternately simply put whatever you need directly into U-boot env below)

     cat <<\EOF > /tmp/bootscript
     echo "Newport Boot Script"
     setenv bootargs ${bootargs} root=/dev/mmcblk${devnum}p${distro_bootpart} rootwait
     # disable USB autosuspend (CN81xx errata)
     setenv bootargs ${bootargs} usbcore.autosuspend=-1
     # disable KPTI (expected chip errata)
     setenv bootargs ${bootargs} kpti=0
     # add console
     setenv bootargs ${bootargs} console=${console}
     # load and boot kernel
     echo "Loading kernel Image from ${devtype} ${devnum}:${distro_bootpart} ${prefix}"
     load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} ${prefix}Image
     booti ${kernel_addr_r} - ${fdtcontroladdr}
     EOF
     mount $PART /mnt
     mkimage -A arm64 -T script -C none -d /tmp/bootscript /mnt/boot/newport.scr
     umount /mnt
     

  5. customize U-Boot env if desired (note this requires understanding where the U-Boot env is located on FLASH)

     # customize U-Boot env
     cat << EOF > /tmp/fw_env.config
     # Device        Device  offset
     $DEV    0xff0000        0x8000
     $DEV    0xff8000        0x8000
     EOF
     # get current config
     fw_printenv --config /tmp/fw_env.config > /tmp/uboot.env
     # append any desired config changes to /tmp/uboot.env
     echo "bootdelay=1" >> /tmp/uboot.env
     # write new config (perform twice so redundant env gets created as well)
     fw_setenv --config /tmp/fw_env.config --script /tmp/uboot.env
     fw_setenv --config /tmp/fw_env.config --script /tmp/uboot.env
     

Ventana NAND flash based boards

Products that use NAND flash present an issue in that they can contain bad blocks. As a result the raw flash devices can differ in size making it difficult to implement a JTAG flash upload/download scenario.

There are several ways of provisioning NAND bootable boards:

• using JTAG (this is what Gateworks uses on our production line)
• using U-Boot (more complex, much faster than JTAG, but does not allow provisioning the SPL)
• using Linux (even more complex, much faster than JTAG, but allows provisioning the SPL)
• a combination of the above

Regardless of the method used for provisioning there are several artifacts that you need in order to provision NAND:

• SPL (secondary program loader)
• u-boot.img (bootloader)
• env (bootloader env)
• ubi (unsorted block image containing ubifs filesystem)

Pulling Software off of an Existing Board

SPL and Bootloader

The SPL and u-boot.img are built artifacts (which can be downloaded from ​​http://dev.gateworks.com/ventana/images).

Bootloader Environment

The env can be blank, which will use built-in defaults, or can be customized and extracted from the flash.

To create and extract a bootloader env:

1. Create the env on a board:

   # blank per-board vars (which are set from eeprom by default, yet overridable via env)
   setenv fdt_file
   setenv ethaddr
   setenv eth1addr
   # perform any other desired changes
   # save
   saveenv
   

2. Extract the env from the board and save to removable storage:
   • from Linux

     dd if=/dev/mtd1 of=env bs=1M
     

   • from U-Boot:

     # read the env (environment) partition into temporary memory, note the size reported below as 0x100000
     
     Ventana > nand read ${loadaddr} env
     
     NAND read: device 0 offset 0x1000000, size 0x100000
      1048576 bytes read: OK
     Ventana > 
     
     # store it to file on micro-SD with an ext4 fs (size re-used from above)
     mmc dev 0 && ext4write mmc 0:1 ${loadaddr} /env 0x100000
     
     # or store it to file on USB mass storage with an ext4 fs
     usb start && usb dev 0 && ext4write usb 0:1 ${loadaddr} /env 0x100000
     

   • Note that you may find it easier to build yourself a custom bootloader with defaults that match your needs rather than deal with extracting and imaging an env flash partition
   • Note that your ext4 filesystem must not have checksums enabled (metadata_csum, a feature added to newer e2fsprogs) as U-Boot does not support this in ext4write.

Root Filesystem

The ubi root filesystem is originally built by the build system of the specific BSP your using, however if you end up imaging this onto a board, and customizing it, you will not be able to bit-per-bit copy it to another board due to flash geometry changes and bad block mapping on the physical raw NAND parts.

Instead, you will have to boot a ramdisk based Linux rescue image, mount the ubifs and create a tarball of the files which you can then use to re-create a ubi on a Linux development host that can be applied to a new target board via U-Boot or Linux.

Procedure:

1. Boot a Linux rescue image that has networking and UBI/EXT4/FAT filesystem support. The buildroot based image is suitable for this. You will need to copy the uImage and dtb files from ​http://dev.gateworks.com/buildroot/ventana/minimal/ to a TFTP server on your network or a removable microSD or USB storage device and boot with one of the following:
   1. boot rescue image from network tftpserver

      setenv bootargs console=ttymxc1,115200
      setenv fsload tftpboot
      setenv bootdir ventana # set this to the prefix of your tftp dir
      run loadfdt && $fsload $loadaddr $bootdir/uImage && bootm $loadaddr - $fdt_addr
      

   2. boot rescue image from microSD (dev 0 partition 1) with ext/fat filesystem

      setenv bootargs console=ttymxc1,115200
      setenv fsload load mmc 0:1
      setenv bootdir . # set this to the prefix of your tftp dir
      run loadfdt && $fsload $loadaddr $bootdir/uImage && bootm $loadaddr - $fdt_addr
      

   3. boot rescue image from USB (dev 0 partition 1) with ext/fat filesystem

      setenv bootargs console=ttymxc1,115200
      setenv fsload load usb 0:1
      usb start
      setenv bootdir . # set this to the prefix of your tftp dir
      run loadfdt && $fsload $loadaddr $bootdir/uImage && bootm $loadaddr - $fdt_addr
      

2. Mount a filesystem on removable storage (you could use a tmpfs filesystem to do this on a ramdisk if you have enough space):

   mkdir /mnt/removable
   mount /dev/sda1 /mnt/removable
   

   • /dev/sd* would be for USB or mSATA devices and /dev/mmc* would be for microSD devices
3. Mount the ubifs:

   ubiattach /dev/ubi_ctrl -m 2 # attach /dev/mtd2 as /dev/ubi0
   mkdir -p /mnt/ubifs # create a mountpoint for it
   mount -t ubifs ubi0:rootfs /mnt/ubifs # mount 'rootfs' volume
   

   • Note that at this point you could create a tarball of the rootfs if you want to perfor the following steps later or on a Linux development host (tar --numeric-owner -cvzf /mnt/removable/rootfs.tar.gz -C /mnt/ubifs .)
4. Create a ubi image for the particular flash geometry of the target device (see wiki:linux/ubi#creatingubi):
   1. 'normal' geometry FLASH devices (2048 byte page size)

      # create a ubifs image
      mkfs.ubifs -F -m 2048 -e 124KiB -c 16248 -x zlib -o /mnt/removable/root.ubifs -d /mnt/ubifs
      # create a ubi image
      cat <<EOF > ubinize.cfg
      [ubifs]
      mode=ubi
      image=/mnt/removable/root.ubifs
      vol_id=0
      vol_type=dynamic
      vol_name=rootfs
      vol_flags=autoresize
      EOF
      # create ubi suitable for 2048byte page size and 128KiB block size devices
      ubinize -m 2048 -p 128KiB -o /mnt/removable/root.ubi ubinize.cfg
      

      • Note that the mkfs.ubifs process can take quite a bit of time with no output depending on the size of your filesystem; takes about 8 minutes for a 1GiB filesystem
   2. 'large' geometry FLASH devices (4096 byte page size)

      # create a ubif image
      mkfs.ubifs -F -m 4096 -e 248KiB -c 8124 -x zlib -o /mnt/removable/root.ubifs -d /mnt/ubifs
      # create a ubi image
      cat <<EOF > ubinize.cfg
      [ubifs]
      mode=ubi
      image=/mnt/removable/root.ubifs
      vol_id=0
      vol_type=dynamic
      vol_name=rootfs
      vol_flags=autoresize
      EOF
      # create ubi suitable for 4096byte page size and 256KiB block size devices
      ubinize -m 4096 -p 256KiB -o /mnt/removable/root.ubi ubinize.cfg
      

      • Note that the mkfs.ubifs process can take quite a bit of time with no output depending on the size of your filesystem; takes about 8 minutes for a 1GiB filesystem
5. Unmount the filesystems:

   umount /mnt/removable
   umount /mnt/ubifs
   ubidetach /dev/ubi_ctrl -m 2
   

Flashing Boards with Pulled Software

Once you have all the artifacts you can re-assemble them into a JTAG image suitable for the Gateworks JTAG adapter and software. The following usage of mkimage_jtag will create a jtagable image matching the partitioning described by 'mtdparts=nand:16m(uboot),1m(env),-(rootfs)'

mkimage_jtag -e SPL@0 u-boot.img@14M env@16M ubi@17M > image.bin

Or, for a faster two-step method of imaging using U-Boot with serial and ethernet (to a tftp server with the ubi):

1. create a JTAG image of the SPL + bootloader + env:

   mkimage_jtag SPL u-boot.img env > image.bin
   

2. once the above is flashed with the Gateworks JTAG adapter and software you can flash the ubi (much more quickly than via JTAG) within U-Boot
3. break out into the bootloader and load the ubi image and flash it:
   1. from network via tftp:

      setenv ipaddr 192.168.1.1 # local ip
      setenv serverip 192.168.1.146 # server ip
      tftp ${loadaddr} image.ubi # tftp ubi image
      nand erase.part rootfs # erase the nand partition named rootfs from the mdtparts variable
      nand write ${loadaddr} rootfs ${filesize} # write the downloaded ubi to rootfs
      

   2. from removable storage via msata (device 0 partition 1):

      sata init
      load sata 0:1 $loadaddr root.ubi
      nand erase.part rootfs
      nand write $loadaddr rootfs $filesize
      

4. from removable storage via USB (device 0 partition 1):

   usb start
   load usb 0:1 $loadaddr root.ubi
   nand erase.part rootfs
   nand write $loadaddr rootfs $filesize
   

5. from removable storage via microSD (device 0 partition 1):

   load mmc 0:1 $loadaddr root.ubi
   nand erase.part rootfs
   nand write $loadaddr rootfs $filesize
   

   • Note writing the nand may take some time; several minutes for a 600MB ubi

Notes:

• you can always elect to build your own bootloader with a custom config rather than pulling the env data off a board

U-Boot environment provisioning

The U-Boot bootloader uses environment variables and scripts to control the booting of a board. Often it is desirable to override the default environment the Gateworks BSP's create and create your own.

The device, location, size, and redundancy details of how the U-Boot environment is stored vary from board to board based on U-Boot's configuration:

• Venice: 32KiB (0x8000) redundant env stored at 16320KiB (0xff0000) offset for MMC
• Newport: 32KiB (0x8000) redundant env stored at 16320KiB (0xff0000) offset for MMC
• Ventana:
  • MMC: 128MiB redundant env stored at 709KiB offset
  • NAND: 128MiB redundant env stored at 16MiB offset

There are some tools from the U-Boot project available in the u-boot-tools package that can be used to alter or create an env that can then be copied directly to flash:

• fw_setenv / fw_getenv - requires a config file and can get or alter the env. If there is no valid env where the config file points to a default env hard coded into the bootloader will be returned.
• mkenvimage - create an env image from scratch with a specified size and redundancy.

For examples of how the default env is created for the various Gateworks products:

• Venice BSP uses fw_setenv to alter the hard-coded default env. For details see:
  • ​https://github.com/Gateworks/bsp-venice/blob/master/Makefile
  • ​https://github.com/Gateworks/bsp-venice/blob/master/venice.env
  • ​https://github.com/Gateworks/bsp-venice/blob/master/fw_env.config
• Newport BSP uses fw_setenv to alter the hard-coded default env. For details see:
  • ​https://github.com/Gateworks/bsp-newport/blob/sdk-10.1.1.0-newport/Makefile
  • ​https://github.com/Gateworks/bsp-newport/blob/sdk-10.1.1.0-newport/newport.env
  • ​https://github.com/Gateworks/bsp-newport/blob/sdk-10.1.1.0-newport/fw_env.config
• Ventana firmware images have a blank env area therefore use hard-coded default env from U-Boot

To create a specific env from scratch (ie without inheriting any defaults) use the mkenvimage tool from the u-boot-tools package

1. Create an env.txt file with 'name=value' pairs. Be sure to include everything that is necessary as no defaults will be available other than variables that are set at runtime on the fly such as serial#, model, and macaddr:

   cat << EOF >> env.txt
   baudrate=115200
   bootcmd=run distro_boot
   ...
   EOF
   

   • Note you need to make sure this is complete and proper for your board - you should do a 'print' of a Gateworks pre-built image as a starting point then remove any variables that are board specific and created by U-Boot at runtime
2. Use mkenvimage taking care to specify the env size the bootloader is configured to use and specify '-r' for redundancy if the bootloader is configured for that:
   • Venice / Newport

     mkenvimage -r -s $((32 * 1024)) -o env.bin env.txt
     

   • Ventana:

     mkenvimage -r -s $((128 * 1024)) -o env.bin env.txt
     

3. Place this image in FLASH:
   • Venice / Newport:
     • Example: compressed disk image for Venice / Newport:

       gunzip disk.img.gz
       dd if=env.bin of=disk.img bs=1K seek=16320 oflag=notrunc
       gzip disk.img
       

     • Example: update from Linux:

       dd if=env.bin of=/dev/mmcblk0 bs=1K seek=16320 oflag=notrunc
       

     • Example: update from U-Boot (16320KiB is equivalent to 0x7f80 512byte blocks in hex)

       tftp $loadaddr env.bin && mmc write $loadaddr 0x7f80 $filesize
       

   • Ventana:
     • Example: JTAG binary image for a Ventana NAND device:

       mkimage_jtag -e SPL@0 u-boot.img@14M env.bin@16M ubi@17M > image.bin
       

     • Example: update from Linux:

       dd if=env.bin of=/dev/mmcblk0 bs=1K seek=16384 oflag=notrunc
       

     • Example: update from U-Boot (16384MiB is equivalent to 0x8000 512byte blocks in hex)

       tftp $loadaddr env.bin && mmc write $loadaddr 0x8000 $filesize
       

micro-SD provisioning

Directly Cloning SD Cards

See linux/blockdev

U-Boot MicroSD Provisioning

The main difference between provisioning removable storage devices such as micro-SD compared to non-removable storage devices (such as NAND flash) is that the removable devices can be potentially booted on a board with a different model, CPU, or memory configuration. This causes us to treat the U-Boot environment differently when we extract it from a configured board.

If you do not want a blank env (which uses built-in defaults) you must provision one board, boot it to the bootloader, customize your env, then extract that env to use when provisioning additional boards.

The Ventana bootloader stores its microSD env on raw block sectors from offset 709K and is 256K in size.

The env can be blank, which will use built-in defaults, or can be customized and extracted.

To create and extract a bootloader env: (only if this is being used, otherwise skip this step)

1. Create the env on a board:

   # blank per-board vars (which are set from eeprom by default, yet overridable via env)
   setenv fdt_file
   setenv ethaddr
   setenv eth1addr
   # perform any other desired changes
   # save
   saveenv
   

2. extract the env from a board that boots to micro-SD:
   • from Linux:

     # copy 256KB from offset 709KB to 'env' file
     dd if=/dev/sdc of=env bs=1K skip=709 count=256 oflag=sync
     

   • from U-Boot:

     mmc read ${loadaddr} 0x58a 0x200 # read 512x 512byte blocks (256K) from block 0x1418
     # store it to file on micro-SD with an ext4 fs
     ext4write mmc 0:1 ${loadaddr} /mmc.env 0x40000
     # store it to file on USB mass storage with an ext4 fs
     usb start && usb dev 0 && ext4write usb 0:1 ${loadaddr} /mmc.env 0x40000
     

   • Note that your ext4 filesystem must not have checksums enabled (metadata_csum, a feature added to newer e2fsprogs) as U-Boot does not support this in ext4write.

To place an extracted env onto a micro-SD:

• from Linux:

  # copy 256KB from file env to offset 709KB:
  dd if=env of=/dev/sdc bs=1K seek=709 count=256 oflag=sync
  

Provisioning root file system from live Newport board

It's always best to create your rootfs from scratch using the Newport BSP when this option is available. If you're prevented from doing so it is possible to provision the root file system from some Newport boards without too much difficulty. Newport SBC's are capable of booting from both eMMC and MMC. The boot device can be selected by the GSC performing a 5x press on the power button. In the following section we will create a bootable microSD card, load it as the primary boot device, image partition 2 of the eMMC using "dd", then create a compressed disk image from this data.

Requirements

• A Newport board with MMC card slot and push button (GSC must be configured to accept push button input—this is default)
• A MMC with capacity enough to accommodate full size of eMMC (8GB) along with BSP for performing recovery. Recommended would be a minimum 16GB or 32GB card.
• A desktop computer with Linux natively installed. This workstation must have a drive capable of accepting an MMC and have packages for DD and Mount.

On your workstation

Download a Gateworks pre-built image. All of our images include the tools necessary to perform this operation, though Ubuntu will likely be the easiest to work with.

wget -N http://dev.gateworks.com/newport/images/focal-newport.img.gz

Insert a micro SD card into your workstation and identify the device name it's been assigned. This can be done using a variety of methods, for example the "dmesg" output. In this example the MMC is "/dev/sdb"

Image the MMC.

zcat focal-newport.img.gz | sudo dd of=/dev/sdb bs=4M

Remove the SD card from your workstation.

On your Newport SBC

Insert the SD card into the SD card reader, then apply power to the SBC (or otherwise turn it on). 5x press the power button, you will see the status LED turn off and back on. The BDK will display microSD as MMC0.

Example output:

	Gateworks Newport SPL (12.7.0-96865d0 Tue Jul 7 21:19:32 UTC 2020)

	GSC     : v55 0xe7e2 RST:BOOT_WDT2 Thermal Protection Enabled
	Temp    : Board:34C/86C CPU:42C/100C
	Model   : GW6400-B1
	MFGDate : 09-23-2019
	Serial  : 802864
	RTC     : 8
	SoC     : CN8020-800BG676-SCP-P12-G 1024KB 800/550MHz 0xa2 Pass 1.2 
	MMC0    : microSD
	MMC1    : eMMC

Alterntitively you can use a ramdisk (performance will be faster). ​Link to Ramdisk section

Proceed with booting to Linux user-space. Once there if you would like to view the rootfs you're about to image it's located at /dev/mmcblk1p2. Doing so is optional.

mount /dev/mmcblk1p2 /mnt

ls /mnt #this will display the eMMC's rootfs

umount /mnt

Image the partition to a file.

dd if=/dev/mmcblk1p2 of=myrootfs.img

"sync" the filesystem and power board off

sync
#remove power

Remove MMC from the SBC.

Create an image

Navigate to the "/tmp" directory on your workstation. In actuality this can be any directory that you have read and write permissions, "/tmp" should only be used if you don't care what happens to these files later.

cd /tmp

Insert the MMC into the SD card reader on your workstation, as before note the the name the device is assigned. Because we created our image from within the rootfs we will need to mount the second partition, for example "/dev/sdb2":

sudo mount /dev/sdb2 /mnt/

ls /mnt/

Copy the image you created of mmcblk1p2 into "/tmp"

cp /mnt/myrootfs.img .

Executing the command "file" on this file will return that it is ext4 filesystem data and the volume name is "rootfs".

Example:

	user@workstation:/tmp$ file myrootfs.img 
	myrootfs.img: Linux rev 1.0 ext4 filesystem data, UUID=951f19a1-cc54-4a07-9b7f-41a54ea8acb4, volume name "rootfs" (needs journal recovery) (extents) (64bit) (large files) (huge files)

Add boot firmware and create a gzipped image.

• Download boot firmware

  wget http://dev.gateworks.com/newport/boot_firmware/firmware-newport.img
  

• name it however you please

  cp firmware-newport.img myimg-newport.img
  

• Create a disk image containing the root filesystem and boot firmware. The boot firmware is 16M so we will "dd" the rootfs using this offset.

  dd if=myrootfs.img of=myimg-newport.img bs=16M seek=1
  

• gzip the image so it can be installed using standard methods on other Newport boards.

  gzip -k -f myimg-newport.img 
  

Be mindful that the image size can't exceed the total DRAM of the board you plan to install it on if you're using the bootloader command "tftpboot". The "update_all", and "update_rootfs" scripts both use this command

ls -lh myimg-newport.img.gz #size must be less than total DRAM of board

Flash this image to a Newport SBC:

In the bootloader:

GW6400-B1> setenv ipaddr 192.168.1.52
GW6400-B1> setenv serverip 192.168.1.56
GW6400-B1> setenv image myimg-newport.img.gz
GW6400-B1> setenv dev 0
GW6400-B1> run update_all 

Provisioning Newport, direct write to eMMC on target

Usint the bootloader's "gzwrite" method for flashig embedded flash has a lot of downsides; the gzipped image must be smaller than the capacity of the ram and it's difficult to create a minimal filesystem to speed up the writes. Depending on your application a more flexible and potentially time efficient aproach would be to use 'linux kernel+ramdisk + provisioning' script approach. Using this method may be neceessary if there are scripts or applications that must be run during the provisioning process.

• Buildroot Ramdisk Wiki

1. Boot to Target - either from the alternate boot device (ie boot from microSD if working on eMMC or visa versa) or boot a kernel+ramdisk so that you can work off either device. We provide a pre-built ramdisk image with all the necessary tools. A secondary boot media can be use although it will not be as fast as the ramdisk.
2. Partition the device with a partition size just large enough to fit a filesystem that can contain your files. Determine the size in bytes needed for your filesystem, there are several ways to do this including simple trial and error. If you have a directory with your files you can use 'du -b --summarize' to get bytes. For a tar.gz a good estimate is the uncompressed size "gzip -l <file.gz>".

   # for an existing directory using du
   SZ_B=$(du -b --summarize /usr/src/ubuntu-rootfs/focal-newport | awk '{
   print $1; }'
   # or just guess and use trial and error
   SZ_B=$((1536*1024*1024)) # 1536MiB fits the Gateworks ubuntu image
   

3. Create partition table Example using GPT and sgdisk. For Newport we need to preserve the FATFS filesystem used by the boot firmware.

   DEV=/dev/mmcblk0
   GUID_BASIC_DATA=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
   GUID_RESERVED=8DA63339-0007-60C0-C436-083AC8230908
   GUID_LINUXFS=0FC63DAF-8483-4772-8E79-3D69D8477DE4
   # first use parted to create a GPT as sgdisk does not like the default
   MBR based partition table
   parted --script $DEV mklabel gpt
   # use sgdisk to create partitions
   sgdisk -og $DEV
   sgdisk -n 1:2048:26623 -c 1:"fatfs" -t 1:$GUID_BASIC_DATA $DEV
   sgdisk -n 2:28672:30719 -c 2:"atf" -t 2:$GUID_RESERVED $DEV
   sgdisk -n 3:30720:32767 -c 3:"uboot" -t 3:$GUID_RESERVED $DEV
   END=$(( 32768 + $((SZ_B/512)) ))
   sgdisk -n 4:32768:$END -c 4:"root" -t 4:$GUID_LINUXFS $DEV
   # set the legacy 'boot' attribute on rootfs partition so the uboot env
   will boot from it
   sgdisk -A 4:set:4 $DEV
   # print partition table
   sgdisk -p $DEV
   

4. Create your filesystem using the index of the partition for root as shown above. Note that rootfs is p4, this is because of the boot firmware (atf/uboot). Format the partition that will contain the rootfs to ext4.

   mkfs.ext4 -q -F -L rootfs ${DEV}p4
   

5. Mount your filesystem, populate it, then umount.

   mount ${DEV}p4 /mnt
   cd /tmp
   udhcpc -e eth0
   wget http://dev.gateworks.com/ubuntu/focal/focal-newport.tar.xz
   wget http://dev.gateworks.com/newport/kernel/linux-newport.tar.xz
   tar -C /mnt -xf focal-newport.tar.xz --keep-directory-symlink
   2>/dev/null || echo Error!!!
   tar -C /mnt -xf linux-newport.tar.xz --keep-directory-symlink
   2>/dev/null || echo Error!!!
   umount /mnt
   #check for errors returned from tar such as not enough space
   

Note: When creating tarballs use the --numeric-owner to preserve UID/GID numbers vs names and there is no need to be selective about files/directories on a non-live filesystem

6. Now you can dd and decompress your image up to the last block in the partition. The last block in the partition is $END which you can see by printing the partition table:

   rm /tmp/*
   dd if=$DEV bs=512 count=$END | gzip > /tmp/disk.img.gz
   

7. Install a compressed disk image:

   cd /tmp
   wget disk.img.gz
   zcat disk.img.gz | dd of=$DEV bs=4M
   

Provisioning Newport removable block storage on Desktop workstation

Provisioning on the target is less than ideal in situations where a traditional workstation can be used. Both methods have their advantages and disadvantges. When using a desktop an issue may arrise when using partitioning tools, as most of them do not like working on 'files' and won't let you do things like partition past the size of the file. In those cases there are, and we provide, tools that create partition tables for you without adhering to any rules.

1. Create a 'file' for your filesystem image. 'truncate' can be used as it creates a 'sparse' file which reports the size it was created with but won't take up space for zero'd blocks.

   truncate -s 1536M fs.img #create a sparse file 
   mkfs.ext4 -q -F -L rootfs fs.img #format it.  
   

2. Mount your filesystem image and populate it:

   sudo mount fs.img /mnt/disk
   wget http://dev.gateworks.com/ubuntu/focal/focal-newport.tar.xz
   wget http://dev.gateworks.com/newport/kernel/linux-newport.tar.xz
   sudo tar -C /mnt/disk/ -xf focal-newport.tar.xz --keep-directory-symlink >/dev/null || echo "Error!!!"
   sudo tar -C /mnt/disk/ -xf linux-newport.tar.xz --keep-directory-symlink >/dev/null || echo "Error!!!"
   sudo umount /mnt/disk
   

3. Create a 'file' for your disk image. This needs to be large enough for your boot firmware (which includes the partition table) as well as the filesystems you will copy to it.

   truncate -s $((16+1536))M disk.img
   

4. Copy the boot firmware to it at offset 0

   dd if=firmware-newport.img of=disk.img conv=notrunc #make sure to use conv=notrunc so it doesn't truncate your image file
   

5. Copy your filesystem image to the correct sectors dictated by the partition table. If needed adjust/re-create the partition table taking care to ensure the boot firmware FATFS does not move.

   dd if=fs.img of=disk.img bs=512 seek=32768 conv=notrunc
   

6. Compress.

   gzip disk.img
   

In conclusion, a partition is simply a chunk of space dictated by a header in the partition table which is in the first few blocks of the disk. It doesn't need to be filled, there can be a large partition with a small filesystem inside of it, that filesystem can be enlarged on first-boot with 'resize2fs'. Additionally the partition size can be incresead as well.