wiki:linux/ubi

  1. Unsorted Block Images (UBI)
  2. FLASH chip geometry
  3. UBI/UBIFS use on Gateworks Ventana
    1. Creating a UBI/UBIFS using existing rootfs, kernel, and device-tree blobs
      1. 'normal' geometry FLASH devices (2048 byte page size)
      2. 'large' geometry FLASH devices (4096 byte page size)
      3. Bad blocks
    2. Working with a UBI in Linux OS (extracting or manipulating contents)
      1. On a (NAND-less) Linux Development Host using nandsim
      2. On a target board with NAND FLASH
    3. Working with UBI in U-boot Bootloader
      1. Updating NAND FLASH with a UBI image
      2. Updating NAND FLASH with a ubifs image
      3. Updating UBI with low RAM devices
        1. Split UBIFS flashing
        2. Split UBI flashing
    4. UBI Fastmap

Unsorted Block Images (UBI)

Unsorted block images (UBI) is an erase block management layer for flash memory devices such as raw NAND devices. UBI serves two purposes:

  • tracking NAND flash bad blocks
  • providing wear leveling

UBI was written specifically for UBIFS so that the filesystem does not have to deal with these issues. UBIFS is a successor to JFFS2 and a competitor to LogFS.

References:

FLASH chip geometry

Because UBI takes care of bad block tracking and wear leveling, it needs to know some details about the geometry of the raw FLASH device when creating an image.

The geometry data can be found in the datasheet for the specific flash device used. Gateworks uses a variety of NAND Flash devices that fall within two different unique geometries we refer to as 'normal' and 'large' which references the erase block size:

NAND Part NAND Size Page Size (Bytes) Block Size Logical Erase Block Size Blocks READ_ID Bytes Gateworks Geometry Name
Cypress S34ML16G2 2GiB 2048 128KiB 124KiB 16384 0x01 0xd5 0xd2 0x95 normal
Micron MT29F2G08 256MiB 2048 128KiB 124KiB 2048 0x2c 0x90 0x95 0x06 normal
Micron MT29F16G08 2GiB 4096 256KiB 248KiB 8192 0x2c 0xd5 0xd1 0xa6 large
Micron MT29F8G08 1GiB 4906 256KiB 248KiB 4096 0x2c 0xdc 0x00 0x15 large
  • The 'Page Size', 'Block Size', and max blocks are specified by the NAND datasheet. The Logical Erase Block size can be calculated as: (Block Size) - 2 * (Page Size)

The geometry can also be determined at runtime in U-Boot using nand info and Linux using sysfs:

  • Cypress S34ML16G202 2048MiB FLASH:
    • U-Boot: 
      Ventana > nand info

Device 0: nand0, sector size 128 KiB
  Page size       2048 b
  OOB size         128 b
  Erase size    131072 b
  subpagesize     2048 b
  options     0x40000200
  bbt options 0x    8000
      • U-Boot above shows a 2048 byte page size and a 128KiB erase block size which dictates a 124KiB logical erase block size
    • Linux (once device is attached via 'ubiattach /dev/ubi_ctrl -m 2' or via 'root=ubi ubi=2' kernel cmdline params: 
      # cat /sys/devices/virtual/ubi/ubi0/min_io_size 
2048
# cat /sys/devices/virtual/ubi/ubi0/eraseblock_size 
126976
# cat /sys/devices/virtual/ubi/ubi0/total_eraseblocks 
16248
      • above shows a 2048 byte page size, a 124KiB logical erase block size (which dictates a 128KiB physical erase block size) and a max of 16248 total erase blocks available in this ubi for a total usable space of 16248*124KiB still available for use by ubi volumes
  • Micron MT29F16G08 2048MiB FLASH:
    • U-Boot: 
      Ventana > nand info

Device 0: nand0, sector size 256 KiB
  Page size       4096 b
  OOB size         224 b
  Erase size    262144 b
  subpagesize     4096 b
  options     0x40000200
  bbt options 0x       0
      • Note the 4096 byte page size and 256KiB erase block size which dictates a 248KiB logical erase block size
    • Linux (once device is attached via 'ubiattach /dev/ubi_ctrl -m 2' or via 'root=ubi ubi=2' kernel cmdline params: 
      # cat /sys/devices/virtual/ubi/ubi0/min_io_size 
4096
# cat /sys/devices/virtual/ubi/ubi0/eraseblock_size 
253952
# cat /sys/devices/virtual/ubi/ubi0/total_eraseblocks 
8124
      • above shows a 4096 byte page size, a 248KiB logical erase block size (which dictates a 256KiB physical erase block size) and a max of 8124 total erase blocks available in this ubi for a total usable space of 253952*248KiB still available for use by ubi volumes
  • Micron MT29F2G08 256MiB FLASH:
    • U-Boot: 
      Ventana > nand info

Device 0: nand0, sector size 128 KiB
  Page size       2048 b
  OOB size          64 b
  Erase size    131072 b
  subpagesize     2048 b
  options     0x40000200
  bbt options 0x    8000
      • U-Boot aqbove shows a 2048 byte page size and a 128KiB erase block size which dictates a 124KiB logical erase block size
    • Linux (once device is attached via 'ubiattach /dev/ubi_ctrl -m 2' or via 'root=ubi ubi=2' kernel cmdline params: 
      # cat /sys/devices/virtual/ubi/ubi0/min_io_size 
2048
# cat /sys/devices/virtual/ubi/ubi0/eraseblock_size 
126976
# cat /sys/devices/virtual/ubi/ubi0/total_eraseblocks 
1912
      • above shows a 2048 byte page size, a 124KiB logical erase block size (which dictates a 128KiB physical erase block size) and a max of 126976 total erase blocks available in this ubi for a total usable space of 126976*124KiB still available for use by ubi volumes

mkfs.ubifs important arguments:

  • -r Build file system from directory DIR
  • -F The -F option causes mkfs.ubifs to set a special flag in the superblock, which triggers a "free space fixup" procedure in the kernel the very first time the filesystem is mounted. This fixup procedure involves finding all empty pages in the UBIFS file system and re-erasing them. This ensures that NAND pages which contain all 0xFF data get fully erased, which removes any problematic non-0xFF data from their OOB areas.
  • -m Minimum I/O unit size (this is the Page Size)
  • -e Logical erase block size (UBI requires 2 minimum I/O units out of each Physical Erase Block (PEB) for overhead thus this is the 'Block Size' minus 2x the Page Size
  • -c Maximum logical erase block count (Number of Blocks; use the largest of the chips in the same geometry in order to create a ubifs that works on all the above NAND devices)
  • -x Compression type - "lzo", "favor_lzo", "zlib" or "none" (default: "lzo")
  • -o Output to FILENAME
  • -h Help, provides full list of options

ubinize important arguments:

  • -o Output file
  • -p Size of the physical eraseblock of the flash in bytes, KiB, or MiB
  • -m Minimum input/output unit size of the flash in bytes (this is the Page Size)

UBI/UBIFS use on Gateworks Ventana

UBI is used on the Gateworks Ventana for raw NAND storage. Other Gateworks Product families used NOR FLASH and eMMC.

The Gateworks Ventana default U-Boot environment scripts make a few assumptions about booting from NAND flash:

  • the mtdparts env var partitions the NAND device into sections for uboot (16MB as this accounts for the SPL, u-boot.img and redundancy for bad blocks), u-boot env (1MB), rootfs (the remainder of the device)
  • the 'rootfs' partition (defined by the mtdparts env var) contains a UBI device/image
  • the UBI device/image in the rootfs partition contains a volume also called 'rootfs'
  • an optional bootscript may exist in boot/6x_bootscript-ventana (the image env var) and if found will be sourced
  • a flattened device tree blob (dtb) may exist in boot/ (named by the automatic env vars fdt_file, fdt_file1, or fdt_file2)
  • a kernel may exist in boot/uImage

The above assumptions are not requirements - the U-Boot env scripts can be changed to suit your needs.

The mtd-utils Ubuntu package provides the ubninize and mkfs.ubifs utilities you need to work with ubi images: 

sudo apt install mtd-utils

Creating a UBI/UBIFS using existing rootfs, kernel, and device-tree blobs

To create a UBI image suitable for booting on a standard Gateworks Ventana bootloader (taking into account the assumptions above) you can do the following on a Linux system provided you have a root filesystem (myrootfs.tar.gz), kernel (uImage), and device-tree dtb files (imx6*-gw*.dtb).

'normal' geometry FLASH devices (2048 byte page size)

To create a UBI image suitable for raw NAND FLASH devices with a 2048 byte page size:

  1. create a temporary directory for our rootfs and untar your filesystem to it 
    mkdir tmp
sudo tar --numeric-owner -xvf myrootfs.tar.gz -C tmp

of files that you have in your rootfs such as root owned /dev files

  1. copy your kernel files (and u-boot bootscript if you have one): 
    cd tmp/
wget http://dev.gateworks.com/ventana/images/linux-ventana.tar.xz
tar -xvf linux-ventana.tar.xz --keep-directory-symlink
depmod $(ls /lib/modules/) # create module dependencies
rm linux-ventana.tar.xz
cd ..
  2. create a ubifs image for the 2048 byte page size 'normal' device geometry: 
    mkfs.ubifs -F -m 2048 -e 124KiB -c 16248 -x zlib -o root.ubifs -d tmp
  • the -c parameter (max-peb-count) of 16248 assures that the filesystem can be sized up to 16248*124KiB or 1.9GiB provided there is remaining space in the ubi
  • the -F parameter sets the free-space-fixup flag causing the filesystem to be resized on mount not to exceed the max-peb-count argument passed with the -c parameter (assuming the volume is dynamic and has blocks remaining)
  1. use ubinize to create a ubi image (which can be written to a suitable raw NAND device in U-Boot using nand write or in Linux via ubiformat): 
    cat <<EOF > ubinize.cfg
[ubifs]
mode=ubi
image=root.ubifs
vol_id=0
vol_type=dynamic
vol_name=rootfs
vol_flags=autoresize
EOF
# create ubi suitable for 2048byte page size and 128KiB block size devices
ubinize -m 2048 -p 128KiB -o image.ubi ubinize.cfg

'large' geometry FLASH devices (4096 byte page size)

To create a UBI image suitable for raw NAND FLASH devices with a 4096 byte page size:

  1. create a temporary directory for our rootfs and untar your filesystem to it 
    mkdir tmp
sudo tar -xvf myrootfs.tar.gz -C tmp
  2. copy your kernel files (and u-boot bootscript if you have one): 
    cd tmp/
wget http://dev.gateworks.com/ventana/images/linux-ventana.tar.xz
tar -xvf linux-ventana.tar.xz --keep-directory-symlink
depmod $(ls /lib/modules/) # create module dependencies
rm linux-ventana.tar.xz
cd ..
  3. create ubifs images for the 4096 byte page size 'large' device geometry: 
    sudo mkfs.ubifs -F -m 4096 -e 248KiB -c 8124 -o root.ubifs -d tmp
    • the -c parameter (max-peb-count) of 8124 assures that the filesystem can be sized up to 8124*248KiB or 1.9GiB provided there is remaining space in the ubi
    • the -F parameter sets the free-space-fixup flag causing the filesystem to be resized on mount not to exceed the max-peb-count argument passed with the -c parameter (assuming the volume is dynamic and has blocks remaining)
  4. use ubinize to create ubi image (which can be written to a suitable raw NAND device in U-Boot using nand write or in Linux via ubiformat: 
    cat <<EOF > ubinize.cfg
[ubifs]
mode=ubi
image=root.ubifs #generate this image with mkfs.ubifs
vol_id=0
vol_type=dynamic
vol_name=rootfs
vol_flags=autoresize
EOF
# create ubi suitable for 4096byte page size and 256KiB block size devices
ubinize -m 4096 -p 256KiB -o image.ubi ubinize.cfg

Bad blocks

NAND flashes have a random amount of initial bad blocks. This means that different devices may have slightly different volume sizes (especially if the UBI auto-resize feature is used).

Working with a UBI in Linux OS (extracting or manipulating contents)

Linux has support for ubi:

  • mounting a ubi
  • creating/renaming/removing ubi volumes
  • reading/writing ubi volumes

Working with UBI and UBIFS volumes on Linux require use of the following:

  • tools (from mtd-utils package):
    • ubiattach
    • ubidetach
    • ubiformat
    • ubirename
    • ubiupdatevol
    • ubinfo
    • ubimkvol
    • ubirmvol
  • kernel support:
    • mtd (the MTD layer)
    • mtdblock (the block layer over MTD)
    • ubi (ubi support)
    • ubifs (ubi filesystem support)
    • nandsim (if you want to simulate a NAND device on a development host)

When using the above tools the MTD partition must not be mounted by your Linux OS meaning you either have to:

Note that the creation of ubifs and ubi images is covered in the section above. This section is all about extracting and / or manipulating the contents of a ubi volume.

References:

On a (NAND-less) Linux Development Host using nandsim

Often you want to work with UBI and UBIFS images on a Linux Development Host which has no NAND itself. While it is not currently possible to mount a UBI or UBIFS as a loopback device (commonly used for other types of block devices) you can use the nandsim kernel module to simulate a NAND device on a NAND-less Linux development host.

To create a simulated NAND device you need to know the first four bytes returned by the READ_ID flash command, which identifies the characteristics of the device. These values can be found in the datasheet of the FLASH devices. Additionally you need to define the partitions used via the 'parts' kernel parameter. As Gateworks boards support two flash sizes, we will show both examples.

To create a virtual NAND device on a NAND-less Linux development system:

  1. load nandsim module with parameters for specific flash device simulating:
    • 256MiB MT29F2G08 FLASH 
      sudo rmmod nandsim
sudo modprobe nandsim
  first_id_byte=0x2c \
  second_id_byte=0xda \
  third_id_byte=0x90 \
  fourth_id_byte=0x95 \
  parts=0x80,0x8,0x778
    • 2GiB Cyrpess S34ML16G202 FLASH 
      sudo rmmod nandsim
sudo modprobe nandsim \
  first_id_byte=0x01 \
  second_id_byte=0xd5 \
  third_id_byte=0xd2 \
  fourth_id_byte=0x95 \
  parts=0x80,0x8,0x778
    • 2GiB Micron MT29F16G08 FLASH: 
      sudo rmmod nandsim
sudo modprobe nandsim \
  first_id_byte=0x2c \
  second_id_byte=0xd5 \
  third_id_byte=0xd1 \
  fourth_id_byte=0xa6 \
  parts=0x80,0x8,0x778
    • the parts argument provides a comma separated list of partition sizes (in units of erase blocks) used to partition the NAND device and the choices above matches the partitions Gateworks uses for the Ventana boards which is defined via the U-Boot env var 'mtdparts' which by default is set to 'mtdparts=nand:16m(uboot),1m(env),-(rootfs)'
    • the above creates the following MTD devices (which you can see via /proc/mtd): 
      $ cat /proc/mtd
dev:    size   erasesize  name
mtd0: 01000000 00020000 "NAND simulator partition 0"
mtd1: 00100000 00020000 "NAND simulator partition 1"
mtd2: 0ef00000 00020000 "NAND simulator partition 2"
    • by default nandsim will use RAM backed storage which makes for much faster read/write access, however if you are cramped for RAM space and simulating a large device (ie 2GB) you can add a cache_file=/tmp/nandimage param to nandsim to tell it to use a file-backed storage instead. You can remove the file once you rmmod nandsim.
  2. populate NAND with an existing ubi: 
    sudo modprobe mtdblock # create /dev/mtdblock devices for each partition
sudo dd if=image.ubi of=/dev/mtdblock2 bs=2048
    • Note we are working with the 3rd MTD partition /dev/mtd2 and /dev/mtdblock2 as this is the partition where we store the rootfs
    • Note that instead of the dd command above you could also have used ubiformat /dev/mtd2 to wipe and format a new ubi but this section is documenting how to extract and alter contents of an existing ubi.
  3. attach the UBI 
    sudo modprobe ubi
sudo ubiattach /dev/ubi_ctrl -m2 -O2048
sudo ubinfo -a # optionally show info about the UBI
    • Note the 'm' parameter specifies the mtd device number and we are working with /dev/mtd2 which is the rootfs partition
  4. mount the ubifs to host 
    sudo modprobe ubifs
sudo mkdir /mnt/ubi # create a mountpoint
sudo mount -t ubifs ubi0 /mnt/ubi
ls /mnt/ubi # optionally list contents of mounted filesystem
    • Note that mount operates on the UBI device of which we have 1 of (thus ubi0) where we have used ubiattach to attach it to the 3rd partition (-m2) above

Once you are done working with the files in /mnt/ubi simply unmount the device: 

sudo umount /mnt/ubi

If you have altered the filesystem and wish to extract the altered UBI you can detach and use dd to copy it to a file: 

sudo ubidetach /dev/ubi_ctrl -m2
sudo dd if=/dev/mtdblock2 of=image-updated.ubi bs=2048
  • Note the 'm' parameter specifies the mtd device number and we are working with /dev/mtd2 which is the rootfs partition

And when you are done working with the simulated NAND device completely, remove the nandsim module to free up your system memory: 

sudo rmmod nandsim

References:

On a target board with NAND FLASH

To extract or manipulate the contents of a UBI within the NAND Flash of a Linux system, Linux must be booted from a secondary medium without the NAND FLASH partition being mounted.

The following examples assume that UBI has been configured to use the 3rd (zero based) MTD FLASH partition (ubi.mtd=2 kernel command line) as we typically use the mtdparts=nand:16m(uboot),1m(env),-(rootfs) env variable in U-Boot to dynamically modify the Linux Device-Tree to partition the NAND flash device into 3 partitions: bootloader, bootloader-env, and rootfs. Thus the bootloader is /dev/mtd0, the bootloader env is /dev/mtd1, and the rootfs is /dev/mtd2.

Examples:

  • Mounting UBI NAND FLASH volume (from /dev/mtd2): 
    ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
ubiattach /dev/ubi_ctrl -m 2 # attach /dev/mtd2 as /dev/ubi0
mount -t ubifs ubi0:rootfs /mnt/ubi #mount 'rootfs' volume from /dev/ubi0 to /mnt/ubi
  • Updating NAND FLASH using linux target with NAND(/dev/mtd2) with a UBI image: 
    ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
sync
ubiformat /dev/mtd2 -y -f image.ubi
  • Updating NAND FLASH partition (/dev/mtd2) with a UBIFS image 
    ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
ubiformat /dev/mtd2 -y # format (will wipe contents!)
ubiattach /dev/ubi_ctrl -m 2 # attach /dev/mtd2 as /dev/ubi0
ubimkvol /dev/ubi0 -N rootfs -m # create a volume on /dev/ubi0 named 'rootfs'
ubiupdatevol /dev/ubi0_0 image.ubifs
  • Updating NAND FLASH partition (/dev/mtd2) with a tarball 
    ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
ubiformat /dev/mtd2 -y # format (will wipe contents!)
ubiattach /dev/ubi_ctrl -m 2 # attach /dev/mtd2 as /dev/ubi0
ubimkvol /dev/ubi0 -N rootfs -m # create a volume on /dev/ubi0 named 'rootfs'
mkdir /mnt/ubi # create a mount-point
mount -t ubifs ubi0:rootfs /mnt/ubi # mount 'rootfs' volume from /dev/ubi0 to /mnt/ubi
tar xvf rootfs.tar.gz -C /mnt/ubi # extract tarball
umount /mnt/ubi # unmount filesystem

Troubleshooting:

The output beneath indicates that /dev/mtd2 is not attached and you may proceed with manipulating the NAND flash. 

root@OpenWrt:/# ubidetach /dev/ubi_ctrl -m 2
ubidetach: error!: cannot detach mtd2
           error 16 (Resource busy)

Working with UBI in U-boot Bootloader

The U-boot bootloader has support for ubi and ubifs. Before working with a ubi image you need to 'attach' an MTD partition, then you can mount and work with ubifs filesystem images. Note that you can remove/create ubi volumes and you can ls and load from ubifs filesystem images but you cant write to ubifs filesystems:

  • determining MTD partition containing ubi's: 
    Ventana > mtd

device nand0 <nand>, # parts = 3
 #: name                size            offset          mask_flags
 0: uboot               0x01000000      0x00000000      0
 1: env                 0x00100000      0x01000000      0
 2: rootfs              0x7ef00000      0x01100000      0

active partition: nand0,0 - (uboot) 0x01000000 @ 0x00000000

defaults:
mtdids  : nand0=nand
mtdparts: mtdparts=nand:16m(uboot),1m(env),-(rootfs)
    • The above shows there is one RAW NAND device (nand0) and it is partitioned into 3 sections: uboot (from 0x00000000 to 0x01000000), uboot env (from 0x01000000 to 0x01100000) and rootfs (from 0x01100000 to 0x7ef00000). The 'uboot' and 'env' are part of the boot firmware, so the section named 'rootfs' is the section that can contain ubi's
    • Note that this partitioning is solely based on the 'mtdids' and 'mtdparts' U-Boot env variables and can be changed if desired. Note also that these variables are used to send information into the kernel so that the kernel creates these sections as specific MTD devices
  • attaching a ubi image to an MTD partition: 
    Ventana > ubi part rootfs
ubi0: default fastmap pool size: 256
ubi0: default fastmap WL pool size: 128
ubi0: attaching mtd1
ubi0: scanning is finished
ubi0: attached mtd1 (name "mtd=2", size 2031 MiB)
ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
ubi0: good PEBs: 16248, bad PEBs: 0, corrupted PEBs: 0
ubi0: user volume: 3, internal volumes: 1, max. volumes count: 128
ubi0: max/mean erase counter: 2/0, WL threshold: 4096, image sequence number: 1592951068
ubi0: available PEBs: 0, total reserved PEBs: 16248, PEBs reserved for bad PEB handling: 320
    • The above 'attaches' a ubi image to an MTD partition (equivalent to ubiattach in Linux) so that you can mount ubi volumes. Note that this operation can take several seconds as it must scan the entire ubi image to construct a bad block table (unless fastmap is used). Once attached you can use ubifsmount and you will need to detach via ubi detach before attaching another or when done using it.
  • showing ubi volumes that exist in a ubi image: 
    Ventana > ubi info layout
Volume information dump:
        vol_id          0
        reserved_pebs   124
        alignment       1
        data_pad        0
        vol_type        3
        name_len        4
        usable_leb_size 126976
        used_ebs        124
        used_bytes      15745024
        last_eb_bytes   126976
        corrupted       0
        upd_marker      0
        name            boot
Volume information dump:
        vol_id          1
        reserved_pebs   210
        alignment       1
        data_pad        0
        vol_type        3
        name_len        6
        usable_leb_size 126976
        used_ebs        210
        used_bytes      26664960
        last_eb_bytes   126976
        corrupted       0
        upd_marker      0
        name            rootfs
Volume information dump:
        vol_id          2
        reserved_pebs   15586
        alignment       1
        data_pad        0
        vol_type        3
        name_len        11
        usable_leb_size 126976
        used_ebs        15586
        used_bytes      1979047936
        last_eb_bytes   126976
        corrupted       0
        upd_marker      0
        name            rootfs_data
Volume information dump:
        vol_id          2147479551
        reserved_pebs   2
        alignment       1
        data_pad        0
        vol_type        3
        name_len        13
        usable_leb_size 126976
        used_ebs        2
        used_bytes      253952
        last_eb_bytes   2
        corrupted       0
        upd_marker      0
        name            layout volume
    • The above shows there are 3 ubi volumes in the attached ubi image named 'boot', 'rootfs', 'rootfs_data'. Note the last dump is the volume layout (think of it as a directory table)
    • Note the contents of the various ubi volumes is unknown by the ubi layer
  • once a ubi volume is mounted 'if' it is a ubifs you can use ubifsmount, ubifsload, ubifsls, ubifsumount to operate on its contents: 
    Ventana > ubifsmount ubi0:boot
Ventana > ubifsls
<LNK>          31  Tue Jun 23 23:51:55 2020  imx6q-gw53xx.dtb
            38158  Tue Jun 23 23:47:33 2020  gateworks-imx6-imx6q-gw5400-a.dtb
            41991  Tue Jun 23 23:47:30 2020  gateworks-imx6-imx6dl-gw53xx.dtb
            43296  Tue Jun 23 23:47:33 2020  gateworks-imx6-imx6q-gw53xx.dtb
<LNK>          32  Tue Jun 23 23:51:55 2020  imx6dl-gw53xx.dtb
...
          3275024  Tue Jun 23 23:51:55 2020  gateworks-imx6-uImage
<LNK>          21  Tue Jun 23 23:51:55 2020  uImage
Ventana > ubifsload $loadaddr uImage
Loading file 'uImage' to addr 0x12000000...
Done
Ventana > ubifsumount
Unmounting UBIFS volume boot!
    • Again you can only use ubifs commands on ubi volumes that contain ubifs filesystems
    • Be sure to unmount your volume when done working with it
  • removing ubi volumes 
    Ventana > ubi remove rootfs_data
Remove UBI volume rootfs_data (id 2)
Ventana > ubi remove rootfs
Remove UBI volume rootfs (id 1)
  • creating ubi volumes 
    Ventana > ubi create rootfs a00000 static
Creating static volume rootfs of size 10485760
  • writing ubi volumes 
    Ventana > tftp $loadaddr ventana/test_large.ubifs
Using FEC device
TFTP from server 192.168.1.146; our IP address is 192.168.1.1
Filename 'ventana/test_large.ubifs'.
Load address: 0x12000000
Loading: #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         ############
         8.1 MiB/s
done
Bytes transferred = 10665984 (a2c000 hex)
Ventana > ubi remove rootfs
Remove UBI volume rootfs (id 1)
Ventana > ubi create rootfs $filesize static                      
Creating static volume rootfs of size 10665984
Ventana > ubi write $loadaddr rootfs $filesize
10665984 bytes written to volume rootfs

Updating NAND FLASH with a UBI image

Video:

If your ubi image is on a tftp server as image.ubi you can use the following to update the 'rootfs' NAND FLASH partition in U-Boot:

Preferred method using the nand_update script will perform the update for you using the image name specified in the image_rootfs env variable. Run the following commands in the bootloader: 

setenv ipaddr 192.168.1.1
setenv serverip 192.168.1.254
setenv image_rootfs image.ubi; 
run nand_update

It may also be necessary to set a gateway or netmask IP address. This can be done using the following commands: 

setenv gatewayip <ipaddress>
setenv netmask <ipaddress>

The more manual method without using the nand_update script, if your ubi image is on a tftp server as image.ubi you can use the following to update the 'rootfs' NAND FLASH partition in U-Boot (update the ip addresses as appropriate for your network environment): 

setenv ipaddr 192.168.1.1
setenv serverip 192.168.1.254
tftpboot ${loadaddr} image.ubi && \
  nand erase.part rootfs && \
  nand write ${loadaddr} rootfs ${filesize}

If your ubi image is on the first partition of a FAT formated MMC called image.ubi you can do the following: 

mmc rescan
fatload mmc 0:1 ${loadaddr} image.ubi && \
  nand erase.part rootfs && \
  nand write ${loadaddr} rootfs ${filesize}

If your ubi image is on the first partition of a ext2/3/4 formated MMC called image.ubi you can do the following: 

mmc rescan
ext2load mmc 0:1 ${loadaddr} image.ubi && \
  nand erase.part rootfs && \
  nand write ${loadaddr} rootfs ${filesize}

If your ubi image is on the first partition of a FAT formatted USB stick called image.ubi you can do the following: 

usb start
usb dev 0
fatload usb 0:1 ${loadaddr} image.ubi && \
  nand erase.part rootfs && \
  nand write ${loadaddr} rootfs ${filesize}

Note that operating on the ubi device as described here destroys the wear-leveling information contained in the UBI therefore it is recommended to work at the ubi volume layer to replace ubifs volumes as described below

Updating NAND FLASH with a ubifs image

When you want to preserve RAW nand wear-leveling you must operate at the ubifs layer (filesystem layer vs ubi layer) so that wear leveling stored in the ubi is maintained.

To update the contents of the rootfs volume with a ubifs in U-Boot: 

ubi part rootfs # select the 'rootfs' partition (from mtdparts env var) for ubi
ubi info layout # list volumes
ubi remove rootfs # remove rootfs volume
ubi create rootfs a00000 static # create a 10MB volume named rootfs
tftpboot ${loadaddr} ventana/root.ubifs && ubi write ${loadaddr} rootfs ${filesize}
ubifsmount ubi0:rootfs # mount rootfs volume from ubi
ubifsls /
ubifsumount

Operating on a NAND device at the ubi volume layer like this preserves the wear-leveling information for your flash device and is therefore recommended, especially if you are doing this often.

Use the information in the section above for examples on how to load data in uboot from various sources such as network, mmc, usb.

Updating UBI with low RAM devices

For some devices, the size of the NAND flash exceeds the size of RAM. In this scenario the instructions from the prior section would fail as you would run out of RAM space before you could finish retrieving the file to program the NAND flash with.

In order to flash a UBI in this situation within U-Boot you must either split the UBI image into chunks and transfer/flash each chunk, or you can use U-Boot to create a UBI volume and split the UBIFS and transfer/flash each chunk of the UBIFS. Flashing a UBI has the disadvantage of erasing all the NAND wear leveling information the UBI has been tracking and won't support the special handling of bad blocks. Therefore it is recommended that you use the split UBIFS method which preserves wear leveling info and works with bad blocks.

Split UBIFS flashing

The following example shows how you can take a UBIFS and split it into chunks that fit into low memory devices and flash them via U-Boot. The example uses tftp to transfer but you can easily replace that instruction with one that loads the chunks from removable storage:

  1. First you must split your ubifs image on your Linux development host and note the total size of the pre-split image which you will need during flashing. This example assumes a ~580MB ubifs that we will split into 200MB chunks to fit within a 256MB DRAM system: 
    $ ls -l focal-ventana_normal.ubifs
-rw-rw-r-- 1 gateworks user 581042176 Sep  9 14:56 focal-ventana_normal.ubifs
$ split -d -b 200M focal-ventana_normal.ubifs focal-ventana_normal.ubifs.part
$ ls -l focal-ventana_normal.ubifs*
-rw-rw-r-- 1 gateworks user 581042176 Sep  9 14:56 focal-ventana_normal.ubifs
-rw-rw-r-- 1 gateworks user 209715200 Sep  9 15:19
focal-ventana_normal.ubifs.part00
-rw-rw-r-- 1 gateworks user 209715200 Sep  9 15:19
focal-ventana_normal.ubifs.part01
-rw-rw-r-- 1 gateworks user 161611776 Sep  9 15:20
focal-ventana_normal.ubifs.part02
$ printf "0x%x\n" 581042176
0x22a20000
  2. Now boot your target board and break into U-Boot. Use the mtd command to determine the MTD partition name that you wish to use for UBI. This is dictated by the mtdids and mtdparts env variables. The gateworks bootloader defaults the name of the MTD partition used for UBI to 'rootfs' as shown below: 
    Ventana > mtd

device nand0 <nand>, # parts = 3
 #: name                size            offset          mask_flags
 0: uboot               0x01000000      0x00000000      0
 1: env                 0x00100000      0x01000000      0
 2: rootfs              0x7ef00000      0x01100000      0

active partition: nand0,0 - (uboot) 0x01000000 @ 0x00000000
...
  3. Use the ubi command to attach the rootfs MTD partition to the ubi layer. The rootfs name here must agree with the partition name from the MTD partition as described above... this is 'not' necessarily the name of your UBI 'volume' used below: 
    Ventana > ubi part rootfs
  4. Use the ubi info layout command to inspect your UBI volumes. You will always have one volume named layout volume which holds the UBI volume information. 
    Ventana > ubi info layout
Volume information dump:
        vol_id          0
        reserved_pebs   15920
        alignment       1
        data_pad        0
        vol_type        3
        name_len        6
        usable_leb_size 126976
        used_ebs        15920
        used_bytes      2021457920
        last_eb_bytes   126976
        corrupted       0
        upd_marker      0
        name            rootfs
Volume information dump:
        vol_id          2147479551
        reserved_pebs   2
        alignment       1
        data_pad        0
        vol_type        3
        name_len        13
        usable_leb_size 126976
        used_ebs        2
        used_bytes      253952
        last_eb_bytes   2
        corrupted       0
        upd_marker      0
        name            layout volume
    • The above shows that there is one data volume named 'rootfs'
  5. remove the existing volume (if you are operating on a blank UBI and only have layout volume skip this step): 
    Ventana > ubi remove rootfs
Remove UBI volume rootfs (id 0)
  6. Create the new volume. Note that the volume name you use here is the name you would pass to the kernel for mounting. We typically call this 'rootfs' (which should not be confused with the MTD partition name). 
    Ventana > ubi create rootfs
No size specified -> Using max size (2021457920)
Creating dynamic volume rootfs of size 2021457920
  7. Load your first chunk: 
    Ventana > tftpboot $loadaddr ventana/focal-ventana_normal.ubifs.part00
...
done
Bytes transferred = 209715200 (c800000 hex)
    • You could alternatively use the load command to load from removable storage such as microSD or USB Mass storage but you will need to set the $filesize variable appropriately.
  8. Write your first chunk - this first chunk you need to provide the total size of the pre-split UBIFS image as the last argument in hex: 
    Ventana > ubi write.part $loadaddr rootfs $filesize 0x22a20000
209715200 bytes written to volume rootfs
    • Note the last argument 0x22a20000 which is the total size of the pre-split ubifs image in hex.
  9. Continue the transfer and flash of your additional chunks, this time leaving off the last argument: 
    Ventana > tftpboot $loadaddr ventana/focal-ventana_normal.ubifs.part01
...
done
Bytes transferred = 209715200 (c800000 hex)
Ventana > ubi write.part $loadaddr rootfs $filesize
209715200 bytes written to volume rootfs
Ventana > tftpboot $loadaddr ventana/focal-ventana_normal.ubifs.part02
...
done
Bytes transferred = 161611776 (9a20000 hex)
Ventana > ubi write.part $loadaddr rootfs $filesize
161611776 bytes written to volume rootfs

Split UBI flashing

Gateworks has created an installable U-Boot script that will apply UBI files containing a single partition image. This can be particularly useful for updating the firmware of a running target.

This method does not work if you have bad blocks and also erases previous wear-leveling data that may be tracked by the previous UBI image. The split UBIFS method above is recommended instead.

To update using a split UBI:

1. Split your UBI

In some situations the RAM of your device will be smaller than the image file required to update your system, therefore it is necessary to split the image file and apply the resulting compressed parts in a piece wise update. The below example uses the split command to split the input file into parts that are reasonably smaller than the RAM size of the board which will be consuming the update. 

FILE=xenial-large.ubi
# Split file every 200M with suffix ".part" and decimal increment
split -d -b 200M ${FILE} ${FILE}.part

# Result will be multiple files such as xenial-large.ubi.partXX that are <= 200M and
# which adhere to the naming convention required by the script (ends with .partXX in sequential order)

2. Install the U-Boot script

On the target machine, source the ​nand_split_update.scr U-Boot script attached to this page (or copy its contents with an editor). For example via tftp: 

setenv serverip 192.168.1.100
tftpboot nand_split_update.scr
source $loadaddr

3. Configure your U-Boot environment

From the script's usage: 

The following environment variables need to be set in order for this script to run:

    splitfile    - full path file name prefix of the split files (eg file.ubi for file.ubi.part00)

An example configuration: 

setenv splitfile xenial-large.ubi          # ubi file from example above

4. Run the added script with run nand_split_update

UBI Fastmap

UBI Fastmap is on optional feature which stores the physical to logical eraseblock relations in a checkpoint (called fastmap) to reduce the initialization time of UBI. Without UBI Fastmap, the init time of UBI is proportional to the number of physical erase blocks on the FLASH device and with Fastmap enabled the scan time is limited to a fixed number of blocks.

While enabling UBI Fastmap could save several seconds in Linux init as well as several seconds in U-Boot ubi attach (depending on number of logical flash blocks) the downside is that you wear out the first 64blocks of your NAND faster. Therefore the Gateworks kernels and bootscripts do not enable this by default.

If you wish to enable this feature because boot time is more critical than NAND lifetime you can do so by:

  • enable MTD_UBI_FASTMAP in U-Boot (to use the UBI Fastmap if it was created by the kernel)
  • enable MTD_UBI_FASTMAP in Linux kernel
  • pass 'ubi.fm_autoconvert=1' via kernel params to instruct the kernel to create a UBI Fastmap if it does not yet exist and use it if it does exist
  • Note that enabling MTD_UBI_FASTMAP in the kernel/uboot won't enable it unless 'ubi.fm_autoconvert=1' was used on the kenrel
Last modified 7 months ago Last modified on 06/05/2024 05:06:25 PM

Attachments (1)

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.