wiki:linux/ubi

Unsorted Block Images (UBI)

Unsorted block images (UBI) is an erase block management layer for flash memory devices such as raw NAND devices. UBI serves two purposes:

  • tracking NAND flash bad blocks
  • providing wear leveling

UBI was written specifically for UBIFS so that the filesystem does not have to deal with these issues. UBIFS is a successor to JFFS2 and a competitor to LogFS.

References:

FLASH chip geometry

Because UBI takes care of bad block tracking and wear leveling, it needs to know some details about the geometry of the raw FLASH device when creating an image.

The geometry data can be found in the datasheet for the specific flash device used. Gateworks uses two different sized devices which do not share geometries.

UBI use on Gateworks Ventana

UBI is used on the Gateworks Ventana for raw NAND storage. Prior Gateworks product families typically JFFS2.

The Gateworks Ventana default U-Boot environment scripts make a few assumptions about booting from NAND flash:

  • the mtdparts env var partitions the NAND device into sections for uboot (16MB as this accounts for the SPL, u-boot.img and redundancy for bad blocks), u-boot env (1MB), rootfs (the remainder of the device)
  • the 'rootfs' partition (defined by the mtdparts env var) contains a UBI device/image
  • the UBI device/image in the rootfs partition contains a volume also called 'rootfs'
  • an optional bootscript may exist in boot/6x_bootscript-ventana (the image env var) and if found will be sourced
  • a flattened device tree blob (dtb) may exist in boot/ (named by the automatic env vars fdt_file, fdt_file1, or fdt_file2)
  • a kernel may exist in boot/uImage

The above assumptions are not requirements - the U-Boot env scripts can be changed to suit your needs.

Creating a UBI using existing rootfs, kernel, and device-tree blobs

To create a UBI image suitable for booting on a standard Gateworks Ventana bootloader (taking into account the assumptions above) you can do the following on a Linux system provided you have a root filesystem (myrootfs.tar.gz), kernel (uImage), and device-tree dtb files (imx6*-gw*.dtb).

256MB FLASH devices

To create a UBI image suitable for the 256MB FLASH devices:

  1. create a temporary directory for our rootfs and untar your filesystem to it
    mkdir tmp
    fakeroot tar --numeric-owner -xvf myrootfs.tar.gz -C tmp
    
    • Note that fakeroot is used because this allows you preserve the ownership of files that you have in your rootfs such as root owned /dev files
  2. copy your kernel files (and u-boot bootscript if you have one):
    mkdir tmp/boot
    cp uImage imx6*-gw*.dtb 6x_bootscript-ventana tmp/boot/
    
  3. create a ubifs image for the 256MB device geometry:
    mkfs.ubifs -F -m 2048 -e 124KiB -c 1912 -x zlib -o root.ubifs -d tmp
    
  4. use ubinize to create ubi images for the 256MB device geometry which can be written to a suitable raw NAND devices in U-Boot using nand write:
    # for the small layout device
    cat <<EOF > ubinize.cfg
    [ubifs]
    mode=ubi
    image=root.ubifs
    vol_id=0
    vol_type=dynamic
    vol_name=rootfs
    vol_flags=autoresize
    EOF
    ubinize -m 2048 -p 128KiB -s 2048 -o image.ubi ubinize.cfg
    

2GB FLASH devices

To create a UBI image suitable for the 2GB FLASH devices (Note that the parameters to mkfs.ubifs and ubinize change):

  1. create a temporary directory for our rootfs and untar your filesystem to it
    mkdir tmp
    fakeroot tar --numeric-owner -xvf myrootfs.tar.gz -C tmp
    
    • Note that fakeroot is used because this allows you preserve the ownership of files that you have in your rootfs such as root owned /dev files
  2. copy your kernel files (and u-boot bootscript if you have one):
    mkdir tmp/boot
    cp uImage imx6*-gw*.dtb 6x_bootscript-ventana tmp/boot/
    
  3. create ubifs images for the 2GB device geometry:
    mkfs.ubifs -F -m 4096 -e 248KiB -c 8124 -x zlib -o root.ubifs -d tmp
    
  4. use ubinize to create ubi image for the 2GB device geometry which can be written to a suitable raw NAND devices in U-Boot using nand write:
    # for the small layout device
    cat <<EOF > ubinize.cfg
    [ubifs]
    mode=ubi
    image=root.ubifs
    vol_id=0
    vol_type=dynamic
    vol_name=rootfs
    vol_flags=autoresize
    EOF
    ubinize -m 4096 -p 256KiB -s 4096 -o image.ubi ubinize.cfg
    

Working with a UBI in Linux OS (extracting or manipulating contents)

Linux has support for ubi:

  • mounting a ubi
  • creating/renaming/removing ubi volumes
  • reading/writing ubi volumes

Working with UBI and UBIFS volumes on Linux require use of the following:

  • tools (from mtd-utils package):
    • ubiattach
    • ubidetach
    • ubiformat
    • ubirename
    • ubiupdatevol
    • ubiinfo
    • ubimkvol
    • ubirmvol
  • kernel support:
    • mtd (the MTD layer)
    • mtdblock (the block layer over MTD)
    • ubi (ubi support)
    • ubifs (ubi filesystem support)
    • nandsim (if you want to simulate a NAND device on a development host)

When using the above tools the MTD partition must not be mounted by your Linux OS meaning you either have to:

Note that the creation of ubifs and ubi images is covered in the section above. This section is all about extracting and / or manipulating the contents of a ubi volume.

References:

On a (NAND-less) Linux Development Host

Often you want to work with UBI and UBIFS images on a Linux Development Host which has no NAND itself. While it is not currently possible to mount a UBI or UBIFS as a loopback device (commonly used for other types of block devices) you can use the nandsim kernel module to simulate a NAND device on a NAND-less Linux development host.

To create a simulated NAND device you need to know the first four bytes returned by the READ_ID flash command, which identifies the characteristics of the device. These values can be found in the datasheet of the FLASH devices. As Gateworks boards support two flash sizes, we will show both examples.

To create a virtual NAND device on a NAND-less Linux development system:

  1. load nandsim module with parameters for specific flash device simulating:
    • 256MB FLASH (what we refer to as the 'normal' flash device):
      sudo rmmod nandsim
      sudo modprobe nandsim
        first_id_byte=0x2c \
        second_id_byte=0xda \
        third_id_byte=0x90 \
        fourth_id_byte=0x95
      
    • 2GB FLASH (what we refer to as the 'large' flash device):
      sudo rmmod nandsim
      sudo modprobe nandsim \
        first_id_byte=0x2c \
        second_id_byte=0xd5 \
        third_id_byte=0xd1 \
        fourth_id_byte=0xa6
      
    • the above create a /dev/mtd0 device on your development host
    • by default nandsim will use RAM backed storage which makes for much faster read/write access, however if you are cramped for RAM space and simulating a large device (ie 2GB) you can add a cache_file=/tmp/nandimage param to nandsim to tell it to use a file-backed storage instead. You can remove the file once you rmmod nandsim.
  2. populate NAND with an existing ubi:
    sudo modprobe mtdblock
    sudo dd if=image.ubi of=/dev/mtdblock0 bs=2048
    
    • the mtdblock module creates a /dev/mtdblock0 device on your development host
    • Note that instead of the dd command above you could also have used ubiformat /dev/mtd0 to wipe and format a new ubi but this section is documenting how to extract and alter contents of an existing ubi.
  3. attach the UBI
    sudo modprobe ubi
    sudo ubiattach /dev/ubi_ctrl -m0 -O2048
    sudo ubiinfo -a # optionally show info about the UBI
    
  4. mount the ubifs to host
    sudo modprobe ubifs
    sudo mkdir /mnt/ubi # create a mountpoint
    sudo mount -t ubifs ubi0 /mnt/ubi
    ls /mnt/ubi # optionally list contents of mounted filesystem
    

Once you are done working with the files in /mnt/ubi simply unmount the device:

sudo umount /mnt/ubi

If you have altered the filesystem and wish to extract the altered UBI you can detach and use dd to copy it to a file:

sudo ubidetach /dev/ubi_ctrl -m0
sudo dd if=/dev/mtdblock0 of=image-updated.ubi bs=2048

And when you are done working with the simulated NAND device completely, remove the nandsim module to free up your system memory:

sudo rmmod nandsim

References:

On a target board with NAND FLASH

To extract or manipulate the contents of a UBI within the NAND Flash of a Linux system, Linux must be booted from a secondary medium on a device with NAND FLASH and without the NAND FLASH partition being mounted.

The following examples assume that UBI has been configured to use the 3rd (zero based) MTD FLASH partition (ubi.mtd=2 kernel command line) as we typically use the mtdparts=nand:16m(uboot),1m(env),-(rootfs) env variable in U-Boot to dynamically modify the Linux Device-Tree to partition the NAND flash device into 3 partitions: bootloader, bootloader-env, and rootfs. Thus the bootloader is /dev/mtd0, the bootloader env is /dev/mtd1, and the rootfs is /dev/mtd2.

Examples:

  • Mounting UBI NAND FLASH volume (from /dev/mtd2):
    ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
    ubiattach /dev/ubi_ctrl -m 2 # attach /dev/mtd2 as /dev/ubi0
    mount -t ubifs ubi0:rootfs /mnt/ubi #mount 'rootfs' volume from /dev/ubi0 to /mnt/ubi
    
  • Updating NAND FLASH using linux target with NAND(/dev/mtd2) with a UBI image:
    ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
    sync
    ubiformat /dev/mtd2 -y -f image.ubi
    
  • Updating NAND FLASH partition (/dev/mtd2) with a UBIFS image
    ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
    ubiformat /dev/mtd2 -y # format (will wipe contents!)
    ubiattach /dev/ubi_ctrl -m 2 # attach /dev/mtd2 as /dev/ubi0
    ubimkvol /dev/ubi0 -N rootfs -m # create a volume on /dev/ubi0 named 'rootfs'
    ubiupdatevol /dev/ubi0_0 image.ubifs
    
  • Updating NAND FLASH partition (/dev/mtd2) with a tarball
    ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
    ubiformat /dev/mtd2 -y # format (will wipe contents!)
    ubiattach /dev/ubi_ctrl -m 2 # attach /dev/mtd2 as /dev/ubi0
    ubimkvol /dev/ubi0 -N rootfs -m # create a volume on /dev/ubi0 named 'rootfs'
    mkdir /mnt/ubi # create a mount-point
    mount -t ubifs ubi0:rootfs /mnt/ubi # mount 'rootfs' volume from /dev/ubi0 to /mnt/ubi
    tar xvf rootfs.tar.gz -C /mnt/ubi # extract tarball
    umount /mnt/ubi # unmount filesystem
    

Working with UBI in U-boot Bootloader

The U-boot bootloader used by Ventana has support for ubi:

  • mounting a ubi
  • removing ubi volumes
  • creating ubi volumes
  • writing ubi volumes

Updating NAND FLASH with a UBI image

Updating NAND FLASH from image on tftp server

Gateworks has created a script to simplify the process of installing a ubi file onto the board. This requires setting up a tftp server on your network.

The following commands are to be ran on the bootloader command prompt on the Gateworks SBC.

setenv ipaddr 192.168.1.1
setenv serverip 192.168.1.254 #replace with IP of actual TFTP Server
setenv image_rootfs image.ubi #where image.ubi is the name of the ubi file on the TFTP Server
run nand_update
  • Note that operating on the ubi device as described here destroys the wear-leveling information contained in the UBI therefore it is recommended to work at the ubi volume layer to replace ubifs volumes as described below
  • Note that if your ubi image is too large to fit within available memory (in the rare case you have a board with larger NAND flash than memory see below)
  • Note eth0 should be used which is natively supported. On special boards like the GW5520, both ports are ran off PCI, thus PCI needs to be enabled in the bootloader as shown here

Updating NAND FLASH from image on removable storage

An Alternative method if your ubi image is on a tftp server as image.ubi you can use the following to update the 'rootfs' NAND FLASH partition in U-Boot (update the ip addresses as appropriate for your network environment):

setenv ipaddr 192.168.1.1
setenv serverip 192.168.1.254
tftp ${loadaddr} image.ubi && \
  nand erase.part rootfs && \
  nand write ${loadaddr} rootfs ${filesize}
  • Note the nand_update script will perform the last step for you using the image name specified in the image_rootfs env variable such that you could do a setenv image_rootfs image.ubi; run nand_update

If your ubi image is on the first partition of a FAT formated MMC called image.ubi you can do the following:

mmc rescan
fatload mmc 0:1 ${loadaddr} image.ubi && \
  nand erase.part rootfs && \
  nand write ${loadaddr} rootfs ${filesize}

If your ubi image is on the first partition of a ext2/3/4 formated MMC called image.ubi you can do the following:

mmc rescan
ext2load mmc 0:1 ${loadaddr} image.ubi && \
  nand erase.part rootfs && \
  nand write ${loadaddr} rootfs ${filesize}

If your ubi image is on the first partition of a FAT formatted USB stick called image.ubi you can do the following:

usb start
usb dev 0
fatload usb 0:1 ${loadaddr} image.ubi && \
  nand erase.part rootfs && \
  nand write ${loadaddr} rootfs ${filesize}

Note that operating on the ubi device as described here destroys the wear-leveling information contained in the UBI therefore it is recommended to work at the ubi volume layer to replace ubifs volumes as described below

Note that if your ubi image is too large to fit within available memory (in the rare case you have a board with larger NAND flash than memory see below)

Updating NAND FLASH with a split UBI Image

In rare situations where your DRAM is smaller than the UBI file required to update your system, it is possible to split the UBI file and apply the resulting parts in a piece wise update. The steps to do so are as follows:

  1. Ubuntu has the split command that will allow you to split your UBI into parts that will fit in your DRAM.
    file=file.ubi
    # Split file into 400M chunks with suffix ".part" and decimal incrementing 
    split -d -b 400M ${file} ${file}.part
    
  2. On the target machine, source the nand_split_update.scr U-Boot script attached to this page (or copy its contents with an editor).
  3. Set the splitfile environment variable to the prefix name of your split UBI (file.ubi for the example above)
  4. Run the added script with run nand_split_update

Updating NAND FLASH with a ubifs image

To update the contents of a ubi with a ubifs in U-Boot:

ubi part rootfs # select the 'rootfs' partition (from mtdparts env var) for ubi
ubi info layout # list volumes
ubi remove rootfs # remove rootfs volume
ubi create rootfs a00000 static # create a 10MB volume named rootfs
tftp ${loadaddr} ventana/root.ubifs && ubi write ${loadaddr} rootfs ${filesize}
ubifsmount ubi0:rootfs # mount rootfs volume from ubi
ubifsls /
ubifsumount

Operating on a NAND device at the ubi volume layer like this preserves the wear-leveling information for your flash device and is therefore recommended, especially if you are doing this often.

Use the information in the section above for examples on how to load data in uboot from various sources such as network, mmc, usb.

Last modified 3 weeks ago Last modified on 07/27/17 14:21:36

Attachments (1)

Download all attachments as: .zip