Changes between Initial Version and Version 1 of linux/ubi


Ignore:
Timestamp:
10/22/2017 05:28:45 AM (23 months ago)
Author:
trac
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • linux/ubi

    v1 v1  
     1[[PageOutline]]
     2
     3= Unsorted Block Images (UBI) =
     4Unsorted block images (UBI) is an erase block management layer for flash memory devices such as raw NAND devices. UBI serves two purposes:
     5 * tracking NAND flash bad blocks
     6 * providing wear leveling
     7
     8UBI was written specifically for UBIFS so that the filesystem does not have to deal with these issues. UBIFS is a successor to JFFS2 and a competitor to LogFS.
     9
     10References:
     11 * http://en.wikipedia.org/wiki/UBIFS#Unsorted_Block_Images
     12 * http://www.linux-mtd.infradead.org/faq/ubi.html
     13
     14
     15= FLASH chip geometry =
     16[=#flashgeometry]
     17Because UBI takes care of bad block tracking and wear leveling, it needs to know some details about the geometry of the raw FLASH device when creating an image.
     18
     19The geometry data can be found in the datasheet for the specific flash device used. Gateworks uses two different sized devices which do not share geometries.
     20
     21
     22= UBI use on Gateworks Ventana =
     23UBI is used on the Gateworks Ventana for raw NAND storage. Prior Gateworks product families typically JFFS2.
     24
     25The Gateworks Ventana default U-Boot environment scripts make a few assumptions about booting from NAND flash:
     26 * the {{{mtdparts}}} env var partitions the NAND device into sections for uboot (16MB as this accounts for the {{{SPL}}}, {{{u-boot.img}}} and redundancy for bad blocks), u-boot env (1MB), rootfs (the remainder of the device)
     27 * the 'rootfs' partition (defined by the {{{mtdparts}}} env var) contains a UBI device/image
     28 * the UBI device/image in the rootfs partition contains a volume also called 'rootfs'
     29 * an optional bootscript may exist in {{{boot/6x_bootscript-ventana}}} (the {{{image}}} env var) and if found will be sourced
     30 * a flattened device tree blob (dtb) may exist in {{{boot/}}} (named by the automatic env vars {{{fdt_file}}}, {{{fdt_file1}}}, or {{{fdt_file2}}})
     31 * a kernel may exist in {{{boot/uImage}}}
     32
     33The above assumptions are not requirements - the U-Boot env scripts can be changed to suit your needs.
     34
     35
     36== Creating a UBI using existing rootfs, kernel, and device-tree blobs ==
     37[=#creatingubi]
     38To create a UBI image suitable for booting on a standard Gateworks Ventana bootloader (taking into account the assumptions above) you can do the following on a Linux system provided you have a root filesystem ({{{myrootfs.tar.gz}}}), kernel ({{{uImage}}}), and device-tree dtb files ({{{imx6*-gw*.dtb}}}).
     39
     40
     41=== 256MB FLASH devices ===
     42To create a UBI image suitable for the 256MB FLASH devices:
     431. create a temporary directory for our rootfs and untar your filesystem to it
     44{{{#!bash
     45mkdir tmp
     46fakeroot tar --numeric-owner -xvf myrootfs.tar.gz -C tmp
     47}}}
     48 * Note that {{{fakeroot}}} is used because this allows you preserve the ownership of files that you have in your rootfs such as root owned /dev files
     492. copy your kernel files (and u-boot bootscript if you have one):
     50{{{#!bash
     51mkdir tmp/boot
     52cp uImage imx6*-gw*.dtb 6x_bootscript-ventana tmp/boot/
     53}}}
     543. create a ubifs image for the 256MB device geometry:
     55{{{#!bash
     56mkfs.ubifs -F -m 2048 -e 124KiB -c 1912 -x zlib -o root.ubifs -d tmp
     57}}}
     584. use {{{ubinize}}} to create ubi images for the 256MB device geometry which can be written to a suitable raw NAND devices in U-Boot using {{{nand write}}}:
     59{{{#!bash
     60# for the small layout device
     61cat <<EOF > ubinize.cfg
     62[ubifs]
     63mode=ubi
     64image=root.ubifs
     65vol_id=0
     66vol_type=dynamic
     67vol_name=rootfs
     68vol_flags=autoresize
     69EOF
     70ubinize -m 2048 -p 128KiB -s 2048 -o image.ubi ubinize.cfg
     71}}}
     72
     73
     74=== 2GB FLASH devices ===
     75To create a UBI image suitable for the 2GB FLASH devices (Note that the parameters to {{{mkfs.ubifs}}} and {{{ubinize}}} change):
     761. create a temporary directory for our rootfs and untar your filesystem to it
     77{{{#!bash
     78mkdir tmp
     79fakeroot tar --numeric-owner -xvf myrootfs.tar.gz -C tmp
     80}}}
     81 * Note that {{{fakeroot}}} is used because this allows you preserve the ownership of files that you have in your rootfs such as root owned /dev files
     822. copy your kernel files (and u-boot bootscript if you have one):
     83{{{#!bash
     84mkdir tmp/boot
     85cp uImage imx6*-gw*.dtb 6x_bootscript-ventana tmp/boot/
     86}}}
     873. create ubifs images for the 2GB device geometry:
     88{{{#!bash
     89mkfs.ubifs -F -m 4096 -e 248KiB -c 8124 -x zlib -o root.ubifs -d tmp
     90}}}
     914. use {{{ubinize}}} to create ubi image for the 2GB device geometry which can be written to a suitable raw NAND devices in U-Boot using nand write:
     92{{{#!bash
     93# for the small layout device
     94cat <<EOF > ubinize.cfg
     95[ubifs]
     96mode=ubi
     97image=root.ubifs
     98vol_id=0
     99vol_type=dynamic
     100vol_name=rootfs
     101vol_flags=autoresize
     102EOF
     103ubinize -m 4096 -p 256KiB -s 4096 -o image.ubi ubinize.cfg
     104}}}
     105
     106
     107== Working with a UBI in Linux OS (extracting or manipulating contents) ==
     108[=#ubiinlinux]
     109Linux has support for ubi:
     110 * mounting a ubi
     111 * creating/renaming/removing ubi volumes
     112 * reading/writing ubi volumes
     113
     114Working with UBI and UBIFS volumes on Linux require use of the following:
     115 * tools (from {{{mtd-utils}}} package):
     116  * {{{ubiattach}}}
     117  * {{{ubidetach}}}
     118  * {{{ubiformat}}}
     119  * {{{ubirename}}}
     120  * {{{ubiupdatevol}}}
     121  * {{{ubiinfo}}}
     122  * {{{ubimkvol}}}
     123  * {{{ubirmvol}}}
     124 * kernel support:
     125  * mtd (the MTD layer)
     126  * mtdblock (the block layer over MTD)
     127  * ubi (ubi support)
     128  * ubifs (ubi filesystem support)
     129  * nandsim (if you want to simulate a NAND device on a development host)
     130
     131When using the above tools the MTD partition must not be mounted by your Linux OS meaning you either have to:
     132 * [#ubiondevhost use the nandsim kernel module to simulate a NAND device]
     133 * [#ubiontarget boot Linux target with NAND storage using another boot media (ramdisk, usb, mmc, sata, etc)]
     134
     135Note that the '''creation''' of ubifs and ubi images is covered in the [#creatingubi section above]. This section is all about extracting and / or manipulating the contents of a ubi volume.
     136
     137References:
     138 * http://free-electrons.com/blog/creating-flashing-ubi-ubifs-images/
     139 * http://elinux.org/UBIFS
     140 * http://www.linux-mtd.infradead.org/faq/ubifs.html
     141
     142
     143=== On a (NAND-less) Linux Development Host ===
     144[=#ubiondevhost]
     145Often you want to work with UBI and UBIFS images on a Linux Development Host which has no NAND itself. While it is not currently possible to mount a UBI or UBIFS as a loopback device (commonly used for other types of block devices) you can use the nandsim kernel module to simulate a NAND device on a NAND-less Linux development host.
     146
     147To create a simulated NAND device you need to know the first four bytes returned by the READ_ID flash command, which identifies the characteristics of the device. These values can be found in the datasheet of the FLASH devices. As Gateworks boards support two flash sizes, we will show both examples.
     148
     149To create a virtual NAND device on a NAND-less Linux development system:
     1501. load {{{nandsim}}} module with parameters for specific flash device simulating:
     151 * 256MB FLASH (what we refer to as the 'normal' flash device):
     152{{{#!bash
     153sudo rmmod nandsim
     154sudo modprobe nandsim
     155  first_id_byte=0x2c \
     156  second_id_byte=0xda \
     157  third_id_byte=0x90 \
     158  fourth_id_byte=0x95
     159}}}
     160 * 2GB FLASH (what we refer to as the 'large' flash device):
     161{{{#!bash
     162sudo rmmod nandsim
     163sudo modprobe nandsim \
     164  first_id_byte=0x2c \
     165  second_id_byte=0xd5 \
     166  third_id_byte=0xd1 \
     167  fourth_id_byte=0xa6
     168}}}
     169 * the above create a {{{/dev/mtd0}}} device on your development host
     170 * by default nandsim will use RAM backed storage which makes for much faster read/write access, however if you are cramped for RAM space and simulating a large device (ie 2GB) you can add a {{{cache_file=/tmp/nandimage}}} param to {{{nandsim}}} to tell it to use a file-backed storage instead. You can remove the file once you {{{rmmod nandsim}}}.
     1711. populate NAND with an existing ubi:
     172{{{#!bash
     173sudo modprobe mtdblock
     174sudo dd if=image.ubi of=/dev/mtdblock0 bs=2048
     175}}}
     176 * the mtdblock module creates a {{{/dev/mtdblock0}}} device on your development host
     177 * Note that instead of the dd command above you could also have used {{{ubiformat /dev/mtd0}}} to wipe and format a new ubi but this section is documenting how to extract and alter contents of an existing ubi.
     1781. attach the UBI
     179{{{#!bash
     180sudo modprobe ubi
     181sudo ubiattach /dev/ubi_ctrl -m0 -O2048
     182sudo ubiinfo -a # optionally show info about the UBI
     183}}}
     1841. mount the ubifs to host
     185{{{#!bash
     186sudo modprobe ubifs
     187sudo mkdir /mnt/ubi # create a mountpoint
     188sudo mount -t ubifs ubi0 /mnt/ubi
     189ls /mnt/ubi # optionally list contents of mounted filesystem
     190}}}
     191
     192Once you are done working with the files in {{{/mnt/ubi}}} simply unmount the device:
     193{{{#!bash
     194sudo umount /mnt/ubi
     195}}}
     196
     197If you have altered the filesystem and wish to extract the altered UBI you can detach and use dd to copy it to a file:
     198{{{#!bash
     199sudo ubidetach /dev/ubi_ctrl -m0
     200sudo dd if=/dev/mtdblock0 of=image-updated.ubi bs=2048
     201}}}
     202
     203And when you are done working with the simulated NAND device completely, remove the {{{nandsim}}} module to free up your system memory:
     204{{{#!bash
     205sudo rmmod nandsim
     206}}}
     207
     208References:
     209 * http://www.linux-mtd.infradead.org/faq/nand.html#L_nand_nandsim
     210
     211
     212=== On a target board with NAND FLASH ===
     213[=#ubiontarget]
     214To extract or manipulate the contents of a UBI within the NAND Flash of a Linux system, Linux must be booted from a secondary medium on a device with NAND FLASH and without the NAND FLASH partition being mounted.
     215
     216The following examples assume that UBI has been configured to use the 3rd (zero based) MTD FLASH partition ({{{ubi.mtd=2}}} kernel command line) as we typically use the {{{mtdparts=nand:16m(uboot),1m(env),-(rootfs)}}} env variable in U-Boot to dynamically modify the Linux Device-Tree to partition the NAND flash device into 3 partitions: bootloader, bootloader-env, and rootfs. Thus the bootloader is {{{/dev/mtd0}}}, the bootloader env is {{{/dev/mtd1}}}, and the rootfs is {{{/dev/mtd2}}}.
     217
     218
     219Examples:
     220 * Mounting UBI NAND FLASH volume (from {{{/dev/mtd2}}}):
     221{{{#!bash
     222ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
     223ubiattach /dev/ubi_ctrl -m 2 # attach /dev/mtd2 as /dev/ubi0
     224mount -t ubifs ubi0:rootfs /mnt/ubi #mount 'rootfs' volume from /dev/ubi0 to /mnt/ubi
     225}}}
     226 * Updating NAND FLASH using linux target with NAND({{{/dev/mtd2}}}) with a UBI image:
     227{{{#!bash
     228ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
     229sync
     230ubiformat /dev/mtd2 -y -f image.ubi
     231}}}
     232 * Updating NAND FLASH partition ({{{/dev/mtd2}}}) with a UBIFS image
     233{{{#!bash
     234ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
     235ubiformat /dev/mtd2 -y # format (will wipe contents!)
     236ubiattach /dev/ubi_ctrl -m 2 # attach /dev/mtd2 as /dev/ubi0
     237ubimkvol /dev/ubi0 -N rootfs -m # create a volume on /dev/ubi0 named 'rootfs'
     238ubiupdatevol /dev/ubi0_0 image.ubifs
     239}}}
     240 * Updating NAND FLASH partition ({{{/dev/mtd2}}}) with a tarball
     241{{{#!bash
     242ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
     243ubiformat /dev/mtd2 -y # format (will wipe contents!)
     244ubiattach /dev/ubi_ctrl -m 2 # attach /dev/mtd2 as /dev/ubi0
     245ubimkvol /dev/ubi0 -N rootfs -m # create a volume on /dev/ubi0 named 'rootfs'
     246mkdir /mnt/ubi # create a mount-point
     247mount -t ubifs ubi0:rootfs /mnt/ubi # mount 'rootfs' volume from /dev/ubi0 to /mnt/ubi
     248tar xvf rootfs.tar.gz -C /mnt/ubi # extract tarball
     249umount /mnt/ubi # unmount filesystem
     250}}}
     251
     252
     253== Working with UBI in U-boot Bootloader ==
     254[=#ubiinuboot]
     255The U-boot bootloader used by Ventana has support for ubi:
     256 * mounting a ubi
     257 * removing ubi volumes
     258 * creating ubi volumes
     259 * writing ubi volumes
     260
     261
     262=== Updating NAND FLASH with a UBI image ===
     263If your ubi image is on a tftp server as {{{image.ubi}}} you can use the following to update the 'rootfs' NAND FLASH partition in U-Boot (update the ip addresses as appropriate for your network environment):
     264{{{#!bash
     265setenv ipaddr 192.168.1.1
     266setenv serverip 192.168.1.254
     267tftp ${loadaddr} image.ubi && \
     268  nand erase.part rootfs && \
     269  nand write ${loadaddr} rootfs ${filesize}
     270}}}
     271 * Note the {{{nand_update}}} script will perform the last step for you using the image name specified in the {{{image_rootfs}}} env variable such that you could do a {{{setenv image_rootfs image.ubi; run nand_update}}}
     272
     273If your ubi image is on the first partition of a FAT formated MMC called {{{image.ubi}}} you can do the following:
     274{{{#!bash
     275mmc rescan
     276fatload mmc 0:1 ${loadaddr} image.ubi && \
     277  nand erase.part rootfs && \
     278  nand write ${loadaddr} rootfs ${filesize}
     279}}}
     280
     281If your ubi image is on the first partition of a ext2/3/4 formated MMC called {{{image.ubi}}} you can do the following:
     282{{{#!bash
     283mmc rescan
     284ext2load mmc 0:1 ${loadaddr} image.ubi && \
     285  nand erase.part rootfs && \
     286  nand write ${loadaddr} rootfs ${filesize}
     287}}}
     288
     289If your ubi image is on the first partition of a FAT formatted USB stick called {{{image.ubi}}} you can do the following:
     290{{{#!bash
     291usb start
     292usb dev 0
     293fatload usb 0:1 ${loadaddr} image.ubi && \
     294  nand erase.part rootfs && \
     295  nand write ${loadaddr} rootfs ${filesize}
     296}}}
     297
     298'''Note that operating on the ubi device as described here destroys the wear-leveling information contained in the UBI therefore it is recommended to work at the ubi volume layer to replace ubifs volumes as described below'''
     299
     300
     301[=#uboot-ubifs]
     302=== Updating NAND FLASH with a ubifs image ===
     303To update the contents of a ubi with a ubifs in U-Boot:
     304{{{#!bash
     305ubi part rootfs # select the 'rootfs' partition (from mtdparts env var) for ubi
     306ubi info layout # list volumes
     307ubi remove rootfs # remove rootfs volume
     308ubi create rootfs a00000 static # create a 10MB volume named rootfs
     309tftp ${loadaddr} ventana/root.ubifs && ubi write ${loadaddr} rootfs ${filesize}
     310ubifsmount ubi0:rootfs # mount rootfs volume from ubi
     311ubifsls /
     312ubifsumount
     313}}}
     314
     315Operating on a NAND device at the ubi volume layer like this preserves the wear-leveling information for your flash device and is therefore recommended, especially if you are doing this often.
     316
     317Use the information in the section above for examples on how to load data in uboot from various sources such as network, mmc, usb.