Linux Block Storage Devices

Imaging a Block Device (microSD) on a Linux PC

These are the instructions to flash something like Ubuntu on a microSD.

The overall theory is that to place a root filesystem on a removable block storage device (mSATA disk, microSD card, USB Mass Storage device) you need to to the following:

1. Partition the device
2. Format the partition(s)
3. Install the rootfs to the partition

The popular Linux filesystem used today is ext4 which has the following features:

• Built-in to most kernels (allowing it to be your root filesystem)
• Journalling support
• Can be re-sized even if mounted (via resize2fs)

The following commands will perform the above steps on a Linux development host PC/Laptop. For convenience we set an env variable to represent the block device - take care to set this properly so as not to format a partition vital to your host (such as its rootfs)

1. Insert the device (microSD) into the Linux PC (a USB to microSD adapter may be needed).

Use the ls command to verify that your microSD was detected by your Linux PC. Example below:

gw@cpu-660:~$ ls /dev/sd*
/dev/sda  /dev/sda1  /dev/sda2  /dev/sda5  /dev/sdb  /dev/sdb1  /dev/sdb2

• Note - /dev/sda is usually the main hard disk of the system. DO NOT write to this.
• In this case, no other media is connected to the PC, and thus the microSD is /dev/sdb
• Note - If nothing besides /dev/sda comes up, please refer to the mmcblk0 section

2. Set the DEVICE variable to the proper device, example given is /dev/sdc:

   DEVICE=/dev/sdc
   

3. Un-mount volumes

   umount ${DEVICE}?*
   

4. Partition device:

   printf "2048,,L,,\n" | sudo sfdisk -uS ${DEVICE}
   

   • Here we create a single Linux (type 83) partition offset at 1MiB - you can adjust the parameters to create multiple partitions depending on your needs

5. Format the partition as ext4

   sudo mkfs.ext4 ${DEVICE}1
   

   • Note that if you put this into a script it has been found you need to have a second or so of a sleep following the sfdisk to allow the OS to re-scan the partition table before it will realize that the partition exists
   • When using e2fsprogs 1.43+ (ie Ubuntu Bionic host) 64bit file system support is enabled by default which is not supported by older U-Boot versions used on Ventana (U-Boot 2017+ should support 64bit ext file systems). To disable using 64bit add '-O ^64bit' to the mkfs.ext4 command line.

6. Mount the partition:

   sudo mount ${DEVICE}1 /mnt/disk
   

   • Make sure /mnt/disk exists - this is simply a mount-point. If it does not exist, use the mkdir command to create it.

7. Un-archive the rootfs tarball to the /mnt/disk location:

   sudo tar -C /mnt/disk -xvf rootfs.tar.bz2 --keep-directory-symlink
   

   • If instead you have a rootfs directory created deboostrap for example you can copy it with a sudo cp -rupv rootfs/* /mnt/disk.

8. Unmount the partition:

   sudo umount /mnt/disk
   

   • Allow unmount to finish completely before removing the drive. Prematurely removing the drive can cause file corruption and strange errors to occur. Running umount in a different shell is not equal to allowing it to complete in the original shell.

9. Remove the device from the PC and insert/connect to Gateworks SBC

Imaging from U-Boot

For block device based boards, Gateworks has created an installable U-Boot script that will apply block device image files containing either an entire disk image or a single partition image (see the below section for creating disk images). This can be particularly useful for updating the firmware of a running target.

To update a block device:

1. Compress your image

In most situations the DRAM of your device will be smaller than the image file required to update your system, therefore it is necessary to split the image file and apply the resulting compressed parts in a piece wise update.

FILE=openwrt_1602_8gb.img
# Split file every 500M with suffix ".part" and decimal increment
split -d -b 500M ${FILE} ${FILE}.part
gzip ${FILE}.*
for i in $(ls ${FILE}.*); do
 mv $i ${FILE}.gz.$(echo $i | cut -d'.' -f3)
done

# Result will be 16 files such as openwrt_1602_8gb.img.gz.partXX 
# which adhere to the naming convention expected by the script

If you have a small image file that when compressed will fit into your device's DRAM, a simple gzip -k ${FILE} will suffice.

2. Install the U-Boot script

On the target machine, source the ​block_image_update.scr U-Boot script attached to this page (or copy its contents with an editor). For example via tftp:

setenv serverip 192.168.1.100
tftp block_image_update.scr
source $loadaddr

3. Configure your U-Boot environment

From the script's usage:

The following environment variables need to be set in order for this script to run:

  imagefile    - full path file name of gzipped (.gz) file
  devtype      - interface type of target block device
  devnum       - device number of target
  partoffset   - start of target partition in bytes (0 for full disk image)

Optional environment variable arguments:

  splitsuffix  - the suffix of the split gzipped file (eg .part for file.img.gz.part00)
  imageload    - the command, up to the file name, that will load your imagefile to 
                 loadaddr (defaults to [tftp 0x12000000])

An example configuration:

setenv imagefile openwrt_1602_8gb.img.gz   # image file from example above
setenv devtype mmc                         # mmc interface for eMMC or uSD
setenv devnum 0                            # first device
setenv partoffset 100000                   # target the 1st partition at 1M offset
setenv splitsuffix ".part"                 # suffix from example above
setenv imageload "tftp ${loadaddr}"        # command to load the image file (can be ext4load)

4. Run the added script with run block_image_update

Disk Images

Compressed Disk Images (technically 'block device images') can be easily created on a Linux system with dd and gzip. This works for any block device (ie rotational disk, SSD, uSD, USB Mass Storage disk) and doesn't care at all what is on the disk. This is a very common technique for distributing pre-built OS images because it preserves the partitioning scheme of the disk you are imaging.

Compared to distributing filesystem images or archives of filesystem contents this has method has some pro's and con's:

• Pros:
  • Other OS's can install this type of image onto another block storage device (such as Windows via ​USB Image Tool)
• Cons:
  • Images are created from a fixed-size block device and require the same or larger size block device to be installed onto
  • After installation any additional storage space (ie installing an image meant for a 2GB microSD onto a 16GB microSD) can not be used until the disk is re-partitioned and re-sized

• Device sizes vary greatly - various manufacturers 4GB stick will likely not have the same number of cylinders (thus bytes) as another. If you try to expand an image onto a smaller device it will fail. Therefore you should probably tell dd to only grab perhaps 95% of the disk. For example, if you are imaging a 4GB stick (1024*0.95=972). I have 3x 4GB block devices here and the smallest is shown by sfdisk as 3.677GiB.

  sudo dd if=/dev/sdc bs=4M count=941 conv=notrunc,noerror | gzip -c > disk-img.gz
  

Creating a Disk Image

Often disk images are provided already by Gateworks. If so, please skip this section. However, to create a disk image from the data already on a microSD, proceed with the following instructions.

Procedure to create disk image from the microSD /dev/sdc:

1. (Optional) Zero out the storage device first (if not already done) to allow for the best compression as even if previous contents have been 'deleted' the data still exists (the filesystem directory table has just unlinked from it):

   sudo dd if=/dev/zero of=/dev/sdc bs=4M
   

2. Copy and gzip to a file:

   sudo dd if=/dev/sdc bs=4M conv=sync,noerror | gzip -c > disk-img.gz
   

   • You can use another compressor other than gzip if you wish but make sure those needing to use these images can support that compression
   • The conv=sync,noerror tells dd to not fail on error and pad any partial or failed blocks with zeros

Note that these steps can take some time depending on the speed of your storage device. For example each step can take 10mins on an 8GB micro-SD over USB or 3mins on a 2GB micro-SD over USB

Using a Disk Image

Video: ​https://youtu.be/WtNqAj8YKoA

This procedure is to write a disk image to a microSD.

Insert the microSD into a Linux desktop or laptop PC. (not the Gateworks SBC)

The microSD should mount to the filesystem as something like /dev/sdb or /dev/sdc.

Verify on the system what letter it mounted as and be sure to not overwrite the main disk on the PC, which is typically /dev/sda.

Adjust the disk letter in the command below to write to the microSD with the pre-built disk image: (/dev/sdc is shown as an example only)

• Linux:

  zcat disk-img.gz | sudo dd of=/dev/sdc bs=4M
  

• Windows:
  • See ​USB Image Tool

Note that this can take a some time depending on the write speed of your storage device. For example you will likely see times such as 4mins for 2GB and 30mins for 8GB micro-SD

Once booted you can use OS specific commands to resize partitions to take advantage any unused space on the device (for example, if the image was created using a 2GB microSD and you placed it onto a 16GB you can expand partitions to use the extra 14GB)

You can do this in Linux for an ext2/3/4 filesystem with resize2fs. Because resize2fs will not alter the partition table you need to do that first.

The following example assumes you have a single ext filesystem and will resize it using resize2fs, partprobe, and sfdisk:

1. Install necessary tools (assuming Ubuntu/debian):

   apt-get install parted sfdisk e2fsprogs
   

2. Use sfdisk to repartition:

   DEVICE=/dev/mmcblk0 # IMX6 microSD host (use /dev/sda for a USB or SATA disk)
   df -h . # show current size/used/avail
   sfdisk --list ${DEVICE} # show current
   printf "2048,,L,,\n" | sfdisk --force  --no-reread -uS ${DEVICE} # re-create partition
   sfdisk --list ${DEVICE} # show new
   

3. Use partprobe (or reboot system) to force kernel to re-read the partition table of device with mounted filesystems:

   partprobe # re-read partition table
   

4. Use resize2fs to resize a partition to take up newly allocated space:

   resize2fs ${DEVICE}*1 # resize first partition