Changes between Version 1 and Version 2 of linux/blockdev


Ignore:
Timestamp:
10/22/2017 06:02:01 AM (7 years ago)
Author:
Chris Lang
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • linux/blockdev

    v1 v2  
    55== Imaging a Block device ==
    66To place a root filesystem on a removable block storage device (mSATA disk, microSD card, USB Mass Storage device) you need to to the following:
    7  1. partition the device
    8  2. format the partition(s)
    9  3. install the rootfs to the partition
     7 1. Partition the device
     8 2. Format the partition(s)
     9 3. Install the rootfs to the partition
    1010
    1111The popular Linux filesystem used today is ext4 which has the following features:
    12  * built-in to most kernels (allowing it to be your root filesystem)
    13  * journalling support
    14  * can be re-sized even if mounted (via resize2fs)
     12 * Built-in to most kernels (allowing it to be your root filesystem)
     13 * Journalling support
     14 * Can be re-sized even if mounted (via {{{resize2fs}}})
    1515
    1616The following commands will perform the above steps on a Linux development host. For convenience we set an env variable to represent the block device - take care to set this properly so as not to format a partition vital to your host (such as its rootfs)
    1717
    18181. Determine device:
    19 {{{
     19{{{#!bash
    2020DEVICE=/dev/sdc
    2121}}}
    2222
    23232. Partition device:
    24 {{{
    25 printf ",,83,,\n" | sudo sfdisk ${DEVICE}
     24{{{#!bash
     25printf "2048,,L,,\n" | sudo sfdisk -uS ${DEVICE}
    2626}}}
    27  * here we create a single Linux (type 83) partition - you can adjust the parameters to create multiple partitions depending on your needs
     27 * here we create a single Linux (type 83) partition offset at 1MiB - you can adjust the parameters to create multiple partitions depending on your needs
    2828
    29 3. format the partition as ext4
    30 {{{
     293. Format the partition as ext4
     30{{{#!bash
    3131sudo mkfs.ext4 ${DEVICE}1
    3232}}}
    33  * Note that if you put this into a script I've found you need to have a second or so of a sleep following the sfdisk to allow the OS to re-scan the partition table before it will realize that the partition exists
     33 * Note that if you put this into a script I've found you need to have a second or so of a sleep following the {{{sfdisk}}} to allow the OS to re-scan the partition table before it will realize that the partition exists
    3434
    35 3. mount the partition:
    36 {{{
     353. Mount the partition:
     36{{{#!bash
    3737sudo mount ${DEVICE}1 /mnt/disk
    3838}}}
    39  * obviously make sure /mnt/disk exists - this is simply a mount-point
     39 * make sure /mnt/disk exists - this is simply a mount-point
    4040
    41 4. un-archive the rootfs tarball:
    42 {{{
     414. Un-archive the rootfs tarball:
     42{{{#!bash
    4343sudo tar -C /mnt/disk -xvf rootfs.tar.bz2
    4444}}}
    45  * if instead you have a rootfs directory created with fakeroot or deboostrap you can copy it with a {{{sudo cp -rupv rootfs/*  /mnt/disk}}}
     45 * If instead you have a rootfs directory created with {{{fakeroot}}} or {{{deboostrap}}} you can copy it with a {{{sudo cp -rupv rootfs/*  /mnt/disk}}}
    4646
    47 5. unmount the partition:
    48 {{{
     475. Unmount the partition:
     48{{{#!bash
    4949sudo umount /mnt/disk
    5050}}}
    5151
    5252
     53=== Imaging from U-Boot ===
     54For block device based boards, Gateworks has created an installable U-Boot script that will apply block device image files containing either an entire disk image or a single partition image (see the [#DiskImages below] section for creating disk images). This can be particularly useful for updating the firmware of a running target.
     55
     56To update a block device:
     57 '''1. Compress your image
     58  In most situations the DRAM of your device will be smaller than the image file required to update your system, therefore it is necessary to split the image file and apply the resulting compressed parts in a piece wise update.
     59 {{{#!bash
     60FILE=openwrt_1602_8gb.img
     61# Split file every 500M with suffix ".part" and decimal increment
     62split -d -b 500M ${FILE} ${FILE}.part
     63gzip ${FILE}.*
     64for i in $(ls ${FILE}.*); do
     65 mv $i ${FILE}.gz.$(echo $i | cut -d'.' -f3)
     66done
     67
     68# Result will be 16 files such as openwrt_1602_8gb.img.gz.partXX
     69# which adhere to the naming convention expected by the script
     70}}}
     71  If you have a small image file that when compressed will fit into your device's DRAM, a simple {{{gzip -k ${FILE} }}} will suffice.
     72 '''2. Install the U-Boot script
     73  On the target machine, source the [http://trac.gateworks.com/raw-attachment/wiki/linux/blockdev/block_image_update.scr block_image_update.scr] U-Boot script attached to this page (or copy its contents with an editor). For example via tftp:
     74{{{#!bash
     75setenv serverip 192.168.1.100
     76tftp block_image_update.scr
     77source $loadaddr
     78}}}
     79 '''3. Configure your U-Boot environment
     80  From the script's usage:
     81{{{
     82The following environment variables need to be set in order for this script to run:
     83
     84  imagefile    - full path file name of gzipped (.gz) file
     85  devtype      - interface type of target block device
     86  devnum       - device number of target
     87  partoffset   - start of target partition in bytes (0 for full disk image)
     88
     89Optional environment variable arguments:
     90
     91  splitsuffix  - the suffix of the split gzipped file (eg .part for file.img.gz.part00)
     92  imageload    - the command, up to the file name, that will load your imagefile to
     93                 loadaddr (defaults to [tftp 0x12000000])
     94}}}
     95  An example configuration:
     96{{{#!bash
     97setenv imagefile openwrt_1602_8gb.img.gz   # image file from example above
     98setenv devtype mmc                         # mmc interface for eMMC or uSD
     99setenv devnum 0                            # first device
     100setenv partoffset 100000                   # target the 1st partition at 1M offset
     101setenv splitsuffix ".part"                 # suffix from example above
     102setenv imageload "tftp ${loadaddr}"        # command to load the image file (can be ext4load)
     103}}}
     104
     105 '''4. Run the added script with {{{run block_image_update}}}
     106
    53107== Disk Images ==
    54 Compressed Disk Images (technically 'block device images') can be easily created on a Linux system with dd and gzip. This works for any block device (ie rotational disk, SSD, uSD, USB Mass Storage disk) and doesn't care at all what is on the disk. This is a very common technique for distributing pre-built OS images because it preserves the partitioning scheme of the disk you are imaging.
     108Compressed Disk Images (technically 'block device images') can be easily created on a Linux system with {{{dd}}} and {{{gzip}}}. This works for any block device (ie rotational disk, SSD, uSD, USB Mass Storage disk) and doesn't care at all what is on the disk. This is a very common technique for distributing pre-built OS images because it preserves the partitioning scheme of the disk you are imaging.
    55109
    56110Compared to distributing filesystem images or archives of filesystem contents this has method has some pro's and con's:
    57  * pro's:
     111 * Pro's:
    58112  * Other OS's can install this type of image onto another block storage device (such as Windows via [http://www.alexpage.de/usb-image-tool/ USB Image Tool])
    59  * con's:
    60   * images are created from a fixed-size block device and require the same or larger size block device to be installed onto
    61   * after installation any additional storage space (ie installing an image meant for a 2GB microSD onto a 16GB microSD) can not be used until the disk is re-partitioned and re-sized
     113 * Con's:
     114  * Images are created from a fixed-size block device and require the same or larger size block device to be installed onto
     115  * After installation any additional storage space (ie installing an image meant for a 2GB microSD onto a 16GB microSD) can not be used until the disk is re-partitioned and re-sized
    62116
    63  * device sizes vary greatly - various manufacturers 4GB stick will likely not have the same number of cylinders (thus bytes) as another.  If you try to expand an image onto a smaller device it will fail.  Therefore you should probably tell dd to only grab perhaps 95% of the disk.  For example, if you are imaging a 4GB stick (1024*0.95=972).  I have 3x 4GB block devices here and the smallest is shown by sfdisk as 3.677GiB.
    64 {{{
     117 * Device sizes vary greatly - various manufacturers 4GB stick will likely not have the same number of cylinders (thus bytes) as another.  If you try to expand an image onto a smaller device it will fail.  Therefore you should probably tell dd to only grab perhaps 95% of the disk.  For example, if you are imaging a 4GB stick (1024*0.95=972). I have 3x 4GB block devices here and the smallest is shown by {{{sfdisk}}} as 3.677GiB.
     118{{{#!bash
    65119sudo dd if=/dev/sdc bs=4M count=941 conv=notrunc,noerror | gzip -c > disk-img.gz
    66120}}}
    67121
    68122=== Creating a disk image ===
    69 Procedure to create disk image from /dev/sdc:
    70 1. (optional) zero out the storage device first (if not already done) to allow for the best compression as even if previous contents have been 'deleted' the data still exists (the filesystem directory table has just unlinked from it):
    71 {{{
     123Procedure to create disk image from {{{/dev/sdc}}}:
     1241. (Optional) Zero out the storage device first (if not already done) to allow for the best compression as even if previous contents have been 'deleted' the data still exists (the filesystem directory table has just unlinked from it):
     125{{{#!bash
    72126sudo dd if=/dev/zero of=/dev/sdc bs=4M
    73127}}}
    74128
    75 2. copy and gzip to a file:
    76 {{{
     1292. Copy and gzip to a file:
     130{{{#!bash
    77131sudo dd if=/dev/sdc bs=4M conv=sync,noerror | gzip -c > disk-img.gz
    78132}}}
    79  * you can use another compressor other than gzip if you wish but make sure those needing to use these images can support that compression
    80  * the conv=sync,noerror tells dd to not fail on error and pad any partial or failed blocks with zeros
     133 * You can use another compressor other than gzip if you wish but make sure those needing to use these images can support that compression
     134 * The {{{conv=sync,noerror}}} tells {{{dd}}} to not fail on error and pad any partial or failed blocks with zeros
    81135
    82136Note that these steps can take some time depending on the speed of your storage device. For example each step can take 10mins on an 8GB micro-SD over USB or 3mins on a 2GB micro-SD over USB
    83137
     138
    84139=== Using a disk image ===
    85 Procedure to write a disk image back to /dev/sdc:
     140Procedure to write a disk image back to {{{/dev/sdc}}}:
    86141 * Linux:
    87 {{{
     142{{{#!bash
    88143zcat disk-img.gz | sudo dd of=/dev/sdc bs=4M
    89144}}}
    90145 * Windows:
    91   - see [http://www.alexpage.de/usb-image-tool/ USB Image Tool]
     146  - See [http://www.alexpage.de/usb-image-tool/ USB Image Tool]
    92147
    93148Note that this can take a some time depending on the write speed of your storage device. For example you will likely see times such as 4mins for 2GB and 30mins for 8GB micro-SD
    94149
     150Once booted you can use OS specific commands to resize partitions to take advantage any unused space on the device (for example, if the image was created using a 2GB microSD and you placed it onto a 16GB you can expand partitions to use the extra 14GB)
     151
     152You can do this in Linux for an ext2/3/4 filesystem with {{{resize2fs}}}. Because {{{resize2fs}}} will not alter the partition table you need to do that first.
     153
     154The following example assumes you have a single ext filesystem and will resize it using {{{resize2fs}}}, {{{partprobe}}}, and {{{sfdisk}}}:
     155 1. Install necessary tools (assuming Ubuntu/debian):
     156{{{#!bash
     157apt-get install parted sfdisk e2fsprogs
     158}}}
     159 2. Use {{{sfdisk}}} to repartition:
     160{{{#!bash
     161DEVICE=/dev/mmcblk0 # IMX6 microSD host (use /dev/sda for a USB or SATA disk)
     162df -h . # show current size/used/avail
     163sfdisk --list ${DEVICE} # show current
     164printf "2048,,L,,\n" | sfdisk --force  --no-reread -uS ${DEVICE} # re-create partition
     165sfdisk --list ${DEVICE} # show new
     166}}}
     167 3. Use {{{partprobe}}} (or reboot system) to force kernel to re-read the partition table of device with mounted filesystems:
     168{{{#!bash
     169partprobe # re-read partition table
     170}}}
     171 4. Use {{{resize2fs}}} to resize a partition to take up newly allocated space:
     172{{{#!bash
     173resize2fs ${DEVICE}*1 # resize first partition
     174}}}