Changes between Version 17 and Version 18 of provisioning

08/19/2021 12:48:47 AM (11 months ago)
Cale Collins

added provisioning notes from ticket 949


  • provisioning

    v17 v18  
     573= Provisioning Newport, direct write to eMMC on target
     575Usint the bootloader's "gzwrite" method for flashig embedded flash has a lot of downsides; the gzipped image must be smaller than the capacity of the ram and it's difficult to create a minimal filesystem to speed up the writes. Depending on your application a more flexible and potentially time efficient aproach would be to use 'linux kernel+ramdisk + provisioning' script approach.  Using this method may be neceessary if there are scripts or applications that must be run during the provisioning process.
     577* [wiki:/buildroot#initrdinitialramdisk Buildroot Ramdisk Wiki]
     5791. Boot to Target - either from the alternate boot device (ie boot from microSD if working on eMMC or visa versa) or boot a kernel+ramdisk so that you can work off either device. We provide a pre-built ramdisk image with all the necessary tools.  A secondary boot media can be use although it will not be as fast as the ramdisk.
     5801. Partition the device with a partition size just large enough to fit a filesystem that can contain your files.  Determine the size in bytes needed for your filesystem, there are several ways to do this including simple trial and error.  If you have a directory with your files you can use 'du -b --summarize' to get bytes. For a tar.gz a good estimate is the uncompressed size "gzip -l <file.gz>".
     582# for an existing directory using du
     583SZ_B=$(du -b --summarize /usr/src/ubuntu-rootfs/focal-newport | awk '{
     584print $1; }'
     585# or just guess and use trial and error
     586SZ_B=$((1536*1024*1024)) # 1536MiB fits the Gateworks ubuntu image
     5881. Create partition table [wiki:/provisioning#ProvisioningfromaLinuxramdiskenvironment Example using GPT and sgdisk]. For Newport we need to preserve the FATFS filesystem used by the boot firmware.
     594# first use parted to create a GPT as sgdisk does not like the default
     595MBR based partition table
     596parted --script $DEV mklabel gpt
     597# use sgdisk to create partitions
     598sgdisk -og $DEV
     599sgdisk -n 1:2048:26623 -c 1:"fatfs" -t 1:$GUID_BASIC_DATA $DEV
     600sgdisk -n 2:28672:30719 -c 2:"atf" -t 2:$GUID_RESERVED $DEV
     601sgdisk -n 3:30720:32767 -c 3:"uboot" -t 3:$GUID_RESERVED $DEV
     602END=$(( 32768 + $((SZ_B/512)) ))
     603sgdisk -n 4:32768:$END -c 4:"root" -t 4:$GUID_LINUXFS $DEV
     604# set the legacy 'boot' attribute on rootfs partition so the uboot env
     605will boot from it
     606sgdisk -A 4:set:4 $DEV
     607# print partition table
     608sgdisk -p $DEV
     6101. Create your filesystem using the index of the partition for root as shown above. Note that rootfs is p4, this is because of the boot firmware (atf/uboot).  Format the partition that will contain the rootfs to ext4.
     612mkfs.ext4 -q -F -L rootfs ${DEV}p4
     6141. Mount your filesystem, populate it, then umount.
     616mount ${DEV}p4 /mnt
     617cd /tmp
     618udhcpc -e eth0
     621tar -C /mnt -xf focal-newport.tar.xz --keep-directory-symlink
     6222>/dev/null || echo Error!!!
     623tar -C /mnt -xf linux-newport.tar.xz --keep-directory-symlink
     6242>/dev/null || echo Error!!!
     625umount /mnt
     626#check for errors returned from tar such as not enough space
     628** Note: When creating tarballs use the --numeric-owner to preserve UID/GID numbers vs names and there is no need to be selective about files/directories on a non-live filesystem **
     6296. Now you can dd and decompress your image up to the last block in the partition. The last block in the partition is $END which you can see by printing the partition table:
     631rm /tmp/*
     632dd if=$DEV bs=512 count=$END | gzip > /tmp/disk.img.gz
     6347. Install a compressed disk image:
     636cd /tmp
     637wget disk.img.gz
     638zcat disk.img.gz | dd of=$DEV bs=4M
     641== Provisioning Newport removable block storage on Desktop workstation
     643Provisioning on the target is less than ideal in situations where a traditional workstation can be used.  Both methods have their advantages and disadvantges.  When using a desktop an issue may arrise when using partitioning tools, as most of them do not like working on 'files' and won't let you do things like partition past the size of the file. In those cases there are, and we provide, tools that create partition tables for you without adhering to any rules.
     6451. Create a 'file' for your filesystem image. 'truncate' can be used as it creates a 'sparse' file which reports the size it was created with but won't take up space for zero'd blocks.
     647truncate -s 1536M fs.img #create a sparse file
     648mkfs.ext4 -q -F -L rootfs fs.img #format it. 
     6501. Mount your filesystem image and populate it:
     652sudo mount fs.img /mnt/disk
     655sudo tar -C /mnt/disk/ -xf focal-newport.tar.xz --keep-directory-symlink >/dev/null || echo "Error!!!"
     656sudo tar -C /mnt/disk/ -xf linux-newport.tar.xz --keep-directory-symlink >/dev/null || echo "Error!!!"
     657sudo umount /mnt/disk
     6591. Create a 'file' for your disk image. This needs to be large enough for your boot firmware (which includes the partition table) as well as the filesystems you will copy to it.
     661truncate -s $((16+1536))M disk.img
     6631. Copy the boot firmware to it at offset 0
     665dd if=firmware-newport.img of=disk.img conv=notrunc #make sure to use conv=notrunc so it doesn't truncate your image file
     6671. Copy your filesystem image to the correct sectors dictated by the partition table. If needed adjust/re-create the partition table taking care to ensure the boot firmware FATFS does not move.
     669dd if=fs.img of=disk.img bs=512 seek=32768 conv=notrunc
     6711. Compress.
     673gzip disk.img
     676In conclusion, a partition is simply a chunk of space dictated by a header in the partition table which is in the first few blocks of the disk. It doesn't need to be filled, there can be a large partition with a small filesystem inside of it, that filesystem can be enlarged on first-boot with 'resize2fs'. Additionally the partition size can be incresead as well.