Changes between Version 17 and Version 18 of provisioning

Timestamp:

08/19/2021 12:48:47 AM (3 years ago)

Author:

Cale Collins

Comment:

added provisioning notes from ticket 949

provisioning

@@ -571 +571 @@
 
 
-
+= Provisioning Newport, direct write to eMMC on target
+
+Usint the bootloader's "gzwrite" method for flashig embedded flash has a lot of downsides; the gzipped image must be smaller than the capacity of the ram and it's difficult to create a minimal filesystem to speed up the writes. Depending on your application a more flexible and potentially time efficient aproach would be to use 'linux kernel+ramdisk + provisioning' script approach.  Using this method may be neceessary if there are scripts or applications that must be run during the provisioning process. 
+
+* [wiki:/buildroot#initrdinitialramdisk Buildroot Ramdisk Wiki]
+
+1. Boot to Target - either from the alternate boot device (ie boot from microSD if working on eMMC or visa versa) or boot a kernel+ramdisk so that you can work off either device. We provide a pre-built ramdisk image with all the necessary tools.  A secondary boot media can be use although it will not be as fast as the ramdisk.
+1. Partition the device with a partition size just large enough to fit a filesystem that can contain your files.  Determine the size in bytes needed for your filesystem, there are several ways to do this including simple trial and error.  If you have a directory with your files you can use 'du -b --summarize' to get bytes. For a tar.gz a good estimate is the uncompressed size "gzip -l <file.gz>".
+{{{#!bash
+# for an existing directory using du
+SZ_B=$(du -b --summarize /usr/src/ubuntu-rootfs/focal-newport | awk '{
+print $1; }'
+# or just guess and use trial and error
+SZ_B=$((1536*1024*1024)) # 1536MiB fits the Gateworks ubuntu image
+}}}
+1. Create partition table [wiki:/provisioning#ProvisioningfromaLinuxramdiskenvironment Example using GPT and sgdisk]. For Newport we need to preserve the FATFS filesystem used by the boot firmware.
+{{{#!bash
+DEV=/dev/mmcblk0
+GUID_BASIC_DATA=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
+GUID_RESERVED=8DA63339-0007-60C0-C436-083AC8230908
+GUID_LINUXFS=0FC63DAF-8483-4772-8E79-3D69D8477DE4
+# first use parted to create a GPT as sgdisk does not like the default
+MBR based partition table
+parted --script $DEV mklabel gpt
+# use sgdisk to create partitions
+sgdisk -og $DEV
+sgdisk -n 1:2048:26623 -c 1:"fatfs" -t 1:$GUID_BASIC_DATA $DEV
+sgdisk -n 2:28672:30719 -c 2:"atf" -t 2:$GUID_RESERVED $DEV
+sgdisk -n 3:30720:32767 -c 3:"uboot" -t 3:$GUID_RESERVED $DEV
+END=$(( 32768 + $((SZ_B/512)) ))
+sgdisk -n 4:32768:$END -c 4:"root" -t 4:$GUID_LINUXFS $DEV
+# set the legacy 'boot' attribute on rootfs partition so the uboot env
+will boot from it
+sgdisk -A 4:set:4 $DEV
+# print partition table
+sgdisk -p $DEV
+}}}
+1. Create your filesystem using the index of the partition for root as shown above. Note that rootfs is p4, this is because of the boot firmware (atf/uboot).  Format the partition that will contain the rootfs to ext4.
+{{{#!bash
+mkfs.ext4 -q -F -L rootfs ${DEV}p4
+}}}
+1. Mount your filesystem, populate it, then umount.
+{{{#!bash
+mount ${DEV}p4 /mnt
+cd /tmp
+udhcpc -e eth0
+wget http://dev.gateworks.com/ubuntu/focal/focal-newport.tar.xz
+wget http://dev.gateworks.com/newport/kernel/linux-newport.tar.xz
+tar -C /mnt -xf focal-newport.tar.xz --keep-directory-symlink
+2>/dev/null || echo Error!!!
+tar -C /mnt -xf linux-newport.tar.xz --keep-directory-symlink
+2>/dev/null || echo Error!!!
+umount /mnt
+#check for errors returned from tar such as not enough space
+}}}
+** Note: When creating tarballs use the --numeric-owner to preserve UID/GID numbers vs names and there is no need to be selective about files/directories on a non-live filesystem **
+6. Now you can dd and decompress your image up to the last block in the partition. The last block in the partition is $END which you can see by printing the partition table:
+{{{#!bash
+rm /tmp/*
+dd if=$DEV bs=512 count=$END | gzip > /tmp/disk.img.gz
+}}}
+7. Install a compressed disk image:
+{{{#!bash
+cd /tmp
+wget disk.img.gz
+zcat disk.img.gz | dd of=$DEV bs=4M
+}}}
+
+== Provisioning Newport removable block storage on Desktop workstation 
+
+Provisioning on the target is less than ideal in situations where a traditional workstation can be used.  Both methods have their advantages and disadvantges.  When using a desktop an issue may arrise when using partitioning tools, as most of them do not like working on 'files' and won't let you do things like partition past the size of the file. In those cases there are, and we provide, tools that create partition tables for you without adhering to any rules. 
+
+1. Create a 'file' for your filesystem image. 'truncate' can be used as it creates a 'sparse' file which reports the size it was created with but won't take up space for zero'd blocks.
+{{{#!bash
+truncate -s 1536M fs.img #create a sparse file 
+mkfs.ext4 -q -F -L rootfs fs.img #format it.  
+}}}
+1. Mount your filesystem image and populate it:
+{{{#!bash
+sudo mount fs.img /mnt/disk
+wget http://dev.gateworks.com/ubuntu/focal/focal-newport.tar.xz
+wget http://dev.gateworks.com/newport/kernel/linux-newport.tar.xz
+sudo tar -C /mnt/disk/ -xf focal-newport.tar.xz --keep-directory-symlink >/dev/null || echo "Error!!!"
+sudo tar -C /mnt/disk/ -xf linux-newport.tar.xz --keep-directory-symlink >/dev/null || echo "Error!!!"
+sudo umount /mnt/disk
+}}}
+1. Create a 'file' for your disk image. This needs to be large enough for your boot firmware (which includes the partition table) as well as the filesystems you will copy to it.
+{{{#!bash
+truncate -s $((16+1536))M disk.img
+}}}
+1. Copy the boot firmware to it at offset 0 
+{{{#!bash
+dd if=firmware-newport.img of=disk.img conv=notrunc #make sure to use conv=notrunc so it doesn't truncate your image file
+}}}
+1. Copy your filesystem image to the correct sectors dictated by the partition table. If needed adjust/re-create the partition table taking care to ensure the boot firmware FATFS does not move.
+{{{#!bash
+dd if=fs.img of=disk.img bs=512 seek=32768 conv=notrunc
+}}}
+1. Compress.
+{{{#!bash
+gzip disk.img
+}}}
+
+In conclusion, a partition is simply a chunk of space dictated by a header in the partition table which is in the first few blocks of the disk. It doesn't need to be filled, there can be a large partition with a small filesystem inside of it, that filesystem can be enlarged on first-boot with 'resize2fs'. Additionally the partition size can be incresead as well.