Changes between Version 17 and Version 18 of provisioning


Ignore:
Timestamp:
08/19/2021 12:48:47 AM (11 months ago)
Author:
Cale Collins
Comment:

added provisioning notes from ticket 949

Legend:

Unmodified
Added
Removed
Modified
  • provisioning

    v17 v18  
    571571
    572572
    573 
     573= Provisioning Newport, direct write to eMMC on target
     574
     575Usint the bootloader's "gzwrite" method for flashig embedded flash has a lot of downsides; the gzipped image must be smaller than the capacity of the ram and it's difficult to create a minimal filesystem to speed up the writes. Depending on your application a more flexible and potentially time efficient aproach would be to use 'linux kernel+ramdisk + provisioning' script approach.  Using this method may be neceessary if there are scripts or applications that must be run during the provisioning process.
     576
     577* [wiki:/buildroot#initrdinitialramdisk Buildroot Ramdisk Wiki]
     578
     5791. Boot to Target - either from the alternate boot device (ie boot from microSD if working on eMMC or visa versa) or boot a kernel+ramdisk so that you can work off either device. We provide a pre-built ramdisk image with all the necessary tools.  A secondary boot media can be use although it will not be as fast as the ramdisk.
     5801. Partition the device with a partition size just large enough to fit a filesystem that can contain your files.  Determine the size in bytes needed for your filesystem, there are several ways to do this including simple trial and error.  If you have a directory with your files you can use 'du -b --summarize' to get bytes. For a tar.gz a good estimate is the uncompressed size "gzip -l <file.gz>".
     581{{{#!bash
     582# for an existing directory using du
     583SZ_B=$(du -b --summarize /usr/src/ubuntu-rootfs/focal-newport | awk '{
     584print $1; }'
     585# or just guess and use trial and error
     586SZ_B=$((1536*1024*1024)) # 1536MiB fits the Gateworks ubuntu image
     587}}}
     5881. Create partition table [wiki:/provisioning#ProvisioningfromaLinuxramdiskenvironment Example using GPT and sgdisk]. For Newport we need to preserve the FATFS filesystem used by the boot firmware.
     589{{{#!bash
     590DEV=/dev/mmcblk0
     591GUID_BASIC_DATA=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
     592GUID_RESERVED=8DA63339-0007-60C0-C436-083AC8230908
     593GUID_LINUXFS=0FC63DAF-8483-4772-8E79-3D69D8477DE4
     594# first use parted to create a GPT as sgdisk does not like the default
     595MBR based partition table
     596parted --script $DEV mklabel gpt
     597# use sgdisk to create partitions
     598sgdisk -og $DEV
     599sgdisk -n 1:2048:26623 -c 1:"fatfs" -t 1:$GUID_BASIC_DATA $DEV
     600sgdisk -n 2:28672:30719 -c 2:"atf" -t 2:$GUID_RESERVED $DEV
     601sgdisk -n 3:30720:32767 -c 3:"uboot" -t 3:$GUID_RESERVED $DEV
     602END=$(( 32768 + $((SZ_B/512)) ))
     603sgdisk -n 4:32768:$END -c 4:"root" -t 4:$GUID_LINUXFS $DEV
     604# set the legacy 'boot' attribute on rootfs partition so the uboot env
     605will boot from it
     606sgdisk -A 4:set:4 $DEV
     607# print partition table
     608sgdisk -p $DEV
     609}}}
     6101. Create your filesystem using the index of the partition for root as shown above. Note that rootfs is p4, this is because of the boot firmware (atf/uboot).  Format the partition that will contain the rootfs to ext4.
     611{{{#!bash
     612mkfs.ext4 -q -F -L rootfs ${DEV}p4
     613}}}
     6141. Mount your filesystem, populate it, then umount.
     615{{{#!bash
     616mount ${DEV}p4 /mnt
     617cd /tmp
     618udhcpc -e eth0
     619wget http://dev.gateworks.com/ubuntu/focal/focal-newport.tar.xz
     620wget http://dev.gateworks.com/newport/kernel/linux-newport.tar.xz
     621tar -C /mnt -xf focal-newport.tar.xz --keep-directory-symlink
     6222>/dev/null || echo Error!!!
     623tar -C /mnt -xf linux-newport.tar.xz --keep-directory-symlink
     6242>/dev/null || echo Error!!!
     625umount /mnt
     626#check for errors returned from tar such as not enough space
     627}}}
     628** Note: When creating tarballs use the --numeric-owner to preserve UID/GID numbers vs names and there is no need to be selective about files/directories on a non-live filesystem **
     6296. Now you can dd and decompress your image up to the last block in the partition. The last block in the partition is $END which you can see by printing the partition table:
     630{{{#!bash
     631rm /tmp/*
     632dd if=$DEV bs=512 count=$END | gzip > /tmp/disk.img.gz
     633}}}
     6347. Install a compressed disk image:
     635{{{#!bash
     636cd /tmp
     637wget disk.img.gz
     638zcat disk.img.gz | dd of=$DEV bs=4M
     639}}}
     640
     641== Provisioning Newport removable block storage on Desktop workstation
     642
     643Provisioning on the target is less than ideal in situations where a traditional workstation can be used.  Both methods have their advantages and disadvantges.  When using a desktop an issue may arrise when using partitioning tools, as most of them do not like working on 'files' and won't let you do things like partition past the size of the file. In those cases there are, and we provide, tools that create partition tables for you without adhering to any rules.
     644
     6451. Create a 'file' for your filesystem image. 'truncate' can be used as it creates a 'sparse' file which reports the size it was created with but won't take up space for zero'd blocks.
     646{{{#!bash
     647truncate -s 1536M fs.img #create a sparse file
     648mkfs.ext4 -q -F -L rootfs fs.img #format it. 
     649}}}
     6501. Mount your filesystem image and populate it:
     651{{{#!bash
     652sudo mount fs.img /mnt/disk
     653wget http://dev.gateworks.com/ubuntu/focal/focal-newport.tar.xz
     654wget http://dev.gateworks.com/newport/kernel/linux-newport.tar.xz
     655sudo tar -C /mnt/disk/ -xf focal-newport.tar.xz --keep-directory-symlink >/dev/null || echo "Error!!!"
     656sudo tar -C /mnt/disk/ -xf linux-newport.tar.xz --keep-directory-symlink >/dev/null || echo "Error!!!"
     657sudo umount /mnt/disk
     658}}}
     6591. Create a 'file' for your disk image. This needs to be large enough for your boot firmware (which includes the partition table) as well as the filesystems you will copy to it.
     660{{{#!bash
     661truncate -s $((16+1536))M disk.img
     662}}}
     6631. Copy the boot firmware to it at offset 0
     664{{{#!bash
     665dd if=firmware-newport.img of=disk.img conv=notrunc #make sure to use conv=notrunc so it doesn't truncate your image file
     666}}}
     6671. Copy your filesystem image to the correct sectors dictated by the partition table. If needed adjust/re-create the partition table taking care to ensure the boot firmware FATFS does not move.
     668{{{#!bash
     669dd if=fs.img of=disk.img bs=512 seek=32768 conv=notrunc
     670}}}
     6711. Compress.
     672{{{#!bash
     673gzip disk.img
     674}}}
     675
     676In conclusion, a partition is simply a chunk of space dictated by a header in the partition table which is in the first few blocks of the disk. It doesn't need to be filled, there can be a large partition with a small filesystem inside of it, that filesystem can be enlarged on first-boot with 'resize2fs'. Additionally the partition size can be incresead as well.