Changes between Version 29 and Version 30 of ubuntu

Timestamp:

10/17/2023 10:41:31 PM (13 months ago)

Author:

Tim Harvey

Comment:

added rootfs generation sections

ubuntu

@@ -14 +14 @@
  * [wiki:ventana/ubuntu Ventana Ubuntu Page]
 [[Image(ubuntufocalwithversion.png,300px)]]
+
+== Ubuntu root filesystem
+Creating a root filesystem comprised of Canonical built packages can be done in a couple of ways:
+ * download a pre-built rootfs tarball
+ * create one using deboostrap
+ * create one using qemu/chroot
+
+This refers to the rootfs only. To easily build an entire system image, including the boot firmware and Ubuntu, look at the BSP for the various product families:
+ * [wiki:malibu/bsp Malibu BSP]
+ * [wiki:venice/bsp Venice BSP]
+ * [wiki:newport/bsp Newport BSP]
+
+=== Downloading pre-built rootfs tarball
+Gateworks has pre-built Ubuntu based rootfs tarballs at http://dev.gateworks.com/ubuntu/ that are created with the [https://github.com/gateworks/ubuntu-rootfs ubuntu-rootfs script].
+
+[=#qemu-chroot]
+=== Creating Ubuntu rootfs using qemu and chroot
+You can use qemu to create an Ubuntu based rootfs with very little in the way of dependencies. To do this you must boot a virtual machine for your architecture therefore you will need a kernel and a minimal root filesystem which can be easily built using buildroot.
+
+Example:
+* aarch64 rootfs:
+1. Start by defining some parameters
+{{{#!bash
+IMG=ubuntu.ext4
+SIZE=500M
+UBUNTU=22.04
+UBUNTU_REL=.3
+}}}
+1. Create a ramdisk based image (similar concept as rescue image but we add a file here via BR2_ROOTFS_OVERLAY)
+{{{#!bash
+sudo apt install build-essential git
+git clone http://github.com/buildroot/buildroot.git
+cd buildroot
+cat << EOF > configs/buildroot_defconfig
+BR2_aarch64=y
+BR2_cortex_a72=y
+BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
+BR2_TOOLCHAIN_BUILDROOT_CXX=y
+BR2_TARGET_GENERIC_HOSTNAME="localhost"
+BR2_ROOTFS_OVERLAY="overlay/"
+BR2_LINUX_KERNEL=y
+BR2_LINUX_KERNEL_USE_ARCH_DEFAULT_CONFIG=y
+BR2_PACKAGE_BUSYBOX_SHOW_OTHERS=y
+BR2_PACKAGE_E2FSPROGS=y
+# BR2_PACKAGE_E2FSPROGS_FSCK is not set
+BR2_PACKAGE_I2C_TOOLS=y
+BR2_PACKAGE_RNG_TOOLS=y
+BR2_PACKAGE_BASH=y
+BR2_PACKAGE_TAR=y
+BR2_PACKAGE_UTIL_LINUX_HWCLOCK=y
+BR2_TARGET_ROOTFS_CPIO=y
+BR2_TARGET_ROOTFS_CPIO_GZIP=y
+EOF
+# create init script
+mkdir -p overlay/etc/init.d
+cat <<EOF > overlay/etc/init.d/S99boostrap-ubuntu.sh
+#!/bin/sh
+
+if [ "\$1" != "start" ]; then
+    exit
+fi
+
+# setup
+mkfs.ext4 -F /dev/vda -b 4096
+mount /dev/vda /mnt
+cd /mnt/
+udhcpc -i eth0
+# install base rootfs from release tarball
+wget -c -P /tmp/ http://cdimage.ubuntu.com/ubuntu-base/releases/${UBUNTU}/release/ubuntu-base-${UBUNTU}${UBUNTU_REL}-base-arm64.tar.gz
+tar zxf /tmp/ubuntu-base-${UBUNTU}${UBUNTU_REL}-base-arm64.tar.gz -C /mnt
+# setup chroot environment
+mount -o bind /proc /mnt/proc/
+mount -o bind /sys/ /mnt/sys/
+mount -o bind /dev/ /mnt/dev/
+mount -o bind /dev/pts /mnt/dev/pts
+mount -t tmpfs tmpfs /mnt/var/lib/apt/
+mount -t tmpfs tmpfs /mnt/var/cache/apt/
+echo "nameserver 8.8.8.8" > /mnt/etc/resolv.conf
+echo "localhost" > /mnt/etc/hostname
+echo "127.0.0.1 localhost" > /mnt/etc/hosts
+export DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true LC_ALL=C LANGUAGE=C LANG=C
+# install additional packages
+chroot /mnt apt update
+chroot /mnt apt upgrade -y
+chroot /mnt apt install --no-install-recommends -y \
+     apt-utils locales less wget procps \
+     psmisc less sudo kmod rng-tools \
+     ifupdown net-tools isc-dhcp-client ntpdate \
+     openssh-server iproute2 iptables iputils-ping \
+     i2c-tools can-utils usbutils pciutils \
+     screen picocom \
+     vim nano
+# set root password
+echo -e "root\nroot" | chroot /mnt passwd
+# teardown chroot env
+umount /mnt/var/lib/apt/
+umount /mnt/var/cache/apt
+chroot /mnt apt clean
+chroot /mnt apt autoclean
+reboot
+EOF
+chmod +x overlay/etc/init.d/S99boostrap-ubuntu.sh
+make buildroot_defconfig
+make -j8
+}}}
+ * generic aarch64 kernel Image and rootfs.cpio.gz will be in output/images
+
+2. Use qemu to kick off a virtual aarch64 machine which will run the init script and  create root filesystem on a virtual disk
+{{{#!bash
+apt install qemu-system-arm
+truncate -s $SIZE $IMG
+qemu-system-aarch64 -m 1G -M virt -cpu cortex-a57 -nographic -smp 1 -no-reboot \
+   -device virtio-net-device,netdev=eth0 -netdev user,id=eth0 \
+   -device virtio-blk-device,drive=hd0 -drive file=$IMG,if=none,format=raw,id=hd0 \
+   -append "console=ttyAMA0 ip=dhcp" \
+   -kernel output/images/Image \
+   -initrd output/images/rootfs.cpio.gz
+}}}
+ * ubuntu.ext4 contains your rootfs - add desired changes to the setup script
+
+
+[=#debootstrap]
+=== Creating Ubuntu rootfs using deboostrap
+A popular way to create an Ubuntu root filesystem is to use the {{{deboostrap}}} utility on a Debian or Ubuntu host. This tool provides a 2-stage install where the second stage is within a chroot environment using qemu.
+
+Gateworks uses a script to do this which you may find at http://github.com/Gateworks/ubuntu-rootfs
+
+Requirements:
+- Linux Ubuntu or Debian System with network connection and sudo permissions
+
+Important notes:
+ * We set and use '''target''' and '''distro''' env variables in step 2 and use those env variables in the remaining steps to make this tutorial more version-agnostic. Please be aware of this and do not deviate from the steps unless or until you completely understand what you are doing.
+ * These steps are not always exactly what we do in our script but give you an idea of how you would go about doing it yourself if you wanted to customize something
+
+Steps:
+1. Install pre-requisites:
+{{{
+#!bash
+sudo apt-get install qemu-user-static debootstrap binfmt-support
+}}}
+
+2. Perform first stage install of minimal filesystem for {{{arm64}}} architecture:
+{{{#!bash
+distro=focal
+arch=arm64
+target=${distro}-${arch}
+qemu_arch=aarch64
+sudo debootstrap --arch=$arch --foreign $distro $target
+# copy qemu binary for the binfmt packages to find it and copy in resolv.conf from host
+sudo cp /usr/bin/qemu-${qemu_arch}-static $target/usr/bin
+}}}
+ * See http://ports.ubuntu.com/ubuntu-ports/dists/ for a list of current Ubuntu releases: oe 20.04=focal (latest LTS)
+ * this minimal rootfs is still missing some core packages and configuration before it can be booted. These steps are taken care of in a 2nd stage install within a chroot shell
+ * the chroot shell below will provide network support (inherited from the host)
+
+3. We now have a minimal Ubuntu rootfs - chroot to it and perform the 2nd stage install:
+{{{
+#!bash
+sudo chroot $target
+# now we are in the chroot - setup env matching the distro above
+distro=focal
+export LANG=C
+# setup second stage
+/debootstrap/debootstrap --second-stage
+}}}
+ * this is the most minimal rootfs we would recommend
+
+4. (optional) add additional apt package repos:
+{{{
+#!bash
+cat <<EOT > /etc/apt/sources.list
+deb http://ports.ubuntu.com/ubuntu-ports $distro main restricted universe multiverse
+deb http://ports.ubuntu.com/ubuntu-ports $distro-updates main restricted universe multiverse
+deb http://ports.ubuntu.com/ubuntu-ports $distro-security main restricted universe multiverse
+EOT
+}}}
+ * you may want to customize the above list, depending on your needs. See [#packages below] for more detail on Ubuntu package feeds
+
+5. (optional) update package database and setup locales (do not skip this step if you are needing to install any packages for the steps below or otherwise)
+{{{
+#!bash
+apt-get update
+apt-get -f install # fixup missing package dependencies
+apt-get install locales dialog
+dpkg-reconfigure locales
+}}}
+
+6. Set hostname:
+{{{
+#!bash
+echo ${distro}-$(uname -m) > /etc/hostname
+}}}
+
+7. set a root passwd so you can login
+{{{
+#!bash
+passwd
+}}}
+ - or consider adding a user via {{{adduser}}}:
+{{{
+#!bash
+adduser myuser
+usermod -a -G tty myuser # add to tty group for tty access
+usermod -a -G dialout myuser # add to dialout group for UART access
+usermod -a -G sudo myuser # add to sudo group for root access
+}}}
+
+8. (optional) configure networking:
+ - wired ethernet with DHCP on eth0
+{{{#!bash
+apt-get install net-tools ifupdown
+cat <<EOF >> /etc/network/interfaces
+allow-hotplug eth0
+auto eth0
+iface eth0 inet dhcp
+
+EOF
+}}}
+ - or static IP:
+{{{#!bash
+apt-get install net-tools ifupdown
+cat <<EOF >> /etc/network/interfaces
+allow-hotplug eth0
+auto eth0
+iface eth0 inet static
+address 192.168.1.1
+netmask 255.255.255.0
+gateway 192.168.1.254
+
+EOF
+}}}
+ - or wireless (requires ~3MB of additional packages):
+{{{
+#!bash
+apt-get install wpasupplicant iw
+cat << EOF >> /etc/network/interfaces
+# Wireless interface
+auto wlan0
+iface wlan0 inet dhcp
+        wireless_mode managed
+        wireless_essid any
+        wpa-driver nl80211
+        wpa-conf /etc/wpa_supplicant.conf
+
+EOF
+wpa_passphrase <myssid> <mypass> >> /etc/wpa_supplicant.conf
+}}}
+
+9. (optional) install some useful packages
+{{{
+#!bash
+apt-get install openssh-server # ssh server for remote access
+apt-get install can-utils i2c-tools usbutils pciutils # cmdline tools for various hardware support
+}}}
+ * Note that by default root ssh access is disabled for security. See [wiki:/ubuntu#SSHServer This link] for info on enabling it
+
+10. Exit the chroot shell and remove files we no longer need
+{{{
+#!bash
+exit
+sudo rm $target/usr/bin/qemu-$qemu_arch-static
+}}}
+
+At this point you have a directory containing a root filesystem (without kernel) and likely want to install it onto removable storage or the on-board FLASH of a target board. Some intermediate formats that are useful to keep around would be a tarball, perhaps an ext4 filesystem image, or a compressed disk image suitable for flashing in the U-Boot bootloader.
+
+To create a tarball which is the most flexible storage format and can be used for a variety of future installation uses:
+{{{#!bash
+sudo tar --keep-directory-symlink -cvJf focal-venice.tar.xz -C rootfs/ .
+}}}
+ * '--numeric-owner' is required to store user/group as a number instead of a name, your specific use case may require this switch. 
+ * the '-C rootfs/' is required to eliminate the rootfs directory prefix
+ * the sudo is needed to be able to read the root owned files
+ * '--keep-directory-symlink' will preserve symbolic links
+
+To create an ext4 filesystem from the directory or tarball requires you choose a size for the filesystem. This size can be increased at runtime using {{{resize2fs}}} as long as the partition table has room for it to grow. The advantage of using an as small as possible size is that the time necessary to flash it onto storage is reduced to a minimum (when flashing you have to write the entire ext4 fs but when formatting or resizing it only has to write periodic markers to FLASH).
+
+For a given size (see SIZEMB variable below) you can create a rootfs with:
+{{{#!bash
+SIZEMB=1536 # 1.5GB - expandable later with resize2fs
+OUT=rootfs.ext4
+# create a file of specific size
+truncate -s ${SIZEMB}M ${OUT}
+# format it as an ext4 filesystem
+mkfs.ext4 -q -F -L rootfs ${OUT}
+# mount it to a temporary mount point
+tmp_mnt=$(mktemp -d -p/tmp)
+mount ${OUT} ${tmp_mnt}
+# copy files to it
+cp -rup rootfs/* ${tmp_mnt}
+# and/or extract files from a tarball
+tar -C ${tmp_mnt} -xf linux-venice.tar.xz --keep-directory-symlink
+# unmount temporary mount point
+umount ${tmp_mnt}
+sync
+# compress it
+gzip -k -f ${OUT}
+}}}
 
 == Ubuntu Webmin