Changes between Version 30 and Version 31 of linux/kernel


Ignore:
Timestamp:
10/18/2022 07:53:48 PM (19 months ago)
Author:
Tim Harvey
Comment:

refactor the kernel build instructions; clean up toolchain definition and add more detail to build artifacts including fit image and bootscript

Legend:

Unmodified
Added
Removed
Modified

  • linux/kernel

    v30 v31  
    4949
    5050[=#toolchain]
    51 == Toolchains ==
    52 A toolchain is a set of compiler tools (typically gcc) and libraries necessary for compiling and linking source (and possibly debugging, diss-assembling, etc).
     51== Cross Toolchains ==
     52A toolchain is a set of compiler tools (typically gcc) and libraries necessary for compiling and linking source (and possibly debugging, diss-assembling, etc). A Cross toolchain (aka cross compiler) runs under a host architecture but produces binaries for a target architecture (ie x86-64 executable compiler producing code for aarch64 architecutre)
    5353
    5454Typically this includes the following (for the GNU C Compiler):
    … …  
    6969
    7070You likely have one of the following toolchains to work with:
    71  * toolchain from an OpenWrt build directory
    72  * toolchain from a prebuilt OpenWrt BSP SDK
    73  * toolchain from a Yocto build directory
    74  * toolchain from a prebuilt Yocto BSP SDK
    75  * pre-built toolchain from somewhere else, like Android
     71 * cross toolchain from your dev host distro (ie Ubuntu)
     72 * cross toolchain from an OpenWrt build directory
     73 * cross toolchain from a prebuilt OpenWrt BSP SDK
     74 * cross toolchain from a Yocto build directory
     75 * cross toolchain from a prebuilt Yocto BSP SDK
     76 * pre-built cross toolchain from somewhere else, like Android
    7677
    7778
    … …  
    103104'''Prerequisites:'''
    104105- Linux development host: Gateworks uses and supports Ubuntu which is used here as an example, but other Linux variants can work as well
    105 - Toolchain for the CPU architecture of the boards you are working with:
    106  * Newport uses the Cavium CN80xx OcteonTX SoC which has ARMv8 CPU cores. You can download the toolchain provided Cavium for their SDK [http://dev.gateworks.com/newport/tools-gcc-6.2.tar.bz2 here] or use one that you have pre-built from OpenWrt
    107  * Ventana uses the NXP/Freescale IMX6 SoC which has ARM Cortex-A9 cores with NEON SIMD support. You can download our pre-built OpenWrt toolchain [http://dev.gateworks.com/openwrt/14.08/imx6/OpenWrt-Toolchain-imx6-for-arm_cortex-a9+neon-gcc-4.8-linaro_uClibc-0.9.33.2_eabi.tar.bz2 here]
    108  * Laguna uses the Cavium cns3xxx SoC which has ARM11 MP-Core cores
    109 
    110 Steps to build the Gateworks kernel using the Gateworks OpenWrt toolchain:
     106- Cross Toolchain for the CPU architecture of the boards you are working with
     107- Various other tools required by the Linux kernel build system
     108
     109Steps to build the full Gateworks kernel (modules, FIT image, dtbs, bootscript and all):
    1111101. Install pre-requisites:
    112111{{{#!bash
    … …  
    115114  * typically 'build-essential' provides enough for compiling, however we need a few other things for kernel development:
    116115   - ncurses-dev is needed for menuconfig
    117    - u-boot-tools, bc, and lzop are needed for uImage
     116   - u-boot-tools, bc, and lzop are needed for uImage (used for ARM 32-bit architectures only)
    118117   - git is needed for checking out the source
    119118
    120 2. Obtain and install compiler toolchain:
    121  * Venice: Use the Venice BSP located [wiki:venice/bsp Venice BSP Wiki Page]
    122  * For Newport, you can use the pre-built toolchain provided by Cavium for their SDK:
    123 {{{#!bash
    124 wget http://dev.gateworks.com/newport/tools-gcc-6.2.tar.bz2
    125 tar -xvf tools-gcc-6.2.tar.bz2
    126 }}}
    127  * For Ventana, you can use the pre-built Gateworks OpenWrt 14.08 BSP tailored to the ARM Cortex-A9 CPU used in the IMX6 SoC:
    128 {{{#!bash
    129 wget http://dev.gateworks.com/openwrt/14.08/imx6/OpenWrt-Toolchain-imx6-for-arm_cortex-a9+neon-gcc-4.8-linaro_uClibc-0.9.33.2_eabi.tar.bz2
    130 tar -xvf OpenWrt-Toolchain-imx6-for-arm_cortex-a9+neon-gcc-4.8-linaro_uClibc-0.9.33.2_eabi.tar.bz2
    131 }}}
    132 
    133 3. Obtain Linux kernel source (see [#linuxsource above] to help you decide which kernel version you should use):
    134  * for mainline '''Linux v4.14''' for example:
     1192. Obtain Linux kernel source (see [#linuxsource above] to help you decide which kernel version you should use):
     120 * for the latest Gateworks kernel for Venice:
     121{{{#!bash
     122git clone https://github.com/Gateworks/linux-venice.git
     123cd linux-venice
     124}}}
     125 * for mainline lastest working tree:
    135126{{{#!bash
    136127git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
    137128cd linux
    138 git checkout v4.14
    139 }}}
    140  * for the Gateworks Newport kernel based on 4.14:
    141 {{{#!bash
    142 git clone https://github.com/Gateworks/linux-newport.git
    143 cd linux-newport
    144 git checkout v4.14.4-newport
    145 }}}
    146  * for the '''Gateworks Linux 3.14.x''' based downstream vendor kernel (with full Ventana support including video capture which hasn't made it fully into mainline yet):
    147 {{{#!bash
    148 git clone https://github.com/Gateworks/linux-imx6.git
    149 cd linux-imx6
    150 git checkout gateworks_fslc_3.14_1.0.x_ga
    151 }}}
    152 
    153 4. Setup shell environment for building which depends on the toolchain you are using. This includes the following env variables used by the kernel build system:
    154  * ARCH - the architecture to build (arm for 32bit ARM such as Ventana IMX6, arm64 for 64bit ARMv8 such as Newport CN80xx)
     129}}}
     130
     1313. Install cross toolchain and setup shell environment for building. This includes the following env variables used by the kernel build system:
     132 * ARCH - the architecture to build (arm64 for 64-bit ARM architectures such as Malibu/Venice/Newport or arm  for 32-bit ARM architectures such as Ventana)
    155133 * CROSS_COMPILE - the prefix the toolchain you are using has prior to the gcc executables (everything before the 'gcc').
    156134 * PATH - make sure the directory where the toolchains compiler (${CROSS_COMPILE}gcc) executable exists. Typically you will prepend this to the default PATH
    157  * INSTALL_MOD_PATH - where to install kernel modules too if using the 'modules_install' build target (in our examples we install to ./install)
    158  * INSTALL_HDR_PATH - where to install header files if using the 'headers_install' build target (in our examples we install to ./install; note that this needs to be passed into make as opposed to an env var)
     135 * INSTALL_MOD_PATH - where to install kernel modules to by default if using the 'modules_install' build target (in our examples we install to ./install)
     136 * INSTALL_MOD_STRIP - set to 1 to strip kernel modules of debugging data (saves a tremendous amount of space)
    159137 * LOADADDR - where to relocate/uncompress the kernel to when using the uImage target
    160138 * Examples:
    161   * using the [http://dev.gateworks.com/newport/tools-gcc-6.2.tar.bz2 Newport toolchain] to build Real-time Linux stable 14.4.59 kernel:
    162 {{{#!bash
    163 export STAGING_DIR=/home/user/builds/newportToolchain
    164 TOOLCHAIN=thunderx-tools-97
    165 PATH=$PATH:$STAGING_DIR/$TOOLCHAIN/bin
     139  - Venice (IMX8M arm64) using Ubuntu's aarch64-linux-gnu-gcc cross toolchain for building code for ARM 64-bit architecture
     140{{{#!bash
     141sudo apt install gcc-aarch64-linux-gnu
    166142export ARCH=arm64
    167 export CROSS_COMPILE=aarch64-thunderx-linux-gnu-
    168 }}}
    169   - using the pre-built Gateworks OpenWrt 14.08 BSP toolchain:
    170 {{{#!bash
    171 export STAGING_DIR=../OpenWrt-Toolchain-imx6-for-arm_cortex-a9+neon-gcc-4.8-linaro_uClibc-0.9.33.2_eabi
    172 TOOLCHAIN=toolchain-arm_cortex-a9+neon_gcc-4.8-linaro_uClibc-0.9.33.2_eabi
    173 PATH=$PATH:$STAGING_DIR/$TOOLCHAIN/bin
     143export CROSS_COMPILE=aarch64-linux-gnu-
     144export INSTALL_PATH=install
     145export INSTALL_MOD_STRIP=1
     146export LOADADDR=0x40200000
     147}}}
     148  - Newport (CN80XX arm64) using Ubuntu's aarch64-linux-gnu-gcc cross toolchain for building code for ARM 64-bit architecture
     149{{{#!bash
     150sudo apt install gcc-aarch64-linux-gnu
     151export ARCH=arm64
     152export CROSS_COMPILE=aarch64-linux-gnu-
     153export INSTALL_PATH=install
     154export INSTALL_MOD_STRIP=1
     155export LOADADDR=0x20000000
     156}}}
     157  - Ventana (IMX6 arm) using Ubuntu's arm-linux-gnueabi cross toolchain for building code for ARM 32-bit architecture
     158{{{#!bash
     159sudo apt install gcc-arm-linux-gnueabi
    174160export ARCH=arm
    175 export CROSS_COMPILE=arm-openwrt-linux-
     161export CROSS_COMPILE=arm-linux-gnueabi-
     162export INSTALL_PATH=install
     163export INSTALL_MOD_STRIP=1
    176164export LOADADDR=0x10008000
    177165}}}
    178    * Note that STAGING_DIR is something required by the OpenWrt toolchain... your toolchain may differ
    179  * Some troubleshooting steps:
    180 {{{#!bash
    181 echo $ARCH # make sure this is not empty and is set to your target CPU architecture
    182 }}}
    183 {{{#!bash
    184 which ${CROSS_COMPILE}gcc #is this your compiler
    185 }}}
    186 {{{#!bash
    187 ${CROSS_COMPILE}gcc -v #is target: your desired architecture
    188 }}}
    189 {{{#!bash
    190 echo $PATH #have you appended the appropriate parameters to your path. When in doubt start new shell
    191 }}}
    192 5. Configure the kernel. Often you will want to start with an existing defconfig file followed by doing a 'make kernel menuconfig' to change any options you desire to modify.
     166
     1674. Configure the kernel. You will want to start with an existing defconfig file followed by doing a 'make kernel menuconfig' to change any options you desire to modify. When using 'menuconfig' you can search for various items using the '/' key which will show you those items as well as their dependencies (you can not enable a kernel config unless all of its dependencies are met)
    193168 * Examples:
    194   * Venice: Use the Venice BSP located [wiki:venice/bsp Venice BSP Wiki Page]
    195   * for the mainline Ventana/IMX6:
     169  - Venice (IMX8M arm64)
     170{{{#!bash
     171make distclean
     172make imx8mm_venice_defconfig # Gateworks default config
     173make menuconfig
     174}}}
     175   * Note that imx8mm_venice_defconfig exists in arch/arm64/configs/ for the Gateworks venice kernels and does contain support for imx8mm as well as imx8mn and imx8mp SoC's. This defconfig contains all kernel modules deemed useful and is rather large. For a more minimal venice kernel defconfig copy [http://dev.gateworks.com/buildroot/venice/minimal/venice_minimal_kernel_defconfig venice_minimal_kernel_defconfig] to arch/arm64/configs and use 'make venice_minimal_kernel_defconfig'
     176  - Newport (CN80XX arm64)
    196177{{{
    197178#!bash
    198 make imx_v6_v7_defconfig # Ventana/IMX6
     179make distclean
     180make newport_defconfig # Gateworks default config
    199181make menuconfig
    200182}}}
    201   * for the mainline Newport/CN80XX:
     183   * Note that newport_defconfig exists in arch/arm64/configs for the Gateworks newport kernels and contains all kernel modules deemed useful and is rather large. For a more minimal newport kernel defconfig copy [http://dev.gateworks.com/buildroot/ventana/minimal/newport_minimal_kernel_defconfig newport_minimal_kernel_defconfig] to arch/arm/configs and use 'make newport_minimal_kernel_defconfig'
     184  - Ventana (IMX6 arm)
    202185{{{
    203186#!bash
    204 make defconfig # ARM64 defconfig
    205 make menuconfig
    206 }}}
    207   - for the Gateworks Newport kernel:
    208 {{{#!bash
    209 make newport_defconfig # Gateworks default config
    210 make menuconfig
    211 }}}
    212   - for the Gateworks Linux 3.14.x based downstream vendor kernel:
    213 {{{#!bash
     187make distclean
    214188make gwventana_defconfig # Gateworks default config
    215189make menuconfig
    216190}}}
    217   * The menuconfig make target launches the ncurses based (serial console) Linux kernel menu configuration tool so that you can add or adjust the support to your needs. A common change to the default kernel configurations above would be to add  support for a USB based device, or wireless (802.11) support for example.
    218 
    219 6. Build kernel targets. What targets you build depends on the product family and the bootloader or boot method you are using. Some common examples are:
    220  * Image - uncompressed kernel (use this for Newport/CN80XX arm64 targets)
    221  * uImage - compressed kernel wrapped in a U-Boot image header (use this for Ventana/IMX6) which requires the $LOADADDR env variable to tell U-Boot where to uncompress the kernel to
     191   * Note that gwventana_defconfig exists in arch/arm/configs for the Gateworks ventana kernels and contains all kernel modules deemed useful and is rather large. For a more minimal ventana kernel defconfig copy [http://dev.gateworks.com/buildroot/newport/minimal/gwventana_minimal_kernel_defconfig gwventana_minimal_kernel_defconfig] to arch/arm64/configs and use 'make gwventana_minimal_kernel_defconfig'
     192
     1935. Build kernel targets. What targets you build depends on the product family and the bootloader or boot method you are using. Some common examples are:
     194 * Image - uncompressed kernel (use this for 64-bit targets)
     195 * uImage - compressed kernel wrapped in a U-Boot image header (use this for 32-bit targets that use 'bootm' from the bootloader). This requires the $LOADADDR env variable to tell U-Boot where to uncompress the kernel to which is SoC specific
    222196 * modules - for building loadable kernel modules (*.ko)
    223  * modules_install - for installing the loadable kernel modules (set the INSTALL_MOD_PATH parameter to the location you wish to install to)
    224  * headers_install - for installing kernel header files for development of userspace applications that use the kernel APIs (set the INSTALL_HDR_PATH parameter to the location you wish to install to)
    225197 * dtbs - for building device-tree blobs (*.dtb) from kernel source tree (Ventana/IMX6; for Newport we use out-of-kernel tree device-tree)
    226198 * Examples:
    227   - Newport/CN80XX:
    228 {{{#!bash
    229 mkdir install
    230 make INSTALL_MOD_PATH=install INSTALL_HDR_PATH=install/usr Image modules modules_install headers_install
    231 }}}
    232   - Ventana/IMX6:
    233 {{{#!bash
    234 mkdir install
    235 make INSTALL_MOD_PATH=install INSTALL_HDR_PATH=install/usr uImage dtbs modules modules_install headers_install
    236 }}}
    237 
    238 7. Copy artifacts (depends on target):
    239  * kernel images (Image/uImage) will be in arch/$ARCH/boot
    240  * dtbs will be in arch/$ARCH/boot/dts
    241  * modules will be in the directory specified by the INSTALL_MOD_PATH parameter. You can create a tarball via:
    242 {{{#!bash
    243 tar -C <INSTALL_MOD_PATH dir> --owner=0 --group=0 -cvJf modules.tar.xz .
    244 }}}
     199  - Venice (IMX8M arm64)
     200{{{#!bash
     201make -j8 Image modules dtbs
     202}}}
     203  - Newport (CN80XX arm64):
     204{{{#!bash
     205make -j8 Image modules
     206}}}
     207   * Note Newport uses the device-tree from the boot firmware not from the kernel repository so we don't bother building dtbs
     208  - Ventana (IMX6 arm):
     209{{{#!bash
     210make -j8 uImage dtbs modules modules_install headers_install
     211}}}
     212   * Note that ARM 32-bit bootloaders typically use uImage for kernel image which requires LOADADDR to be defined with a specific address for the SoC
     213
     2146. Install additional files to proper directories. Various kernel make targets exist to help you install kernel modules and header files to a directory. In addition depending on the target we may need to copy dtbs and create a bootscript and a kernel fit image (if that is what the bootscript expects).
     215 * modules_install - for installing the loadable kernel modules (set the INSTALL_MOD_PATH 'parameter' or 'evn' to the location you wish to install to)
     216 * headers_install - for installing kernel header files for development of userspace applications that use the kernel APIs (set the INSTALL_HDR_PATH 'parameter' not env to the location you wish to install to)
     217 * Examples:
     218  - Venice (IMX8M)
     219{{{#!bash
     220mkdir -p $INSTALL_PATH/boot
     221make modules_install INSTALL_MOD_PATH=$INSTALL_PATH # install modules
     222make headers_install INSTALL_HDR_PATH=$INSTALL_PATH/usr # install headers
     223cp arch/arm64/boot/Image arch/arm64/boot/dts/freescale/imx8*venice*.dtb* $INSTALL_PATH/boot # install kernel and dtbs
     224gzip -fk $INSTALL_PATH/boot/Image
     225mkimage -f auto -A $ARCH -O linux -T kernel -C gzip -n "Kernel" \
     226  -a $LOADADDR -e $LOADADDR -d $INSTALL_PATH/boot/Image.gz $INSTALL_PATH/boot/kernel.itb # FIT image
     227wget https://raw.githubusercontent.com/Gateworks/bsp-venice/master/boot.scr -O boot.scr
     228mkimage -A $ARCH -T script -C none -d boot.scr $INSTALL_PATH/boot/boot.scr # boot script
     229}}}
     230  - Newport (CN80XX)
     231{{{#!bash
     232mkdir -p $INSTALL_PATH/boot
     233make modules_install INSTALL_MOD_PATH=$INSTALL_PATH # install modules
     234make headers_install INSTALL_HDR_PATH=$INSTALL_PATH/usr # install headers
     235cp arch/arm64/boot/Image $INSTALL_PATH/boot # install kernel
     236gzip -fk $INSTALL_PATH/boot/Image
     237mkimage -f auto -A $ARCH -O linux -T kernel -C gzip -n "Kernel" \
     238 -a $LOADADDR -e $LOADADDR -d $INSTALL_PATH/boot/Image.gz $INSTALL_PATH/boot/kernel.itb # FIT image
     239mkimage -A arm64 -T script -C none -d newport/ubuntu.scr ./newport.scr
     240wget https://raw.githubusercontent.com/Gateworks/bsp-newport/sdk-10.1.1.0-newport/ubuntu.scr -O boot.scr
     241mkimage -A $ARCH -T script -C none -d boot.scr $INSTALL_PATH/boot/newport.scr # boot script
     242}}}
     243  - Ventana (IMX6)
     244{{{#!bash
     245mkdir -p $INSTALL_PATH/boot
     246make modules_install INSTALL_MOD_PATH=$INSTALL_PATH # install modules
     247make headers_install INSTALL_HDR_PATH=$INSTALL_PATH/usr # install headers
     248mkimage -A $ARCH -T script -C none -d gwventana_bootscript $INSTALL_PATH/boot/6x_bootscript-ventana # boot script
     249}}}
     250
     2517. Create tarball of build artifacts:
     252{{{#!bash
     253tar -C $INSTALL_PATH --owner=0 --group=0 -cvJf linux.tar.xz .
     254}}}
     255  - This kernel can be installed on your rootfs via {{{tar -C / -xvf linux.tar.xz --keep-directory-symlink}}}
     256
     257
     258Troubleshooting:
     259 * ensure you have the ARCH env variable set properly for your target CPU architecture:
     260{{{#!bash
     261echo $ARCH # should be arm64 for 64-bit ARM targets and arm for 32-bit ARM targets
     262}}}
     263 * ensure that ${CROSS_COMPILE}gcc exists in your $PATH and is configured for your desired architecture:
     264{{{#!bash
     265user@build:/usr/src/linux$ ${CROSS_COMPILE}gcc -v
     266Using built-in specs.
     267COLLECT_GCC=aarch64-linux-gnu-gcc
     268COLLECT_LTO_WRAPPER=/usr/lib/gcc-cross/aarch64-linux-gnu/9/lto-wrapper
     269Target: aarch64-linux-gnu
     270Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.4.0-1ubuntu1~20.04.1' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libquadmath --disable-libquadmath-support --enable-plugin --enable-default-pie --with-system-zlib --without-target-system-zlib --enable-libpth-m2 --enable-multiarch --enable-fix-cortex-a53-843419 --disable-werror --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=aarch64-linux-gnu --program-prefix=aarch64-linux-gnu- --includedir=/usr/aarch64-linux-gnu/include
     271Thread model: posix
     272gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
     273}}}
     274  - the above (from an Ubuntu 22.04 jammy host) shows the toolchain is based on GCC v9.4.0 built to execute on x86_64 architecture (--host) for a target of aarch64 (ARM 64bit) architecture (--target)
    245275
    246276
    … …  
    252282The 'make headers_install' command exports the kernel's header files in a form suitable for use by userspace programs. These headers describe the API for user space programs attempting to use kernel services and are used by the system's C library (ie glibc or uClibc) to define available system calls as well as constants and structures to be used with these system calls. The C library's header files are usually installed in /usr/include and the kernel's headers are usually in /usr/include/linux and /usr/include/asm. Kernel headers are backwards compatible but not forwards compatible meaning a program built against a C library using older kernel headers should run on a newr kernel (although possibly missing access to new features) but a program built against newer kernel headers may not work on an older kernel.
    253283
    254 The 'make headers_install' command uses two optional arguments to define the architecture (ARCH) and the location to install the files (INSTALL_HDR_PATH)
     284The 'make headers_install' command uses two optional arguments to define the architecture (ARCH which defaults to $ARCH) and the location to install the files (INSTALL_HDR_PATH)
    255285
    256286Reference:
    … …  
    407437For a more simple process for bringing in a new kernel tarball on top of Ubuntu: (change wget link accordingly....)
    408438{{{
    409 wget http://dev.gateworks.com/ventana/images/gateworks-linux-5.15.48.tar.xz
    410 tar -C / -xvf gateworks-linux-5.15.48.tar.xz --keep-directory-symlink
     439wget http://dev.gateworks.com/ventana/images/linux-ventana.tar.xz
     440tar -C / -xvf linux-ventana.tar.xz --keep-directory-symlink
    411441sync
    412442reboot