wiki:OpenWrt/building

Version 38 (modified by Ryan Erbstoesser, 9 months ago) ( diff )

update default example to venice

  1. Building OpenWrt for Gateworks Products
    1. OpenWrt BSP Branches
    2. OpenWrt BSP Pre-Built Binaries
    3. Building Pre-Requisites
    4. Downloaded Source Files
    5. Step by Step Instructions
    6. Building JTAG Binary File
    7. Building MMC Disk Image
    8. Customizing
    9. Updating to Latest Gateworks Source
    10. Troubleshooting Build Failures
    11. More OpenWrt Information

Building OpenWrt for Gateworks Products

Gateworks uses the OpenWrt Linux Distribution for its Board Support Package (BSP). From time to time, Gateworks may have patches against upstream OpenWrt to support some features which have not made it upstream or perhaps are not suitable for upstream OpenWrt. Additionally Gateworks provides a Makefile wrapper around OpenWrt to aid in pinning versions of Opensource tree's to versions that Gateworks has tested with. Please do not allow this to dissuade you from building mainline OpenWrt, particularly if being current is of importance. There are only a small number of features you will find in Gateworks-OpenWrt that do not exist in mainline, generally related to the GSC.

When using upstream OpenWrt be aware it will be necessary for your boot firmware to be updated to the latest version Gateworks offers.

OpenWrt BSP Branches

The Gateworks OpenWrt repository consist of a snapshot of upstream OpenWrt at specific points, with a few additional patches added on top that have not made it upstream yet. Additionally a gateworks directory consisting of various Gateworks tested openwrt config files and flash image creation scripts. The repository can be found on the ​Gateworks ​GitHub openwrt page.

Our branch naming policy is as follows:

  • Each new bump of 'master' will cause a new branch to be created: bump date_openwrt revision. This is the bleeding edge OpenWrt software from Gateworks and thus includes Gateworks patches on top.
  • When we decide to cut a stable release, the branch name will be: last two char of year.numerical month.version
  • master - This branch is not recommended to be used. It only serves as a reference point for the latest dated branch and is missing any patches that Gateworks has that have not been submitted or accepted upstream as well as the Gateworks support scripts and default config files.

Our ​​http://github.com/Gateworks/openwrt repository branches currently include:

  • Gateworks 23.03 OpenWrt Release
    • Is the latest OpenWrt software from Gateworks (recommended)
    • Latest build: ​​http://dev.gateworks.com/openwrt/23.03/
      • Please view the .manifest file per target to inspect installed packages + their respective versions
    • Based on OpenWrt master branch
      • Linux 5.15.98
      • Wireless drivers from linux backports-6.1-rc8
      • busybox-1.36.0
      • gcc-12.2.0
      • binutils-2.37
    • General:
      • Added Venice support
      • Updated kernel and drivers
    • Venice:
      • squashfs+f2fs overlay for sysupgrade/factory-reset support
    • Newport:
      • squashfs+f2fs overlay for sysupgrade/factory-reset support
    • Ventana:
      • squashfs+ubifs and squashfs+f2fs overlay for sysupgrade/factory-reset support
    • revision log: ​​https://github.com/Gateworks/openwrt/commits/23.03
  • Gateworks 20.06 OpenWrt Release
    • Latest build: ​​http://dev.gateworks.com/openwrt/20.06/
      • Please view the .manifest file per target to inspect installed packages + their respective versions
    • Based on OpenWrt fd0cc72d9 (2020-06-09)
      • Linux 5.4.45
      • Wireless drivers from linux backports-5.7-rc3-1
      • busybox-1.31.1
      • gcc-8.4.0
      • binutils-2.31.1
      • Package feeds pinned to 2020-06-15
    • General:
      • Added Gateworks System Controller drivers
      • dropped Laguna support
      • dropped GW16083 support
    • Newport:
      • squashfs+f2fs overlay for sysupgrade/factory-reset support
    • Ventana:
      • squashfs+ubifs and squashfs+f2fs overlay for sysupgrade/factory-reset support
    • revision log: ​​https://github.com/Gateworks/openwrt/commits/20.06

OpenWrt BSP Pre-Built Binaries

To get an already built image to flash onto the SBC, please visit the following:

Building Pre-Requisites

These instructions require having a desktop or laptop computer with Ubuntu installed to compile Gateworks OpenWrt.

Gateworks has tested with desktop systems with LTS Ubuntu Releases.

On an Ubuntu desktop development system for example the following should install all necessary pre-requisites: 

sudo apt-get install git-core subversion build-essential  ;# std dev tools
sudo apt-get install python3 python3-distutils-extra zlib1g-dev \
 libncurses5-dev libssl-dev libsnmp-dev libpam0g-dev liblzma-dev \
 gawk flex unzip gettext quilt wget rsync curl cpio xz-utils;#  other libs/tools needed by OpenWrt Buildroot
sudo apt-get install bsdmainutils # other tools needed by Gateworks patches to OpenWrt
  • NOTE: If you are experiencing difficulties, Here is a list of the pre-requisites from the general OpenWrt Site. OpenWrt buildroot prerequisites listed ​here

Ubuntu 14.04/16.04 Notes:

  • Note: There is a known bug with Ubuntu 14.04/16.04 when building Python 2.7. that may occur. Because the system Python is used during this build, you may encounter the following error: 
    ImportError: No module named _sysconfigdata_nd
configure: error: python2.7 interpreter not found
make[3]: *** [~/openwrt-next/build_dir/target-arm_cortex-a9+neon_musl-1.1.11_eabi/Python-2.7.11/.configured_yy
yyyyyyy] Error 1
  • In order to fix it, please execute the following: 
    sudo ln -s /usr/lib/python2.7/plat-*/_sysconfigdata_nd.py /usr/lib/python2.7/

Please ensure your development system has all of the listed packages installed before proceeding and note that your development box will need access to the internet in order to download the files for the development tree.

WARNING! Building with GCC 4.7.x is not fully supported. Please use 4.8.2+ or 4.6.x and below. We have also found 6.2+ to have some issues.

Downloaded Source Files

OpenWrt uses a Linux build system called buildroot. Like all Linux build systems it consists of recipes that go through the process of fetching, patching, compiling, packaging various applications. The 'fetching' process involves downloading source tarballs from the Internet (which is why an Internet connection is required). OpenWrt will fetch all sources into the 'dl' directory in the OpenWrt tree (ie 14.08/dl from the Gateworks directory). To avoid downloading sources each time you do a clean build it is recommended that you create a static location on your development host to store downloaded packages and symlink this to the dl directory. The Gateworks Makefile which fetches OpenWrt and patches it, will create this link for you if the /usr/src/dl directory exists on your development host, so if you create that and make it writable by your development user all downloads will go there automatically and be able to be used for subsequent builds. This not only helps reduce build time but it can also help if a package's source isn't available because of a site or mirror being down.

Step by Step Instructions

Video: ​https://youtu.be/cnObX6tErhw

  1. Getting Source Code:
    • There are multiple code branches (see above for recommended branch per product), please checkout the one you prefer with the following commands:
      • Gateworks 23.03 branch 
        git clone https://github.com/Gateworks/openwrt/ gateworks-openwrt -b 23.03
cd gateworks-openwrt
  1. Building:
    • 23.03:
      • To build OpenWrt for a specific target device you perform the following steps: 
        # optional; create a smylink to dl dir for shared downloads
ln -s /usr/src/dl .
# update package feeds
./scripts/feeds update -a
./scripts/feeds install -a
# (optional) start with an example config
cp gateworks/configs/venice/diffconfig .config
make defconfig
# select target and/or customize config
make menuconfig
# build
make -j8 V=s
      • For Venice you can use gateworks/configs/venice/diffconfig as a starting point
      • For Newport you can use gateworks/configs/newport/diffconfig as a starting point
      • For Ventana you can use gateworks/configs/ventana/diffconfig as a starting point
    • 20.06:
      • Our OpenWrt Git repository has a 'gateworks' directory added consisting of a Makefile providing targets that aim to simplify the entire process of:
        • using a (per target) Gateworks provided .config file
        • producing JTAG binary images compatible with the Gateworks JTAG dongle
      • Once the branch has been checked out from the ​Gateworks GitHub account, you are ready to build. The Gateworks Makefile provides the following targets:
        • all : Build both Newport (octeontx) and Ventana (imx6)
        • octeontx : build BSP for the octeontx target (Newport product family)
        • imx6 : build BSP for the imx6 target (Ventana product family)
        • images : build binary flash images (suitable for JTAG flashing) for supported product families
        • dirclean : remove all downloaded and built directories
        • Examples:
          • In order to build the octeontx target for the Newport product family, the following steps would occur: 
            # Build the octeontx target, using the configuration from gateworks/configs/octeontx/.config.
# This will take a while
make -C gateworks/ octeontx
          • In order to build the imx6 target for the Ventana product family, the following steps would occur: 
            # Build the imx6 target, using the configuration from gateworks/configs/imx6/.config.
# This will take a while
make -C gateworks/ imx6
        • The build process will take some time depending on your Internet connection and host system. On an Intel Quad Core @ 3.10GHz the build takes a couple of hours to build all product families and takes appx 15GB of disk space.

  1. Artifacts
    • The resulting kernel/rootfs files will be in the bin/<target> directory.
      • For Example, a Ventana flash image is located at: bin/targets/imx6/generic/gateworks-imx6-ventana-squashfs-nand.ubi
      • Which ubi size to use, normal or large, has to do with the flash geometry size (see here to determine your flash geometry)
  1. Installing:
    • To install the OpenWrt image onto the Gateworks SBC, please follow the instructions:

Building JTAG Binary File

For the Gateworks 20.06 BSP JTAG binaries are built for Ventana NAND Flash:

  • gateworks/images/ventana_large.bin
  • gateworks/images/ventana_normal.bin

Building MMC Disk Image

If you wish to create an MMC 'disk image' for Ventana boards that boot from eMMC or microSD: 

# create a filesystem image
OUTFS=owrt.ext4
SIZEFSMB=256M
MNT=$(mktemp -d -p/tmp)
truncate -s $SIZEFSMB $OUTFS
sudo mkfs.ext4 -q -F -O ^64bit -L rootfs $OUTFS
sudo mount $OUTFS $MNT
sudo tar -C $MNT -xf bin/targets/imx6/generic/gateworks-imx6-ventana-squashfs-bootfs.tar.gz
sudo tar -C $MNT -xf bin/targets/imx6/generic/gateworks-imx6-rootfs.tar.gz
sudo umount $MNT

# create a disk image
SIZEMB=7248M
OUTIMG=ventana.img
truncate -s $SIZEMB $OUTIMG
# partition disk - single ext partition
printf "2048,,L,,\n" | sudo sfdisk -uS $OUTIMG
sudo dd if=ventana/images/SPL of=$OUTIMG bs=512 seek=2
sudo dd if=ventana/images/u-boot.img of=$OUTIMG bs=512 seek=138
sudo dd if=$OUTFS of=$OUTIMG bs=1M seek=1
# create a compressed disk image that can be installed via U-Boot's gzwrite
gzip -f -k $OUTIMG && ls -lh $OUTIMG*

# create a jtag binary image
mkimage_jtag --emmc -e --partconf=user $OUTIMG@user:erase_all:0x0- > $OUTIMG.bin
  • SIZEFSMB above is the size of the filesystem to create in MB - this needs to be large enough to accommodate the files you are putting there... a good rule of thumb is to keep it as small as possible and use resize2fs on it to grow it to the size of the partition you place it in after first boot
  • SIZEMB above is the size of the disk image to create in MB which needs to be large enough to fit the filesystem + the 1MB offset of the partition table yet small enough to fit the actual device you will copy this to in the future (such as microSD or eMMC). Note that 7248M is the appropriate size for the 8GiB eMMC's that Gateworks uses
  • ventana.img is a non-compressed disk image containing SPL, U-Boot and the ext4 partition which can be copied directly to a microSD or eMMC via:
    • U-Boot with a 'mmc write'
    • Linux with a 'dd if=ventana.img.gz of=$blockdev'
  • ventana.img.gz is a compressed disk image which can be flashed onto an MMC device via:
    • U-Boot for example with a 'gzwrite mmc $dev $loadaddr $filesize'
    • Linux for example with a 'zcat ventana.img.gz | sudo dd of=$blockdev'
  • ventana.bin is a JTAG'able binary image containing SPL, U-Boot and the ext4 partition

To flash the file to the board, please see this link: JTAG Programming Instructions

Customizing

Video: ​https://youtu.be/HHhT6z0J5TU

Once a target has been built using the above instructions, a user may customize their image.

OpenWrt can be customized by running the make menuconfig command. In this new window, you can select/deselect packages you want installed. If kernel customization is required, a make kernel_menuconfig will open the make menu for the kernel.

Once complete, a simple make -j8 V=s will re-run the make command to re-create the rootfs.

Please read more at this page here.

Updating to Latest Gateworks Source

The steps to update your current project to the latest Gateworks patches varies depending on the branch you're trying to update. Please see the below two sections that help to detail this out. However, please note that using Git is the most developer friendly.

  • To update to the latest patches on the same branch: 
    # Make sure you have latest ref's
git remote update

# This will update the current working branch to the latest upstream,
# then attempt to rebase any local patches on top.
git pull --rebase
  • To change to a new branch, using 16.02 as an example branch: 
    # Make sure you have latest ref's
git remote update

# Checkout upstream branch 16.02 and create local branch named 16.02
git checkout remotes/origin/16.02 -b 16.02

After this is complete, please continue to build as normal. For more detail on working with git, please refer to ​​this site. For information on git pull, please refer to ​​this page. For information about git rebase, please refer to ​this page.

Troubleshooting Build Failures

One of the most commonly occurring errors that we have run into are source URL's disappearing which causes the source files to not be there during build-time. To help alleviate this, Gateworks has put up source tarballs at ​our development site. Our build environment takes advantage of this by uploading files for feeds that we require and automatically selecting them if all other options fail. In general, it will 1: look for the tarball locally; 2: try the original source URL; and 3: try ​our ​dev site. Below are other issues that may crop up:

  1. Finding an Error - Often the error will be 'reported' at the end of the build. However, the error will have actually occurred earlier on. Scroll up in the logs and look for an error. Typically it is from a package or dependency that was not able to be downloaded.
  2. Missing Dependencies - The build will pull files from mirrors around the web including a Gateworks mirror at ​​http://dev.gateworks.com/sources. If for some reason you can't download dependencies then you likely have a network issue. Note that files can be placed manually in the dl directory to avoid downloading.
  3. Errors - By default, the build is ran on many threads with the parameter -j8. To achieve better debugging information after a failure, re-run the build with the following -j1 flag from the trunk directory: 
    make -j1 V=s

More OpenWrt Information

For more information about OpenWrt see:

Note: See TracWiki for help on using the wiki.