wiki:Yocto/Building

Version 12 (modified by Tim Harvey, 6 years ago) ( diff )

updated description of images based on flash geometry

  1. Building Yocto & Installing for the Gateworks Ventana Family
  2. Pre-Compiled Yocto Binaries
  3. Obtaining and Compiling the Source Code
    1. Troubleshooting build failures
      1. Fetch failure
      2. Upstream Breakage
    2. Keeping up to date and/or pinning source with repo
    3. Customizing and Adding Packages
    4. Updating and rebuilding
    5. Disk usage and keeping things clean
    6. Building / Updating / Adding Packages
    7. Building a toolchain and/or SDK
    8. Other useful links
  4. Installing Firmware
    1. NAND FLASH
    2. Installing on Removable storage: mSATA/USB/uSD

NOTE In conjunction with the Yocto BSP, please be sure to use the latest bootloader for proper operation. You can find Bootloader Instructions here.

Building Yocto & Installing for the Gateworks Ventana Family

The following versions of Yocto are supported:

  • Yocto v2.3 (Pyro) Poky 17.0 released on 05/24/2017 (Recommended)
    • using the gateworks_fslc_3.14_1.0.x_ga kernel
    • Considered stable

CollapsibleStart(Old Releases)

  • Yocto v1.8 (Fido) Poky 13.0 released on 02/23/2016
    • using the gateworks_fslc_3.14_1.0.x_ga kernel
  • Yocto v1.7 (Dizzy) Poky 12.0 released on 02/25/2015
    • using the gateworks_3.10.17_1.0.0_ga kernel
  • Yocto v1.6 (Daisy) Poky 11.0 released on 4/25/2014
    • using the gateworks_3.10.17_1.0.0_ga kernel

CollapsibleEnd

The Gateworks Ventana Yocto BSP provides a layer of package recipes that replace or override recipes in the upstream Yocto project such as:

  • kernel
  • bootloader
  • filesystem images
  • support utilities

For details on Yocto releases please see:

Pre-Compiled Yocto Binaries

For convenience Gateworks has some pre-built ubi files available. Note that you need to download a UBI image appropriate for the FLASH size on your board. Gateworks Ventana boards come with either 256MB NAND Flash or 1GB/2GB NAND Flash. You can see the size output from the bootloader over the serial console.

Installation instructions here

Most recent releases:

  • Yocto 2.3 (Pyro) 20170524:
    • Multimedia: (supports gstreamer video in/out encode/decode, but no GUI desktop)
      • ​Yocto-2.3-ventana_normal.ubi - UBI image for 2K page size 'normal' geometry FLASH (see here to determine your flash geometry) (sha256sum:e2b41e2ab9d0c7829996249b1bf04cb902cbdc1d300a1d4411d9f96540a2f91a)
      • ​Yocto-2.3-ventana_large.ubi - UBI image for 4K page size 'large' geometry FLASH (see here to determine your flash geometry) (sha256sum:055b7644b1db05b96e33f948e1f314d39e7745fb1fd08dab248fe39f975c6eb2)
      • ​rootfs tarball (sha256sum:aa8db12b24a091db994d5cba1357e608b03be71c947a040954d70bd0ce7636d4)
    • QtGlVidDemo: Qt5 QML Application showcasing video rendering over meshes using EGLFS and GStreamer:
      • ​QtGlVidDemo_normal.ubi - UBI image for 2K page size 'normal' geometry FLASH (see here to determine your flash geometry) (sha256sum:5de3a85c77d3b07ded77087211faa74a83bc9685abfaf3693c98b9c90f774998)
      • ​QtGlVidDemo_large.ubi - UBI image for 4K page size 'large' geometry FLASH (see here to determine your flash geometry) (sha256sum:0ae1fca40cdca781e14b4a80c2be2bd5e67d80b853a5e1f2b04702882c3eae7d)
      • ​QtGlVidDemo.rootfs tarball (sha256sum:e0691776a27dce01d63d63c6f857ee7941aeba2dc1ff6051820342d8b1995f25)

CollapsibleStart(Old Releases)

  • Yocto 1.8.2 (Fido) 20160328-1730:
    • Multimedia: (supports gstreamer video in/out encode/decode, but no GUI desktop)
    • GUI: (supports gstreamer video in/out encode/decode and SATO X11 desktop with Matchbox window manager and Midori web browser)
  • Yocto 1.8.1 (Fido) 20160223-1730:
  • Yocto 1.7.1 (Dizzy) 20150904-1730:
    • Multimedia:
  • Yocto 1.6.2 (Daisy) 20150221_0538:
    • Multimedia:
      • ​Yocto-1.6.2-mm_normal.ubi - UBI image for 2K page size 'normal' geometry FLASH (see here to determine your flash geometry) (sha256sum:c05a6cca542bd2004cf51bc161d5c73b1e63322a9a543c6f566a8e876ec9274a)
      • ​ubi Yocto-1.6.2-mm_large.ubi - UBI image for 4K page size 'large' geometry FLASH (see here to determine your flash geometry) (sha256sum:10b27d97025df0c389972995497381a5e8e15beb79915f35c760aa0439370b7a)
      • ​Yocto-1.6.2-mm.rootfs tarball (sha256sum:b383e6bcc4543719423c318cae588897fde36d5f68f4b1f989054bd287f7557b)

CollapsibleEnd

Version History:

  • Yocto v2.3:
    • Yocto 2.3 (Pyro) 20170524:
    • kernel: ​3.14.48-1.0.x_ga+yocto+g3ae25e0
    • wireless drivers from linux-backports: 20160122

CollapsibleStart(Old Releases)

  • Yocto v1.8:
    • Yocto 1.8.2 (Fido) 20160328-1730:
      • kernel: ​3.14.48-1.0.x_ga+yocto+g3ae25e0
      • wireless drivers from linux-backports: 20160122
    • Yocto 1.8.1 (Fido) 20160223-1730:
      • kernel: 3.14.48-1.0.x_ga+yocto+g7a5ffca6
      • wireless drivers from linux-backports: 20150129
      • fixed: gsc_update no longer erases user EEPROM data
      • fixed: VPU firmware update to resolve various encoding timeouts
      • fixed: FEC ethernet tx queue stalls
      • fixed: NAND stability issue
      • fixed: PWM pinmux
      • fixed: mxc_v4l2_capture initialization
      • fixed: reboot fix for GW5100 A/B revisions
      • updated: bootscript updated to v1.06 to remove LVDS display detection
      • added: added user controllable quality steps to gst-variable-rtsp-server
      • added: gpsd
      • added: crda
      • added: ethtool
      • added: hostapd-conf
      • added: SPI for GW522x
      • added: USB serial drivers
      • added: 7" LVDS display touchscreen controller (gt9x)
      • added: use GSC for power-down and restart
    • Yocto 1.8 (Fido) 20150904-2139:
      • Initial Gateworks Yocto v1.8 release
      • kernel: 3.14.48-1.0.x_ga+yocto+g4ba0a59
      • wireless drivers from linux-backports: 20150129
      • much improved ath10k performance/support including adhoc
      • added gstreamer 1.0 / gstreamer-imx
      • added avc8000 8x D1 miniPCIe capture card support
      • added gsc-update
      • added uboot-envtools
      • added gwsoc (GW16113) support)
  • Yocto v1.7:
    • Yocto 1.7.1 (Dizzy) 20150904-1730:
      • kernel: 3.10.17-1.0.0_ga+yocto+g4078cea
      • wireless drivers from linux-backports: 20141221
      • default to ldo-enabled mode
      • i2c: add retries on i2c nak's
      • gsc: fix gsc hwmon negative temperature readings
      • video: mxc: add support for HDMI interlace out
      • add UHS-I support
      • enable SATA_AHCI support
      • enable PCA953x IRQ
      • update IMX6SDL voltage set-points
      • disable PCIe Gen2
      • imx-thermal: set thresholds based on CPU temperature grade
      • tda1997x: default to yuv422smp capture mode for 1080p60Hz
      • tda1997x: fixed ITU709 colorimetry colorspace conversion
      • sgtl5000: fix audio pop
    • Yocto 1.7.1 (Dizzy) 20150221-0225:
      • Initial Gateworks Yocto v1.7 release
      • kernel: 3.10.17-1.0.0_ga+yocto+g4d177f6
  • Yocto v1.6:
    • Yocto 1.6.2 - 20150221_0538:
      • added support for GW551x, GW552x
      • added cryptodev module
      • wireless drivers:
      • updated drivers to compat-wireless_20141221
      • use internal regdb
      • kernel: 3.10.17-1.0.0_ga+yocto+g4d177f6
      • added support for DLC800FIGT3 8in XGA (1024x768) capacitive multi-touch touchscreen
      • added support for DLC700JMGT4 7in WSVGA (1024x600) capacitive multi-touch touchscreens
      • bumped IMX6Q/IMX6DL operating point voltages for VDD_ARM/VDD_SOC
      • added GSC drivers (Watchdog / Input)
      • fix gpio input state detect for PCA953x port expanders
      • added support for GW551x, GW552x
      • HDMI input:
        • fixed EDID handling
        • add supoprt for HDMI input in 16bit YUV422 mode (requires alternate device-tree configuration)
      • fix audio output format details and constrain samplerate
      • disable IMX6 busfreq driver
      • add i210 support
      • add GW16103 support
      • GW51xx: fix invalid PPS gpio
      • disable evbug driver
      • add LTC3676 PMIC support and ldo-bypass mode for GW53xx/GW52xx/GW51xx/GW551x/GW552x) (lowers overall power-reduction and thermal envelope)
    • Yocto 1.6.1 - 20141024:
      • kernel: 3.10.17-1.0.0_ga+yocto+gb5914e9
      • occasional PCIe link issue resolved
      • video sink issue resolved for same resolution input/output
      • dual-display HDMI+CVBS fixed
      • GW16082 support fixed
      • PCIe IRQ slot mapping issue fixed
      • GW5400 stability issue fixed

CollapsibleEnd

For instructions on flashing UBI files, please see this section.

Obtaining and Compiling the Source Code

The following build instructions refer to Debian GNU/Linux 7.4 and Ubuntu 12.04 - 15.10.

Please make sure the following packages are installed 

sudo apt-get install chrpath libsdl-dev texinfo curl build-essential git subversion diffstat gawk

Instructions:

  1. fetch the repo tool (if you don't have it already in your path): 
    mkdir ~/bin
curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo
  1. Setup source repos: 
    PATH=${PATH}:~/bin
mkdir ~/ventana-yocto
cd ventana-yocto
repo init -u https://github.com/Gateworks/gateworks-yocto-bsp-platform -b pyro
    • in the above command, pyro is the name of the branch that refers to Yocto v2.3
  1. (optional) pin the repo versions to the latest version which was known to have been successfully built by the Gateworks nightly build server: 
    wget http://dev.gateworks.com/yocto/1.8/snapshot.xml # fetch the pinned manifest from the last successful nightly build
mv snapshot.xml .repo/manifest.xml # copy over the generic un-pinned manifest
    • the repo init command will fetch a repo 'manifest' which refers to various source repositories and branches but will fetch the latest changes from those branches (which occasionally may fail to build due to upstream errors common with community based projects). The Gateworks nightly build server posts the manifest pinned to the specific revision used in the build on completion of a successful build. You can also use a snapshot from a previously released pre-built binary by using the other snapshot files in ​​http://dev.gateworks.com/yocto/1.8
  1. Download latest sources: 
    repo sync -c -j8
    • the -c parameter tells repo to just pull down the current branch specified by the manifest file (thus is more network and disk usage efficient)
    • the -j parameter tells repo to use multiple processes (8 in this example) which will speed up fetching
    • you can later repeat the repo sync to pull down the latest sources specified for the repo manifest file for the current branch specified in the repo init in step 2 above
  1. Activate bitbake environment. You will have to accept license agreement. 
    . ./setup-environment build
    • do this every time for a new shell when you wish to use bitbake
  1. Build a filesystem image recipe. gateworks-image-multimedia is recommended. 
    bitbake gateworks-image-multimedia
    • For a detailed description of the Gateworks specific images, please see the images page.
    • This could take several hours depending on your Internet speed and development host
    • core-image-minimal results in 1.8GB of downloaded sources, 14GB of space used in build/tmp, and takes ~70mins on a Quad-core 2.5GHz build host, not accounting for your network connection.
    • By default, images, kernels, etc are found in tmp/deploy/images/ventana.
  1. Grab .ubi file which contains both kernel and root filesystem and program it to the board with instructions below: 
    tmp/deploy/images/ventana/gateworks-image-multimedia-ventana_normal.ubi
tmp/deploy/images/ventana/gateworks-image-multimedia-ventana_large.ubi

Notes:

  • to download any updates and rebuild, repeat the above starting with step 3
  • to re-activate a new shell repeat the above starting with step 4 (you can only have 1 shell activated at a time - you need to activate a new shell if you have exited a previously activated shell)
  • to clean a specific recipe use bitbake -f -c clean <recipe>. Note that to represent the kernel you can use the virtual recipe name 'virtual/kernel'
    • Note: After cleaning a recipe, rebuild with the --no-setscene command line argument to bitbake, e.g. bitbake --no-setscene <recipe>
  • to clean all built items (but not remove downloaded sources or the sstate-cache) you can rm -rf tmp.
  • to clean everything and start over you can remove the entire build directory and repeat the above starting with step 4 (this will remove all downloaded sources as well)

Useful References:

Troubleshooting build failures

The OpenEmbedded build system does a fairly good job of reporting build errors in a sane way to help you diagnose the problem.

In general, for each package that failed, you will see three lines of output from bitbake for each package. For example, the following shows a 'fetch error' for the evtest_1.25 package: 

ERROR: Function failed: Fetcher failure for URL: 'http://cgit.freedesktop.org/~whot/evtest/snapshot/evtest-1.25.tar.bz2;name=archive'. Unable to fetch URL from any source.
ERROR: Logfile of failure stored in: /usr/src/tharvey/nightly/yocto/danny/20140820_8bd8be2d/build/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/evtest-1.25-r0/temp/log.do_fetch.23170
ERROR: Task 452 (/usr/src/tharvey/nightly/yocto/danny/20140820_8bd8be2d/sources/meta-openembedded/meta-oe/recipes-support/evtest/evtest_1.25.bb, do_fetch) failed with exit code '1'

The first ERROR line here shows the cause of the issue, the second shows where the detailed log of the failure is, and the third simply says what package and task failed. If you need assistance understanding a particular error, you will want to provide the three lines above as well as the log file pointed to in the second line.

Fetch failure

The most common build failure (of any linux build system) is failure to be able to fetch sources for the 100's of OpenSource projects the build system uses from the Internet. This can be caused by a network issue on your end, a network issue on the source package sites end, or a site/file that has been permanently moved/removed.

If this occurs you will see an 'ERROR: Function failed: Fetcher failure' message such as the following output from bitbake: 

ERROR: Function failed: Fetcher failure for URL: 'http://cgit.freedesktop.org/~whot/evtest/snapshot/evtest-1.25.tar.bz2;name=archive'. Unable to fetch URL from any source.

If you encounter such an error the most obvious thing to make sure your Internet connection is solid and try first is simply bitbake your desired package again. If it occurs again, its likely an issue on the site that the package URI specifies as the package source. This could be a temporary issue, or the site/file could have been permanently moved/removed for some reason and your either building an older BSP or the package source simply hasn't been updated to deal with the change (as I mentioned, this is a common issue).

To combat this issue, there are mirrors where you may be able to find the missing file(s). Note that Gateworks provides such a mirror that it tries to keep updated at ​​http://dev.gateworks.com/sources. Once you find the file, copy it to your download directory, touch the file indicating the fetch is done (the source file with a .done on the end) and bitbake your package again. For example, the above shows a failure to fetch the evtest-1.25.tar.bz2 file: 

wget http://dev.gateworks.com/sources/evtest-1.25.tar.bz2
cp evtest-1.25.tar.bz2 downloads/
touch downloads/evtest-1.25.tar.bz2.done
bitbake gateworks-image-multimedia

Upstream Breakage

It is not uncommon for community based projects such as Yocto to encounter build failures after changes because of the complexity of the build system. In some cases breakage may occur simply because of collisions between upstream Yocto repos and the packages that Gateworks overrides.

If you are building an 'un-pinned' release (you are not using a snapshot of the manifest file which has sha revisions for each repository) and you encounter a build failure, you may want to switch to a manifest file that produced a successful build by the Gateworks nightly build server. Manifests point to source code repositories by network server and branch, but typically do not pin the branch to a specific repository commit. A 'pinned' snapshot can be used instead that does specify a specific commit of each repository and branch. Refer to step 3 in the instructions above.

Keeping up to date and/or pinning source with repo

When building projects that use multiple source repositories any repository may change and thus make your working directory out of date. Because Yocto uses several package feeds with their own repositories this can make it difficult to keep in sync - this is where the repo tool comes in handy.

The repo sync command will update all repositories with upstream changes: 

repo sync --quiet

You can override the default Manifest by using a local manifest file if you want to keep in sync with the upstream manifest yet make some minor change like pin a specific project or add a couple of projects. Reasons for doing this could be:

  • adding new projects
  • pinning certain projects
  • removing certain projects

To use a local manifest create .repo/local_manifest.xml and it will be merged with .repo/manifest.xml by the repo tool whenever the manifest is used. You can use the remove-project directive to remove a project that you don't want and even add it back with your own choices if you want something different. For example: 

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
   <remove-project path="hardware/qcom/display" name="CyanogenMod/android_hardware_qcom_display" />
   <project path="hardware/qcom/display" name="WinSuk/android_hardware_qcom_display" />
</manifest>

The repo manifest command will create a snapshot of your current project's manifest allowing you to create a pinned version that can be used later to create a working directory with the various projects at the exact same state as your current working directory: 

repo manifest -o snapshot.xml -r

This snapshot can then be copied over .repo/manifest.xml in a different build directory to pin the repository sources.

The Gateworks nightly build server creates a manifest snapshot like this and uploads the latest successful build to ​​http://dev.gateworks.com/yocto so that it can be used to re-create a successful nightly build.

References:

Customizing and Adding Packages

Please visit the following Gateworks wiki page: Yocto Packages and Customizing

Updating and rebuilding

From time to time the various repositories including in the Yocto build (a combination of Gateworks packages, community packages, Freescale community packages, and Yocto community packages) will get updates to various packages to add features or resolve issues.

The repo tool which manages the set of repositories in your yocto directory (based on a package manifest) can 'sync' all repos to the latest state using the following in the main yocto directory (the directory containing the sources and build subdirs: 

./repo sync

Following a sync, you can bitbake your filesystem image again to build a new filesystem with all updates: 

# activate a bitbake shell if not already done
. ./setup-environment build
bitbake gateworks-image-multimedia
  • build the filesystem image of your liking

Disk usage and keeping things clean

Yocto builds can chew up a lot of disk space (true really of any OS build system).

Bitbake will keep copies of all workdirs for old packages, so over time if you update recipes (or repo sync which may update recipes) your disk usage will grow. You can safely remove the old packages manually from build/tmp if you wish, or use bitbake -cclean <recipe.bb> to clean specific recipes (note that bitbake -cclean <package> will only clean the current preferred version, not old packages).

Another thing that can cause disk usage bloat is filesystem images. Each time you build a filesystem image (ie gateworks-image-*) bitbake will create a new one with a timestamp in build/tmp/deploy/images/ and the build system will keep around several artifacts such as the filesystem tarball, the ubifs, and the ubi (thus a filesystem image resulting in a 100MB UBI actually chews up over 300MB of disk space each time you build it depending on compression). Its very likely that if you are re-building an image, you probably don't care about the old ones, so you might want to get in the habit of removing them before building. For example, if you want to build gateworks-image-multimedia: 

rm -rf build/tmp/deploy/images/gateworks-image-multimedia* ;# remove old images we don't care about
bitbake gateworks-image-multimedia ;# build a new one

For Yocto, all temporary files will be in build/tmp so if you want to clear out everything and start over you can: 

rm -rf build/tmp

Building / Updating / Adding Packages

For information concerning building, updating, and adding packages please see Yocto/packages.

Building a toolchain and/or SDK

See Yocto/SDK for information about a pre-built downloadable SDK.

You can use bitbake to create a toolchain (cross-compiler) or an Software Development Kit (SDK) comprised of a cross-toolchain and libs.

Toolchain: 

bitbake meta-toolchain

SDK (contains meta-toolchain as well as -dev and -dbg packages with headers and libs): 

bitbake -cpopulate_sdk gateworks-image-multimedia
  • The produced SDK will be a self-extracting shell-script in tmp/deploy/sdk that contains all the include headers and libs for the packages in the image - you can build an SDK for any image by replacing 'gateworks-image-multimedia' with any other buildable filesystem image.

For those that may want to compile the Freescale Image:

Installing Firmware

NAND FLASH

There are 2 options:

  1. TFTP Server (recommended) - Read below
  2. JTAG Programmer (more steps and slower, requires no network)- Link here

Boards with a NAND FLASH large enough to accommodate your image (Ventana boards have a 256MB NAND flash by default which is large enough for fsl-image-multimedia) can have the UBI filesystem created by the build process placed on them. A ubi image will be built in tmp/deploy/images directory alongside the kernel and filesystem tarballs. The ubi image will end with _normal.ubi which is suitable for standard NAND flash sizes on Ventana boards. If you have a Ventana with a 1GB or larger NAND device, you need to build for the 'large' NAND flash layout which you can do by changing the UBI_CONFIG variable in sources/meta-gateworks/conf/machine/ventana.conf from 'normal' to 'large' which will result in a ubi image ending with _large.ubi.

To install firmware to a Ventana board using Serial/ENET, do the following:

First, get yourself into a sane state:

  • Copy your UBI file to a proper location that your tftpserver has access to
  • connect to the serial console of the target board (instructions may be found here if you are unfamiliar with this)
  1. Connect your target board to your network, set ipaddress and serverip in uboot 
    # Break out of the bootloader
setenv ipaddr <localip>
setenv serverip <serverip>

For Example: 

setenv ipaddr 192.168.1.211
setenv serverip 192.168.1.14
  1. Modify the boot variables to point to the image file on the tftp server and update: 
    # Note: Adjust file path/name to match file structure on your tftpserver
setenv image_rootfs gateworks-image-gui-ventana.ubi
  1. Run the nand update script to pull the image from the tftp server and load it: 
    run nand_update
    • Be sure that the tftp server is working properly and started on the host PC
    • An error such as 'ERROR: Need valid 'usbnet_devaddr' to be set at drivers/usb/gadget/ether.c:2362/usb_eth_init()' is simply because the ethernet transfer failed and it is attempting to use the USB network which needs to be configured.
  1. Once it is done loading, type the command boot 
    boot
  1. The login is root with no default password.

Installing on Removable storage: mSATA/USB/uSD

For this, do not use the .ubi file. Use the tarball, for example: gateworks-image-gui-ventana.tar.bz2

To image the built firmware onto a removable flash device such as an mSATA disk, uSD card, or USB stick you can use the following script which creates a single ext4 partition for rootfs. 

DISK=/dev/sdc
# create a single Linux partition
printf ",,L,,\n" | sudo sfdisk -uS ${DISK}
# format it as ext4
sudo mkfs.ext4 -O ^64bit ${DISK}1
# mount it
sudo mount ${DISK}1 /mnt/disk
# untar the filesystem image
sudo tar -C /mnt/disk -xvf tmp/deploy/images/ventana/gateworks-image-gui-ventana.tar.bz2
sudo umount /mnt/disk
  • Notes:
    • set DISK to the appropriate device on your Linux Host (it may very well differ from /dev/sdc). If you do not do this, you risk overwriting your OS partition. Once the flash device is plugged into the computer, you can find the device by typing mount and examining the output. For example if A flash stick with 1 partition shows as /dev/sdc1 then the block device is /dev/sdc (the 1 is the first partition)

Using the latest bootloader and bootloader env scripts simply placing this removable media in a Ventana board will boot it.

See above for pre-built rootfs tarballs of our latest releases

Note: See TracWiki for help on using the wiki.