Newport CN80XX/CN81XX Boot Firmware

The 'Boot Firmware' for Newport is defined as the combination of the First level 'boot stub' and the additional firmware stages through the bootloader. This can be broken down into the following stages:

• Boot ROM (internal on CN80XX/CN81XX SoC): fetch first level boot stub (192KB limit) from boot device (MMC or SPI FLASH)
• SPL (Secondary Program Loader)
• Bootloader (U-Boot)

For a Secondary Program Loader, or SPL, Gateworks currently uses the Cavium Board Development Kit (BDK) provided by their OCTEON-TX Software Development Kit (SDK). In this current implementation the Boot ROM loads and executes the BDK, the BDK loads and executes the ARM Trusted Firmware (ATF) and the ATF loads and executes the U-Boot bootloader in multiple stages as such:

• First level: Bootstub (firmware/bdk)
• Second level: ATF (firmware/atf)
• Third level: U-Boot (bootloader/u-boot)

Gateworks provides a pre-built Boot Firmware (firmware-newport.img) ready to flash onto boot devices as well as source for building and/or modifying the boot firmware yourself.

Cavium CN80XX/CN81XX BOOT ROM

The BOOT ROM is firmware baked into the CN80XX/CN81XX SoC and is in charge of loading code from the 'boot device' into an L2 cache scratchpad, verifying signatures (if using trusted boot) and executing it. The BOOT ROM will only access the first 4MB (0x00000-0x7ffff) of the boot device.

Cavium Board Development Kit (BDK)

The Cavium Board Development Kit (BDK) is an open source codebase that supports the OCTEON TX family of processors and provides:

• Boot Stub (boot.bin) - the code fetched and executed by the internal BOOT ROM (limited to 192KB). This is run out of an L2 cache scratch area and is responsible for loading the next phase of code which is init.bin (normal boot) or setup.bin (manufacturing mode).
• init.bin - the code fetched and executed by boot.bin responsible for loading the board device tree file (dtb), configuring the SDRAM controller and loading the next stage which is the Arm Trusted Firmware (ATF)
• setup.bin - an application that can be used for board configuration and setup (we use this for 'Manufacturing Mode' to allow the GSC EEPROM to be programmed)
• diagnostics.bin - an application that can be used for very low level board diagnostics typically used during board validation

The BDK uses an embedded FAT12 filesystem for its various components (various .bin and .dtb files). We also typically place a Flattened Image Table (FIT) image containing the kernel kernel.itb as well as a legacy U-Boot uImage bootscript newport.scr in this filesystem as well where the Newport Bootloader scripts will find and execute them. The reason for putting the kernel and bootscript here is that the Bootloader can access FAT filesystems but can not access some of the newer filesystems that you may be using as a root filesystem such as F2FS or BTRFS.

The FAT12 filesystem is created and managed using a host tool built by the BDK called fatfs-tool.

The bdk-create-fatfs-image.py python script creates the bdk.bin including the embedded FATFS and its contents. The size of the FATFS is controlled by the FATFS_SIZE variable within the script and the offset of it controlled by FIXED_SIZE.

The bdk repository provides the bdk.bin build artifact which contains the 512byte MBR with the partition table, the trusted and non-trusted boot stubs (boot.bin), and the FAT12 filesystem. It represents 0x000000 to 0x400000 of the boot firmware (first 4MB).

The bdk.bin is created via bin/bdk-create-fatfs-image python script (utils/scripts/bdk-create-fatfs-image.py) called from the 'all' target in bdk/boards/Makefile. The script does the following:

• create/update the non-trusted boot stub (192K at 0x20000) and headers (at 0x10000/0x10100)
• create/update the encrypted trusted boot stub (192K at 0x50000) and headers (at 0x10200/0x10300)
• create/update the non-trusted SCP BL1 (at 0x80000) and headers (at 0x10400/0x10500)
• create/update the trusted SCP_BL1 (at 0xc0000) and headers (at 0x10600/0x10700)
• create/update the non-trusted MCP BL1 (at 0x100000) and headers (at 0x10800/0x10900)
• create/update the trusted MCP BL1 (at 0x140000) and headers (at 0x10a00/0x10b00)
• create the FATFS filesystem (at 0x180000 2.5MB in size) and populate it with a set of files passed as additional arguments to the script

Note that the SCP and MCP components are only requires for the CN9XXX and therefore Gateworks uses a modified versoin of the bdk-create-fatfs-image.py script that skips them allowing us to reclaim that space and grow the FATFS filesystem to approximately 13MB.

ARM Trusted Firmware (ATF)

The atf repository provides the ARM Trusted firmware and the tools to create a 'Firmware Image Package (FIP)'. The ​Firmware Image Package (FIP) packages the bootloader image into a single archive that can be loaded by the ARM Trusted Firmware from non-volatile platform storage.

The FIP package containing the bootloader fip.bin is created using the tool found in tools/fiptool.

Much of the terminology used here comes from the ARM Trusted Firmware Documentation:

• ​ARM Trusted Firmware Image Terminology
• ​ARM Trusted Firmware Documentation

The ATF Boot process is described in detail in the ​ATF firmware design documentation.

Some additional terminology:

• AP - Application Processor
• BL1 - first stage bootloader
• BL2 - second stage bootloader

The ATF BL1 has the following responsibilities:

• Enable the Trusted Watchdog
• Initialize the console (displays 'Booting Trusted Firmware')
• Configure the Interconnect to enable hardware coherency
• Enable the MMU and map the memory it needs to access
• Configure any required platform storage to load the next bootloader image (BL2)

The ATF BL2 has the following responsibilities:

• Initialize the console.
• Configure any required platform storage to allow loading further bootloader images.
• Enable the MMU and map the memory it needs to access.
• Perform platform security setup to allow access to controlled components.
• Reserve some memory for passing information to the next bootloader image EL3 Runtime Software and populate it.
• Define the extents of memory available for loading each subsequent bootloader image.

Firmare Image

The firmware image contains all of the components of the 'Boot Firmware':

• MBR partition table
• trusted boot boot-stub (boot.bin)
• non-trusted boot boot-stub (boot.bin)
• embedded FATFS containing init.bin/setup.bin/*.dtb and possibly kernel
• ATF
• U-Boot (in an ATF FIP image)

The Cavium BSP uses a make-bootfs.py python script that combines the bdk.bin with the ATF components including the U-Boot FIP image.

Gateworks uses a customized version of this script that moves some of the components around a bit to create a large embedded FAT12 filesystem so that we have room for a kernel image and bootloader script there.

Newport Boot Firmware Image Map:

start-end	len	item	notes
0x0000000 - 0x0010000	64KB	MBR Partition Table	see newport/ptgen, first 512B only used
0x0010000 - 0x0020000	64KB	CVM_CLIB flash headers	built by newport/bdk-create-fatfs-image.py and required by BOOT ROM
0x0020000 - 0x0050000	192KB	boot stub (non-trusted) ap_bl1	bdk boot.bin loaded to L2 scratch and executed by BOOT ROM
0x0050000 - 0x0080000	192KB	boot stub (trusted) ap_bl1	bdk boot.bin loaded to L2 scratch and executed by BOOT ROM
0x0080000 - 0x0100000	1024KB	unused
0x0100000 - 0x0E00000	13312KB	FAT12 filesystem	contains various portions of the BDK as well as the device-tree, kernel.itb and bootscript
0x0E00000 - 0x0F00000	1024KB	ATF_BL1	atf/bl1.bin
0x0F00000 - 0x0FF0000	960KB	ATF_BL2 fip.img	U-Boot
0x0FF0000 - 0x1000000	64KB	U-Boot env	redudant 32KB env

• We remove the 960KB SCP_BL1/MCP_BL1 required by the CN9XXX
• We removed the 1M apparently unused ATF Table Header (ATF_TBL)
• We shifted the images around to maximize the size of the FAT12 filesystem while still leaving 960KB for the U-Boot fip.img

The firmware image is created by the newport/make-bootfs.py python script which creates the various headers around the objects required by the BOOT ROM and stitches them together (boot stub, ATF, ATF FIP containing the bootloader). This is called from the 'firmware' target in the Gateworks Newport BSP Makefile.

The Makeifle builds this with the newport/make-bootfs.py script:

make-bootfs.py --bdk-image bdk/target-bin/bdk.bin --atf-bl1 atf/build/t81/release/bl1.bin --atf-fip fip.img --bootfs firmware-newport.img

• --bdk-image - the bdk/target-bin/bdk.bin is created by the bdk project, specifically by the bdk-create-fatfs-image.py python script. The size of the FATFS is controlled by the FATFS_SIZE variable within that script
• --atf-bl1 - the atf/build/t81/release/bl1.bin is created by the atf project
• --atf-fip - the fip.img is created by the atf project (atf/build/t81/release/fip.bin) and the fiptool app is used to update it with the U-Boot image
• --bootfs - the output file to create

While you can modify these scripts to adjust your firmware mapping note the following:

• the BDK init.bin (apps/init/app.c) has a #define for the location of the ATF (ATF_ADDRESS)
• the ATF defines the the location of the FIP image (plat/cavium/common/thunder_io_storage.c fip_block_spec.offset)
• the U-Boot Bootloader config defines the location of the U-Boot env (CONFIG_ENV_OFFSET/CONFIG_ENV_SIZE/CONFIG_ENV_OFFSET_REDUND)