Changes between Initial Version and Version 1 of Android/Updates


Ignore:
Timestamp:
10/22/2017 05:28:45 AM (7 years ago)
Author:
trac
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • Android/Updates

    v1 v1  
     1{{{#!html
     2          <div id="wikipage" class="trac-content"><p>
     3</p><div class="wiki-toc">
     4<ol>
     5  <li>
     6    <ol>
     7      <li>
     8        <a href="#AndroidUpdatesandRecovery">Android Updates and Recovery</a>
     9        <ol>
     10          <li>
     11            <a href="#AndroidRecovery">Android Recovery</a>
     12          </li>
     13          <li>
     14            <a href="#CreatinganOTAimage">Creating an OTA image</a>
     15          </li>
     16          <li>
     17            <a href="#CustomizingOTA">Customizing OTA</a>
     18            <ol>
     19              <li>
     20                <a href="#Androidreleasetools.py">Android releasetools.py</a>
     21              </li>
     22              <li>
     23                <a href="#CustomRecoveryUI">Custom Recovery UI</a>
     24              </li>
     25            </ol>
     26          </li>
     27          <li>
     28            <a href="#ApplyinganOTA">Applying an OTA</a>
     29            <ol>
     30              <li>
     31                <a href="#GettingoutofRecoverymode">Getting out of Recovery mode</a>
     32              </li>
     33              <li>
     34                <a href="#OTAmanager-FSLOta">OTA manager - FSLOta</a>
     35              </li>
     36            </ol>
     37          </li>
     38        </ol>
     39      </li>
     40    </ol>
     41  </li>
     42</ol>
     43</div><p>
     44</p>
     45<h2 id="AndroidUpdatesandRecovery">Android Updates and Recovery</h2>
     46<p>
     47Android has a built-in recovery and update system comprising of the following:
     48</p>
     49<ul><li>recovery boot mode (Android can instruct the board to reboot into recovery mode)
     50</li><li>recovery partition consisting of its own kernel, device-tree, and ramdisk consisting of filesystem tools, and the recovery console application
     51</li><li>extensible recovery console (RC) which is the application that is run when booting into recovery mode
     52</li><li><a class="ext-link" href="http://androidxref.com/5.1.1_r6/xref/frameworks/base/core/java/android/os/RecoverySystem.java"><span class="icon">​</span>android.os.RecoverySystem</a> class providing OTA verification and power management calls that you can use to implement your own OTA application
     53</li><li>android build system 'make dist' target to build an OTA image with extensible features
     54</li></ul><p>
     55In general, Android can boot into 'normal' mode, or 'recovery' mode. Normal mode is used to boot Android for normal usage. Recovery mode is used to boot Android into either an automated update mode, or a minimal menu driven UI for basic recovery commands.
     56</p>
     57<p>
     58An 'Over The Air' (OTA) update is commonly used to distribute updates and typically they are in the form of a zip file containing a system image and an update script. Once the OTA image is downloaded to the cache partition, the system is rebooted in recovery mode which automatically detects the zip, applies it, then reboots to standard mode.
     59</p>
     60<p>
     61The OTA updates have an embedded digital signature (verified by android.os.RecoverySystem.verifyPackage()).
     62</p>
     63<p>
     64Note that the Android recovery mechanism is not without its flaws:
     65</p>
     66<ul><li>does not allow disk re-partitioning which causes wasted space (must leave slack space in /system,  boot, and recovery for future android updates, /cache needs to be large enough to hold a compressed sw update which typically is two-thirds of /system size to be safe)
     67</li><li>device is unusable during update
     68</li></ul><p>
     69The Gateworks Android BSP provides the following OS OTA support:
     70</p>
     71<ul><li>single or two-step OTA generation and usage
     72</li><li>single OTA supporting both NAND/ubifs and blockdev/ext4 support
     73</li></ul><h3 id="AndroidRecovery">Android Recovery</h3>
     74<p>
     75The 'reboot to recovery' mode is a bit device-specific with regards to how the device is rebooted and how it determines if it should boot into recovery mode or normal mode (specifically, the communication between the OS and the bootloader). The policy used needs to be supported in both the kernel (or Android bionic libc) as well the bootloader (or bootscript). Freescale's Android BSP uses a register in the IMX6 SoC which retains its value on a chip-level reset and checks for this in their bootloader. For the Gateworks Ventana Android BSP we changed this to use an EEPROM register in the GSC so that a full board power-cycle can still detect if the board needs to boot to recovery. Technically the bootloader could use the same mechanisms that the OS and recovery application use but that typically puts too many requirements on the capaiblities of the bootloader.
     76</p>
     77<p>
     78Android recovery is similar to the BOOT image in that it has a kernel, device-tree blobs, and ramdisk with Android init (using init.recovery.*.rc) and an fstab (recovery_fstab). Recovery mode boots data from a separate 'recovery' partition instead of the typical 'boot' partition.
     79</p>
     80<p>
     81The recovery filesystem is built in $(OUTDIR)/recovery/root which is built into a ramdisk ($(OUTDIR)/ramdisk-recovery.img) and wrapped with a U-Boot header ($(OUTDIR)/uramdisk-recovery.img). In addition, an 'Android image' is built in $(OUTDIR)/recovery.img however we don't use Android images for Ventana because they do not allow for multiple kernel device-tree's.
     82</p>
     83<p>
     84The Android project that builds the recovery 'application', which typically consists of a simple menu-driven interface and 'edify' script processor is in the bootable/recovery directory.
     85</p>
     86<p>
     87The <a class="ext-link" href="http://androidxref.com/5.1.1_r6/xref/frameworks/base/core/java/android/os/RecoverySystem.java"><span class="icon">​</span>RecoverySystem class</a> provides the functionality of kicking off a recovery. The job of obtaining a file is outside of the scope of the Android Open Source project (AOSP), however Freescale has a simple app that can be used or modified to your needs. The RecoverySystem class will erase the log file (/cache/recovery/log), and write to the command file. For example when installPackage(Context context, File packageFile) is called, a command of '--update_package=&lt;filename&gt;' will be stored in /cache/recovery/command then perform a reboot via <a class="missing wiki">PowerManager?</a>.reboot().
     88</p>
     89<h3 id="CreatinganOTAimage">Creating an OTA image</h3>
     90<p>
     91The Android build system <tt>make dist</tt> target will create OTA images. These images will be in the form of .zip files in the $(PRODUCT_OUT) dir and are also copied to the $(DIST_DIR) directory which can be specified in the env (make dist DIST_DIR=dist_output). OTA package names will be names by the target device (ie ventana) and the BUILD_ID (from device/gateworks/ventana/build_id.mk)
     92</p>
     93<p>
     94The 'make dist' target will first create a 'target_files' .zip archive in the DIST_DIR directory (&lt;target-device&gt;-target_files-&lt;BUILD_ID&gt;.zip) which contains all files needed for a complete Android install (all partitions: system, boot, and recovery). You will want to keep this file for any releases you push as it may be useful later to create very small 'incremental' OTA releases from. The <tt>make dist</tt> target creates the following archives in DIR_DIR:
     95</p>
     96<ul><li>&lt;name&gt;-apps-&lt;BUILD_ID&gt;.zip - A zip of Android apps
     97</li><li>&lt;name&gt;-img-&lt;BUILD_ID&gt;.zip - A zip of the various .img files (These are Android Images, which we don't use)
     98</li><li>&lt;name&gt;-ota-&lt;BUILD_ID&gt;.zip - An OTA update package
     99</li><li>&lt;name&gt;-symbols-&lt;BUILD_ID&gt;.zip - symbols directory
     100</li><li>&lt;name&gt;-target_files-&lt;BUILD_ID&gt;.zip - a zip of the directories that map to the target filesystem which can be used to create an OTA package or filesystem image as a post-build step
     101</li></ul><p>
     102The <tt>ota_from_target_files</tt> tool uses this archive to create either 'full' or 'incremental' OTA update packages. The resulting OTA package is named &lt;target-device&gt;-ota-&lt;BUILD_ID&gt;.zip and will be in the PRODUCT_OUT directory (out/target/product/ventana/) as well as the DIST_DIR (which defaults to out/dist).
     103</p>
     104<p>
     105The OTA images are signed with a key defined by PRODUCT_DEFAULT_DEV_CERTIFICATE (which defaults to device/fsl/common/security/testkey). You can read more about Freescale's 'test key' from device/fsl/common/security/README. Once development is complete and you need to distribute a 'user' build OTA image, you need to generate a user image using your own 'private keys' as detailed in <a class="ext-link" href="https://source.android.com/devices/tech/ota/sign_builds.html"><span class="icon">​</span>Signing builds for release</a>. Note that the key used in OTA's must match the key used for the original system image.
     106</p>
     107<p>
     108An 'incremental' update contains a set of binary patches to be applied to the data already on the device which can result in considerably smaller update packages. These images can only be installed on a device that has a build where you still have the 'target_files' .zip archive to reference. If you have a previous 'target_files' .zip archive. Gateworks does not currently support the creation and installation of incremental updates.
     109</p>
     110<p>
     111A 'two-step' OTA will update the recovery partition first, then update the system partition. This takes place over multiple boot cycles and is necessary when the recovery requires a kernel or tool update in order to perform its duty. By default the <tt>make dist</tt> target will create an single-step OTA that only updates the system partition. In order to create a 'two-step' you must run it manually and provide the <tt>--two-step</tt> argument.
     112</p>
     113<p>
     114There are additional options that can be passed to the <em>ota_from_target_files</em> tool:
     115</p>
     116<ul><li><tt>-w</tt>,<tt>--wipe_user_data</tt> - wipe the user data partition when installed
     117</li><li><tt>-n</tt>,<tt>--no_prereq</tt> - omit the timestamp prereq check normally included at the top of the build scripts allowing an older OTA to install over a newer build ID
     118</li><li><tt>-e</tt>,<tt>--extra_script &lt;file&gt;</tt> - insert contents of file at the end of the update script
     119</li><li><tt>-2</tt>,<tt>--two_step</tt> - Generate a 'two-step' OTA package where recovery is updated first, so that any changes made to the system partition are done using the new recovery (updated kernel, etc)
     120</li></ul><p>
     121References:
     122</p>
     123<ul><li><a class="ext-link" href="https://source.android.com/devices/tech/ota/index.html"><span class="icon">​</span>OTA Updates</a>
     124</li></ul><h3 id="CustomizingOTA">Customizing OTA</h3>
     125<p>
     126An OTA image can be customized in the following ways:
     127</p>
     128<ul><li>add additional data files to the OTA image by adding them to the target-files .zip (add to BUILT_TARGET_FILES_PACKAGE dependency in build/core/Makefile)
     129</li><li>adding executables to recovery for use in scripts (add them to PRODUCT_PACKAGES)
     130</li><li>copy and install files to the target device during the OTA (modify the releasetools.py script)
     131</li><li>modify existing script in build/tools/releasetools/ota_from_target_files (WriteFullOTAPackage function)
     132</li><li>modify the recovery user interface (UI) (provide a custom implementation via TARGET_RECOVERY_UI_LIB)
     133</li></ul><p>
     134See <a class="ext-link" href="https://source.android.com/devices/tech/ota/device_code.html"><span class="icon">​</span>https://source.android.com/devices/tech/ota/device_code.html</a> for more details on customization of OTA's.
     135</p>
     136<h4 id="Androidreleasetools.py">Android releasetools.py</h4>
     137<p>
     138During OTA creation a target can provide a python script that extends the OTA release in a couple of ways:
     139</p>
     140<ul><li>define TARGET_RELEASETOOLS_EXTENSIONS directory that contains releasetools.py
     141</li><li>provide a releasetools.py in the $(TARGET_DEVICE_DIR)/../common directory
     142</li></ul><p>
     143Typically a custom releasetools.py is used to do very target specific things such as:
     144</p>
     145<ul><li>update the bootloader or SPL
     146</li><li>update device firmware
     147</li></ul><p>
     148If present, the following functions can be defined that will be called allowing you to provide simple 'additions' to the existing OTA script:
     149</p>
     150<ul><li>FullOTA_Assertions - called after emiiting the block of assertions at the top of a full OTA package
     151</li><li>FullOTA_InstallBegin - called at the start of full OTA installation
     152</li><li>FullOTA_InstallEnd - called at the end of full OTA installation (typically used to install the image for the device's baseband processor)
     153</li><li>IncrementalOTA_Assertions - called after emiiting the block of assertions at the top of an incremental OTA package
     154</li><li>IncrementalOTA_InstallBegin - called at the start of an incremental OTA installation
     155</li><li>IncrementalOTA_InstallEnd - called at the end of an incremental OTA installation (typically used to install the image for the device's baseband processor)
     156</li></ul><p>
     157The Gateworks Android BSP uses this to extend the the AOSP releasetools to add updating of the bootloader and SPL.
     158</p>
     159<p>
     160Sometimes the releasetools.py extensions do not offer enough customization. Gateworks has modified the ota_from_target_files script for the following:
     161</p>
     162<ul><li>we do not use recovery.img and boot.img Android Images - instead we use filesystems so that we can have a more flexible choice in device-tree's when booting the kernel
     163</li></ul><p>
     164For an example see device/gateworks/common/releasetools.py
     165</p>
     166<h4 id="CustomRecoveryUI">Custom Recovery UI</h4>
     167<p>
     168A customized version of the recovery user interface can be provided by setting TARGET_RECOVERY_UI_LIB. The default implementation is to use <a class="ext-link" href="http://androidxref.com/5.1.1_r6/xref/bootable/recovery/default_device.cpp"><span class="icon">​</span>bootable/recovery/default_device.cpp</a>
     169</p>
     170<h3 id="ApplyinganOTA">Applying an OTA</h3>
     171<p>
     172Once an OTA image has been created you can deploy it to your devices. The Life of an OTA update is well described in on the Android <a class="ext-link" href="https://source.android.com/devices/tech/ota/index.html#android-device-layout"><span class="icon">​</span>OTA Updates</a> page.
     173</p>
     174<p>
     175Typically a custom Android application will poll an update server for OTA updates and when one is found will download its description and present the user with a dialog asking if they wish to download and install the update. Once the update is downloaded and verified the recovery process can begin. This is done by the application copying the file to the /cache/recovery directory, creating a /cache/recovery/command file with recovery commands, and rebooting the board into recovery mode.
     176</p>
     177<p>
     178The 'reboot to recovery' mode is a bit device-specific with regards to how the device is rebooted and how it determines if it should boot into recovery mode or normal mode. The policy used needs to be supported in both the kernel (or Android bionic libc) as well the bootloader (or bootscript). Freescale's Android BSP uses a register in the IMX6 SoC which retains its value on a chip-level reset and checks for this in their bootloader. For the Gateworks Ventana Android BSP we changed this to use an EEPROM register in the GSC so that a full board power-cycle can still detect if the board needs to boot to recovery.
     179</p>
     180<p>
     181You can test an OTA manually without the need for an OTA manager by copying the OTA to the cache partition, creating a command file, and rebooting to recovery mode. For example the following via a shell prompt:
     182</p>
     183<pre class="wiki">busybox wget -P /cache http://server/ota.zip
     184echo "--update_package=/cache/recovery/ota.zip" &gt; /cache/command
     185reboot recovery
     186</pre><h4 id="GettingoutofRecoverymode">Getting out of Recovery mode</h4>
     187<p>
     188If you are stuck continually booting into recovery mode you can clear the bootmode flag with the following command in the bootloader:
     189</p>
     190<pre class="wiki">i2c dev 0 &amp;&amp; i2c mw 51 80 0xff 1
     191</pre><h4 id="OTAmanager-FSLOta">OTA manager - FSLOta</h4>
     192<p>
     193There is no Android Open Source Project (AOSP) project that provides an OTA manager, but Freescale has a simple one called FSLOta that can be used, or used for reference: <a class="ext-link" href="https://github.com/Gateworks/fsl_imx_demo/tree/imx_kk4.4.3_2.0.0-ga/FSLOta"><span class="icon">​</span>FSLOta source</a>
     194</p>
     195<p>
     196Android has some built-in classes that aid in the recovery process:
     197</p>
     198<ul><li><a class="ext-link" href="http://androidxref.com/5.1.1_r6/xref/frameworks/base/core/java/android/os/RecoverySystem.java"><span class="icon">​</span>android.os.RecoverySystem</a> - provides a verifyPackage function, an installPackage function, and a handleAftermath function which is called by the BootReciever following a recovery and deletes the contents of /cache/recovery
     199</li><li><a class="ext-link" href="http://androidxref.com/5.1.1_r6/xref/frameworks/base/core/java/android/os/PowerManager.java"><span class="icon">​</span>android.os.PowerManager</a> - provides a reboot function that can reboot into recovery mode (Note that RecoverySystem.installPackage() calls this)
     200</li></ul
     201}}}