wiki:linux/encryption

Version 15 (modified by Ryan Erbstoesser, 3 years ago) ( diff )

add venice links

  1. Linux Crypto API
    1. Linux userspace crypto support
      1. AF_ALG
      2. Cryptodev
      3. Performace Comparisons
    2. OpenSSL library
      1. OpenSSL with AF_ALG
        1. Building OpenSSL prior to 1.1.0 with AF_ALG support
        2. Building OpenSSL 1.1.0 and later with AF_ALG support
        3. Enabling the AF_ALG kernel module
        4. Checking AF_ALG engine installation
        5. Evaluating AF_ALG performance
      2. OpenSSL with cryptodev
        1. Building OpenSSL versions prior to 1.1.1 with cryptodev support
        2. Building OpenSSL versions 1.1.1 and later with cryptodev support
        3. Enabling the cryptodev kernel module
        4. Checking cryptodev engine installation
        5. Evaluating cryptodev performance
    3. Gateworks BSP Support
  2. Some Security Thoughts
    1. Other Resources

Linux Crypto API

This page contains general information regarding the crypto functionality available to the Linux kernel. For more information including hardware acceleration and driver support, see the family specific wiki pages:

The Crypto API, introduced in Linux 2.5.45, is a generic cryptography library API introduced in the kernel containing support for all popular block ciphers, hash functions, AEAD, HMAC, and compression algorithms. The kernel contains software implementations for major symmetric ciphers as well as allows plugging in implementations which take advantage of hardware components such as the IMX6 CAAM and OcteonTX ZIP/CPT hardware blocks that can accelerate encryption/compression.

The Linux Kernel Crypto API backend modules transparently accelerate kernelspace crypto users such as IPsec, 802.11, 802.15.4, Bluetooth, and and dm-crypt (search the kernel for 'crypto_alloc_' to find all users).

Software vs Hardware Crypto:

  • Software:
    • Example: OpenSSL SW engine
    • pros:
      • full control over the algorithm
      • no black box
    • cons
      • consumes CPU cycles
      • often slower than dedicated software
  • Hardware:
    • Example: IMX6 CAAM, Octeon-Tx CPT/ZIP
    • pros
      • minimal CPU cycles
      • generally faster than software counterpart
    • cons
      • limited subset of algorithms
      • black box

Note that often modern processors can outperform hardware crypto offload by using software crypto (at the expense of using up CPU cycles). In these cases or the case where the benefit of hardware crypto results in the same performance the difference is the off-loading to conserve CPU cycles. Perform benchmarks before choosing a solution that's right for you.

References:

Linux userspace crypto support

Accessing the Linux kernel Crypto API from userspace in order to take advantage of hardware crypto offload can be done in one of two ways:

  • AF_ALG: the in-tree/official solution (since OpenSSL 1.1.0) and supported in older OpenSSL via an out-of-tree plugin
  • Cryptodev: out-of-tree kernel module ported from BSD with better performance and natively supported in OpenSSL

You can use either of the above kernel API's directly or you can use a crypto library that may support them such as OpenSSL or GnuTLS.

AF_ALG

AF_ALG is a netlink-based interface that adds an AF_ALG address family introduced in 2.6.38.

Kernel configuration:

  • CONFIG_CRYPTO_USER_API
  • CONFIG_CRYPTO_USER_API_HASH
  • CONFIG_CRYPTO_USER_API_SKCIPHER
  • CONFIG_CRYPTO_USER_API_RNG
  • CONFIG_CYRPTO_USER_API_AEAD

Comparison to Cryptodev (+ indicates a pro, - indicates a con):

  • supports CIPHER, HASH
  • socket based interface
  • + in kernel since 2.6.38
  • + inherently asynchronous
  • - OpenSSL 1.1.0+ supports AF_ALG natively but prior versions require out-of-tree engine plugin to be built
  • - GnuTLS does not have support for AF_ALG
  • - not many examples
  • - higher latency compared to cryptodev

Example native code: 

#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/socket.h>
#include <linux/if_alg.h>
#include <linux/socket.h>
#define SHA256_DIGEST_SZ 32

int main(void)
{
        struct sockaddr_alg sa = {
                .salg_family = AF_ALG,
                .salg_type = "hash",
                .salg_name = "sha256"
        };
        unsigned char digest[SHA256_DIGEST_SZ];
        char *input = "Hello World!";
        int i, sockfd, fd;
        sockfd = socket(AF_ALG, SOCK_SEQPACKET, 0);
        bind(sockfd, (struct sockaddr *)&sa, sizeof(sa));
        fd = accept(sockfd, NULL, 0);
        write(fd, input, strlen(input));
        read(fd, digest, SHA256_DIGEST_SZ);
        close(fd);
        close(sockfd);
        for (i = 0; i < SHA256_DIGEST_SZ; i++)
                printf("%02x", digest[i]);
        printf("\n");
        return 0;
}
  • use the splice() call to push data
  • page-align the buffers with payloads

Cryptodev

The Cryptodev-linux kernel module has to be compiled as it is not part of the kernel tree. It is API compatible with OpenBSD's Cryptographic Framework (OCF or /dev/crypto) and it is GPLv2 licensed which means one day it could be included directly in the linux kernel. It enables userspace application access to the Crytpo API backend modules already present in the kernel.

Comparison to AF_ALG (+ indicates pro, - indicates con):

  • Supports CIPHER, HASH, AEAD
  • users character device interface (/dev/crypto)
  • + compatible with OpenBSP /dev/crytpo (API compatible, not code compatible)
  • + OpenSSL has engine for cryptodev
  • + GnuTLS has support for cryptodev
  • + nice examples (in the examples directory of the linux kernel driver source)
  • + lower latency than AF_ALG
  • - out of tree kernel driver (for years now)
  • - Adds arbitrary IOCTLs
  • - Synchronous only

Even though cryptodev is out-of-tree its quite easy to compile it against your kernel:

  • In general: 
    tar xvf crptodev-linux-*.tar.gz
make
insmod cryptodev.ko
  • For examples cross compiling on host for Newport or Venice devices see the BSP Makefile

For examples using the cryptodev API via /dev/crypto see the cryptodev-linux/examples:

References:

Performace Comparisons

These openssl speed tests were done using a v5.4.0 kernel and an out of tree cryptodev module on an Ubuntu Focal rootfs. In order to reproduce these results, you must use the config script within openssl's build directory to disable dynamic engine support. Otherwise openssl will almost always default to using the faster CPU method (software). 

apt install build-essential pkg-config libssl-dev
wget https://www.openssl.org/source/openssl-1.1.1g.tar.gz
tar xvf openssl-*
cd openssl-*
# Use linux-armv4 in place of linux-aarch64 for Ventana boards
./Configure enable-engine enable-dso enable-devcryptoeng enable-afalgeng disable-dynamic-engine no-hw-padlock linux-aarch64
make
make install
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/ # prioritize built shared object path for linker this boot

# Remove cryptodev module to test afalg
rmmod cryptodev; for i in af_alg algif_hash algif_skcipher algif_rng algif_aead; do modprobe $i; done
openssl speed -evp aes-256-cbc -engine afalg -elapsed

# Remove afalg modules to test cryptodev
rmmod algif_hash algif_aead algif_skcipher algif_rng af_alg; modprobe cryptodev
openssl speed -evp aes-256-cbc -engine devcrypto -elapsed

# Remove both to ensure pure software test
rmmod cryptodev; rmmod algif_hash algif_aead algif_skcipher algif_rng af_alg
openssl speed -evp aes-256-cbc -elapsed

  • Device: GW7300-00-AA
  • CPU Speed: 1200 MHz
  • OpenSSL version: 1.1.1g (aes-256-cbc)
Number of Blocks Processed in 3s / Block Size in Bytes
Engine 16 64 256 1024 8192 16384
AF_ALG 15.47k61.66k245.18k961.12k 6416.15k 10759.56k
cryptodev 15.75k62.78k250.07k978.99k 6572.84k11110.23k
software 125752.24k334926.68k564419.24k693793.79k 743309.31k746651.65k
  • Device: GW6404-B
  • CPU Speed: 1500 MHz
  • OpenSSL version: 1.1.1g (aes-256-cbc)
Number of Blocks Processed in 3s / Block Size in Bytes
Engine 16 64 256 1024 8192 16384
AF_ALG 243.96k992.81k3705.26k12669.95k 55855.79k 75311.79k
cryptodev 361.48k1453.65k5723.56k20779.35k 74918.57k93077.50k
software 33182.40k72847.42k102824.70k115210.24k 119368.36k119581.35k

Note: Performance of each engine may vary between systems. In general, most results are similar between engines but can vary at the low and high ends for block sizes and by individual ciphers/digests types. It's recommended to evaluate each engine on your system to determine their performance and choose the best combination that fits your use case.

OpenSSL library

Instead of accessing crypto functions directly via CPU instructions or the kernel APIs, we opted to use the OpenSSL library to wrap that functionality for us. There are a few steps to enable OpenSSL for each kernel API.

https://image.slidesharecdn.com/slideshare-linuxcrypto-160411115704/95/slideshare-linux-crypto-4-638.jpg

To see what engine support exists in OpenSSL use the openssl engine command: 

root@bionic-armhf:~/openssl-1.1.0g# openssl engine
(cryptodev) BSD cryptodev engine
(dynamic) Dynamic engine loading support
  • the above shows that dynamic engine loading is supported as well as the cryptodev engine

To evaluate performance vs CPU load tradeoff use the openssl speed command:

  • Example: GW5304 (IMX6Q@1.2GHz) 
    root@bionic-armhf:~/openssl-1.1.0g# openssl speed -evp aes-128-cbc -elapsed
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-128-cbc for 3s on 16 size blocks: 27715 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 64 size blocks: 27570 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 256 size blocks: 27010 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 1024 size blocks: 23792 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 8192 size blocks: 9095 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 16384 size blocks: 4963 aes-128-cbc's in 3.00s
OpenSSL 1.1.0g  2 Nov 2017
built on: reproducible build, date unspecified
options:bn(64,32) rc4(char) des(long) aes(partial) blowfish(ptr) 
compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS -DOPENSSLDIR="\"/usr/lib/ssl\"" -DENGINESDIR="\"/usr/lib/arm-linux-gnueabihf/engines-1.1\"" 
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-cbc        147.81k      588.16k     2304.85k     8121.00k    24835.41k    27104.60k
root@bionic-armhf:~/openssl-1.1.0g# openssl speed -evp aes-128-cbc -engine cryptodev
engine "cryptodev" set.
Doing aes-128-cbc for 3s on 16 size blocks: 27750 aes-128-cbc's in 0.19s
Doing aes-128-cbc for 3s on 64 size blocks: 27562 aes-128-cbc's in 0.04s
Doing aes-128-cbc for 3s on 256 size blocks: 26970 aes-128-cbc's in 0.06s
Doing aes-128-cbc for 3s on 1024 size blocks: 23868 aes-128-cbc's in 0.08s
Doing aes-128-cbc for 3s on 8192 size blocks: 9045 aes-128-cbc's in 0.01s
Doing aes-128-cbc for 3s on 16384 size blocks: 4882 aes-128-cbc's in 0.05s
OpenSSL 1.1.0g  2 Nov 2017
built on: reproducible build, date unspecified
options:bn(64,32) rc4(char) des(long) aes(partial) blowfish(ptr) 
compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS -DOPENSSLDIR="\"/usr/lib/ssl\"" -DENGINESDIR="\"/usr/lib/arm-linux-gnueabihf/engines-1.1\"" 
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-cbc       2336.84k    44099.20k   115072.00k   305510.40k  7409664.00k  1599733.76k
  • Use -elapsed argument when evaluating software crypto to see how many operations within a wall time of 3s can be performed compared to how much CPU time and how many operations can be performed within a wall time of 3s when using a hardware crypto engine: The above test shows for 16 byte blocks of aes-128-cbs's you can get 27715 iterations within 3s with software crypto and 27750 iterations within the same 3s (wall-clock time) but only using 0.19s of CPU time (ever so slight of a performance boost but significantly less CPU use) which results in hardware crypto having a much higher bandwidth if you look at it in terms of what it can do in 3s of CPU time
  • Use -engine cryptodev to use the cryptodev engine (if available)
  • Use -engine af_alg to use the AF_ALG engine (if available)
  • Use -elapsed and no -engine param to use software crypto to show wall-clock time vs CPU time

OpenSSL with AF_ALG

CollapsibleStart(Building OpenSSL prior to 1.1.0 with AF_ALG support)

Building OpenSSL prior to 1.1.0 with AF_ALG support

OpenSSL added native AF_ALG support 1.1.0 (Aug 25 2016). If you are using a version prior to that you need to build a plugin for OpenSSL.

Note that a Linux v4.1 kernel or newer is required (AF_ALG was added in 2.6.38 but a newer feature of async support on a socket is needed which came with 'net: socket: add support for async operations' in v4.1).

Note that Ubuntu's OpenSSL does not enable AF_ALG support by default so you will still need to build/rebuild OpenSSL to enable it.

To build the out-of-tree OpenSSL af_alg plugin for OpenSSL versions prior to 1.1.0: 

# install deps
apt install build-essential pkg-config libssl-dev
# build
git clone https://github.com/sarnold/af_alg.git
cd af_alg/
./autogen.sh
./configure
make
# install plugin to engines directory for currently installed libssl
DIR=$(dpkg -L libssl1.0.0 | grep engines$)
# openssl version -d # shows OPENSSLDIR
sudo cp libaf_alg.so $DIR
sudo chmod 644 $DIR/libaf_alg.so
  • Note that this plugin is incompatible with OpenSSL 1.1.x and will fail to build for those versions due to API changes

CollapsibleEnd

Building OpenSSL 1.1.0 and later with AF_ALG support

Note that prior to OpenSSL 1.1.1 AF_ALG support is not enabled by default thus you need to rebuild it and add the 'enable-afalgeng' config option. 

apt install build-essential pkg-config libssl-dev
wget https://www.openssl.org/source/openssl-1.1.1c.tar.gz
tar xvf openssl-*
cd openssl-*
./config enable-engine enable-dso enable-afalgeng
make
make install
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/ # prioritize built shared object path for linker this boot

See ​here for information on shared object loading for permanent linker path solution

Note that for Ubuntu / Debian Linux distros it is preferred to download source package, modify debian/rules and recompile the package: 

apt install build-essential pkg-config ubuntu-dev-tools debhelper bc
apt-get build-dep openssl
apt-get source openssl
cd openssl-*/
sed -i "s/CONFARGS  =/CONFARGS  = enable-engine enable-dso enable-afalgeng/" debian/rules
dch -i "Enabled AF_ALG support"
debuild
sudo dpkg -i ../*.deb

For either installation method you can run the make _tests target of the openssl source Makefile to run a series of self tests.

Enabling the AF_ALG kernel module

To ensure you are loading the kernel modules (af_alg, algif_hash, algif_skcipher):

  • load them on boot: 
    # ensure these modules are loaded at bootup
echo af_alg >> /etc/modules
echo algif_hash >> /etc/modules
echo algif_skcipher >> /etc/modules
  • load them for current boot: 
    # load them now
modprobe af_alg algif_hash algif_skcipher

Checking AF_ALG engine installation

On OpenSSL versions prior to 1.1.0, you can check if the AF_ALG engine was successfully installed by running the following command: 

openssl engine # check if af_alg is present

On OpenSSL versions 1.1.0 and later, the AF_ALG engine is dynamically loaded which means that the engine will not appear when you run the command above.

Evaluating AF_ALG performance

For OpenSSL versions prior to 1.1.0: 

openssl speed -evp aes-128-cbc -engine af_alg -elapsed

For OpenSSL versions 1.1.0 and later: 

openssl speed -evp aes-128-cbc -engine afalg -elapsed

OpenSSL with cryptodev

CollapsibleStart(Building OpenSSL versions prior to 1.1.1 with cryptodev support)

Building OpenSSL versions prior to 1.1.1 with cryptodev support

Because Cryptodev is not available by default on Linux distributions OpenSSL has to be compiled with additional flags to include support for them: 

apt install build-essential pkg-config
wget https://www.openssl.org/source/openssl-1.1.1c.tar.gz
tar xvf openssl-*
cd openssl-*
./config -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS
make
make install
  • Additionally the sysroot should have the cryptodev header installed: /usr/include/crypto/cryptodev.h

Note that for Ubuntu / Debian Linux distros it is preferred to download source package, modify debian/rules and recompile the package: 

apt install build-essential pkg-config ubuntu-dev-tools debhelper
apt-get build-dep openssl
apt-get source openssl
cd openssl-*/
sed -i "s/CONFARGS  =/CONFARGS = -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS/" debian/rules
dch -i "Enabled cryptodev support"
DEB_BUILD_OPTIONS=nocheck debuild # disable checks to avoid issue with api check failing
sudo dpkg -i ../openssl*.deb

CollapsibleEnd

Building OpenSSL versions 1.1.1 and later with cryptodev support

Starting with version 1.1.1, the cryptodev engine is now called devcrypto and is implemented against the cryptodev-linux kernel module. To build OpenSSL with devcrypto support, the 'enable-devcryptoeng' flag is used during configuration. 

apt install build-essential pkg-config
wget https://www.openssl.org/source/openssl-1.1.1c.tar.gz
tar xvf openssl-*
cd openssl-*
./config enable-engine enable-dso enable-devcryptoeng
make
make install
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/ # prioritize built shared object path for linker this boot

See ​here for information on shared object loading for permanent linker path solution

For Ubuntu / Debian Linux distros: 

apt install build-essential pkg-config ubuntu-dev-tools debhelper bc
apt-get build-dep openssl
apt-get source openssl
cd openssl-*/
sed -i "s/CONFARGS  =/CONFARGS = enable-engine enable-dso enable-devcryptoeng/" debian/rules
dch -i "Enabled cryptodev support"
debuild # disable checks to avoid issue with api check failing
sudo dpkg -i ../*.deb

For either installation method you can run the make _tests target of the openssl source Makefile to run a series of self tests.

Enabling the cryptodev kernel module

To ensure you are loading the kernel module (cryptodev):

  • load them on boot: 
    # ensure these modules are loaded at bootup
echo cryptodev >> /etc/modules
  • load them for current boot: 
    # load them now
modprobe cryptodev

Checking cryptodev engine installation

For OpenSSL versions prior to 1.1.1: 

openssl engine # show engines available - look for cryptodev
openssl version -f # show compiler flags openssl was built with; look for -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS

For OpenSSL versions 1.1.1 and later: 

openssl engine # look for devcrypto

Evaluating cryptodev performance

OpenSSL performance can be tested with the 'openssl speed' command:

For OpenSSL versions prior to 1.1.1 

openssl speed -evp aes-128-cbc -engine cryptodev -elapsed

For OpenSSL versions 1.1.1 and later 

openssl speed -evp aes-128-cbc -engine devcrypto -elapsed

References:

Gateworks BSP Support

The embedded linux community tends to prefer cryptodev so that is what we tend to support at Gateworks in our various BSP's:

  • Newport:
    • Ubuntu - software only (openssl needs to be rebuilt for cryptodev support)
    • OpenWrt - cryptodev
  • Ventana:
    • Ubuntu - software only (openssl needs to be rebuilt for cryptodev support)
    • OpenWrt - cryptodev
    • Yocto - cryptodev

Some Security Thoughts

The below is simply some thoughts for implementing security / encryption on the Gateworks SBCs:

Linux full disk encryption can be provided by the dm-crypt kernel driver/module and file or file-system based encryption can be provided by the fscrypt or eCryptFS projects. All of these solutions require a key used to crypt/decrypt. Additionally, the kernel itself can't mount a root filesystem that uses encryption so initrd's (initial ramdisk) are used for this such that a ramdisk filesystem is built into the kernel with enough support to do the encrypted device/file/fielsystem mounting.

As far as keys are concerned you can either use symmetric keys which crypt/decrypt is the same key, or asymmetric keys where crypt/decrypt keys are different (priv/pub RSA keys for example). On desktop/smarphone the key used to encrypt the filesystem could be derived from a user password/passphrase entered interactively (which is how Android phone encryption works for example). On an embedded system without user interaction the key 'should' be stored encrypted in the filesystem or in secure storage (such as a TPM). This isn't a requirement but then what is the point of encrypting something if a hacker can exploit some security hole and end up finding the key?

Note that i.MX SoC's have a Hardware Unique Key (HUK) per chip that can be accessed only by the IMX Cryptographic and Assurance Module (CAAM). This HUK can be used to encrypt a key and store it in the filesystem during manufacturing. During boot CAAM would be used to decrypt the key and restore the plain key that would be used to decrypt the filesystem. Since the key inside CAAM is not accessible the encrypted key is always protected.

You could also use external hardware like a Secure Element or a TPM device which usually provide secure storage that could be used to store a master key that could be used to encrypt/decrypt a filesystem encryption key.

Another alternative is a Trusted Execution Environment (TEE) where the code executed and the data accessed is isolated and protected in terms of confidentiality (nobody has access to the data) and integrity (nobody can change the code and its behavior). This is discussed here: venice/tee

Note also that if you are going to use encryption you had better be using secure boot as well otherwise as mentioned a hacker could easily insert their own code to invalidate any encryption that is done. Encrypted boot on top of secure boot is also an option but if you are using a TPM device (or IMX CAAM) the keys are protected and therefore encrypted boot firmware may not really be necessary.

These topics are extremely complex and unique to each customer. The customer can implement these or there are companies that specialize in such services.

Other Resources

Note: See TracWiki for help on using the wiki.