Features

Secure Boot

The firmware component of TRS unconditionally enables UEFI secure boot for all supported platforms. There are some hardware requirements that will dictate how Secure Boot is configured and enabled on your hardware.

[UEFI] (§ 32.3.6 Platform Firmware Key Storage Requirements) defines that the Platform and Key exchange keys must be stored in a non-volatile storage which is tamper protected.

On Arm servers this is usually tackled by having a dedicated flash which is only accessible by the secure world.

Hardware which was designed with security in mind has the following options.

Hardware	UEFI Secure Boot	Measured Boot
RPMB 1	x	x
Discrete TPM		x
Flash in secure world	x

The reality on embedded boards is different though. In the embedded case, we don’t have a dedicated flash. What’s becoming more common though is eMMC devices with an RPMB partition.

If the board has a RPMB and OP-TEE support, Trusted Substrate will use that device to store all the EFI variables.

$skinparam sequenceMessageAlign center participant RPMB #ff5e13 participant Supplicant #0093a6 participant "U-Boot" as Uboot #0093a6 participant "TF-A(OPTEED)" as TFA #CE5756 participant "OP-TEE" as OPTEE #6b8724 participant "OP-TEE PTA" as PTA #6b8724 participant StMM as Stmm #7773cf Uboot --> TFA: TEEC_InvokeCommand TFA --> OPTEE: OP-TEE message PTA --> Stmm: MM buffer note over OPTEE,PTA #cccc00: PTA decapsulates the MM message\nfrom the OP-TEE message and\npasses it on to StMM Stmm --> OPTEE: FFA calls to OP-TEE Storage API for RPMB OPTEE --> Supplicant: RPC calls note over OPTEE,Uboot #cccc00: encrypt the data and send REE file operations to the supplicant through a series of RPC calls ...some time later... Supplicant --> RPMB: Read/Write to RPMB Supplicant --> Stmm: Read/Write finished Stmm --> PTA: MM buffer note over OPTEE,PTA #cccc00: PTA encapsulates the MM message,\ncreates an OP-TEE message and\npasses it on to OP-TEE OPTEE --> Uboot: OP-TEE response$

However for boards that don’t have an RPMB the UEFI public keys (PK, KEK, DB etc) are built-in into the firmware binary. Bundling those keys comes with it’s own set of limitations. The most notable ones being that in order to update any security related EFI variable, you need to update the bootloader and you can only boot signed binaries by default. Other, non security critical, EFI variables are stored in a file located in the ESP.

1: Requires OP-TEE support and a way to program the RPMB with a unique per hardware key (e.g a fuse accessible only from the secure world). Setting EFI variables at runtime (from the OS) not supported

Hardware and UEFI variable limitations

The firmware automatically enables and disables UEFI Secure Boot based on the existence of the Platform Key (PK). As a consequence boards that embed the keys in the firmware binary will only be allowed to boot signed binaries and you won’t be able to change the UEFI keys. See Building with your own certificates

On the other hand boards that store the variables in the RPMB come with an empty PK and the user must provision one during the setup process in order to enable Secure Boot.

$usecase "Board has OP-TEE?" as optee usecase "Board has RPMB?" as rpmb usecase "EFI variables in ESP.\nPK, KEK, db and dbx\nbuilt-in into the\nfirmware binary" as esp usecase "EFI variables in RPMB" as efirpmb usecase "Provision PK?" as provpk usecase "Secure Boot enabled" as sben usecase "Secure Boot disabled" as sbend rpmb -d-> optee: yes rpmb -d-> esp: no optee -d-> efirpmb: yes optee -r-> esp: no efirpmb -d-> provpk provpk -> sben: yes esp -> sben provpk -l-> sbend: no$

Measured Boot

TRS is designed to take advantage of Trusted Platform modules. The firmware part of TRS supports the EFI TCG Protocol as well as TCG PC Client Specific Platform Firmware Profile Specification and provides the building blocks the OS needs for Measured Boot.

In TRS the software components that extend measurements are

TF-A (QEMU only), creates an EventLog, which U-Boot will later replay on the TPM.
U-Boot will measure all components described by the afforementioned TCG specs.
The Linux kernel EFI-stub will measure the loaded initramfs and the EFI LoadOptions.

Note

PCR7 contains the UEFI Secure Boot keys and state. PCR9 will differ depending on your kernel version. Prior to 5.18 PCR9 will be empty. Past 5.18 and prior to 6.1 PCR9 will contains the initrd measurement. Post 6.1 it will contain the initrd and EFI LoadOptions measurements.

Trusted Platform module

TPMs are microcontrollers designed for cryptographic tasks. They contain a set of Platform Configuration Registers (PCRs) which are used to measure the system configuration and software.

PCRs start zeroed out and can only reset with a system reboot. Those can be extended by writing a SHA hash (typically SHA-1/256/384/512 for TPMv2) into the PCR. To store a new value in a PCR, the existing value is extended with a new value as follows: PCR[N] = HASHalg( PCR[N] || ArgumentOfExtend )

Trusted Substrate is designed to work with either discrete TPMs or provide an [fTPM] running in OP-TEE.

usecase "Board has discrete TPM" as dtpm
usecase "Board has RPMB & OP-TEE" as optee
usecase "fTPM Trusted Applicaiton" as ftpm
usecase "Measured Boot enabled" as mben
usecase "Measured Boot disabled" as mbend

dtpm -d-> mben: yes
dtpm -r-> optee: no
optee -d-> ftpm: yes
ftpm -d-> mben: yes
ftpm -d-> mbend: no

LUKS2 disk encryption

If a TPM is present on the device TRS will automatically detect it. If secure boot is enabled, then TRS will generate a random password on first boot, seal it against PCRs 7 and encrypt the root filesystem using aes-xts-plain.

TRS is designed to work regardless of the TPM implementation. We support devices with a discrete TPM, an [fTPM] or for QEMU a [SWTPM]

$skinparam sequenceMessageAlign center participant "(f)TPM" as TPM #6b8724 participant RPMB #ff5e13 participant Supplicant #ff5e13 participant "U-Boot" as Uboot #0093a6 participant "Linux Kernel" as kernel #7773cf participant "initramfs" as initramfs #CE5756 participant "rootfs" as rootfs #CE5756 Uboot --> initramfs: Measure firmware OS-loader\n and OS to PCRs0-9 ||| note over RPMB, Supplicant #cccc00: If an fTPM is used OP-TEE \n and the tee-supplicant are\n required to store data to\n the RPMB ||| initramfs --> TPM: If secure boot enabled, then on first boot \nSeal the encryption password\n against PCRs 7\n\n On subsequent boots\n Unseal the password ||| initramfs -> rootfs: If secure boot enabled, then on first boot\n Create a LUKS volume \nEncrypt and mount roots\n\n On subsequent boots\n decrypt and mount rootfs ||| initramfs --> reboot: reboot if decryption\n failed |||$

Note

You can find a full list of the components and recipes needed by running make find name=ledge-initramfs

LUKS2 Encryption

LUKS2 Decryption

OP-TEE OS

OP-TEE is our Secure World OS of choice in TRS. We use it for a number of reasons with the most notable ones being

Run [fTPM] is the hardware doesn’t have a discrete TPM.
Store EFI variables on boards that have an RPMB.
Provide a DRBG if the hardware doesn’t provide a TRNG.
Provide a PKCS#11 provider to PARSEC.

Conceptually the components interacting with OP-TEE in the TRS build can be seen in the image below. The Features lane there indicates which exceptions levels are involved in a certain use case. For example, “TEE: Secure Storage” is all kept in (S)EL-0 and (S)EL-1.

Note that this image is rather generic as depicted here. We have other areas that could (and should) be added as well, for example SCMI, Xen, FF-A, SwTPM to name a few. But perhaps it’s better to add them as separate diagrams to avoid making the images too complex.

Xen

When Xen is enabled, GRUB menu provides an entry TRS Xen (if supported) for booting Xen hypervisor.

Xen hypvervisor’s EFI program and configuration file (xen.cfg) both are placed in the root folder of boot parition. The configuration file contains the info for Xen’s log debugging level, Linux kernel image path and Linux kernel command line, etc; Xen hypvervisor parses the configuration file and boot Linux kernel image.

Note, Xen hypervisor doesn’t load initial ramdisk, this is different from the booting flow in bare metal mode which loads both initial ramdisk and Linux kernel image.

# SPDX-License-Identifier: MIT

[global]
default=xen

[xen]
options=noreboot dom0_mem=4096M bootscrub=0 iommu=on loglvl=error guest_loglvl=error
kernel=Image console=hvc0 earlycon=xenboot rootwait root=PARTUUID=f3374295-b635-44af-90b6-3f65ded2e2e4

After the system booting up, we can use the command xl list to list Xen domains, the Xen Dom0 with naming Domain-0 is created by default.

root@trs-qemuarm64:~# xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  4096    32     r-----      63.2

At this time the goal is to use the same rootfs when booting Dom0 and DomU. The root file system in Xen Dom0 doesn’t contain anything for Xen DomU, otherwise, we will run into the nested issue for building TRS image. For this reason, we need to take several steps to deploy virtual machine with Xen DomU, below gives instructions for how to do it.

Firstly, you need to create a virtual machine configuration file ewaol-guest-vm1.cfg:

# Copyright (c) 2022, Arm Limited.
#
# SPDX-License-Identifier: MIT

name = "ewaol-guest-vm1"
memory = 6144
vcpus = 4
extra = " earlyprintk=xenboot console=hvc0 rw"
root = "/dev/xvda2"
kernel = "/boot/Image"
disk = ['format=qcow2, vdev=xvda, access=rw, backendtype=qdisk, target=/usr/share/guest-vms1/trs-vm-image.rootfs.wic.qcow2']
vif = ['script=vif-bridge,bridge=xenbr0']

The configuration file ewaol-guest-vm1.cfg can be saved into the folder /etc/xen/auto/ so the virtual machine can be automaticially launched in later’s booting.

Secondly, we need to copy TRS root file system image to target. In below example, we firstly create a folder /usr/share/guest-vms1/ on the target:

root@trs-qemuarm64:~# mkdir -p /usr/share/guest-vms1/

Then we copy TRS’s qcow2 image from the host to the target, please replace <IP_ADDRESS> with your target’s IP address.

$ cd trs-workspace/build/tmp_trs-qemuarm64/deploy/images/trs-qemuarm64
$ scp trs-image-trs-qemuarm64.wic.qcow2 root@<IP_ADDRESS>:/usr/share/guest-vms1/trs-vm-image.rootfs.wic.qcow2

No need to copy kernel image, the virtual machine can reuse the same kernel image with the Xen Dom0 which has been already placed in /boot/Image.

With above preparations, it’s ready for luanching the virtual machine in Xen domU. We can create a virtual machine with command:

root@trs-qemuarm64:~# xl create /etc/xen/auto/ewaol-guest-vm1.cfg

After created the virtual machine, we can list all Xen domains:

root@trs-qemuarm64:~# xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  4096    32     r-----      63.2
ewaol-guest-vm1                              1  6143     4     r-----       4.5

We can see a new domain ewaol-guest-vm1 running in Xen DomU (ID is 1 with 4 virtual CPUs).

For accessing a Xen DomU’s console, you could use the command xl console followed by a domain name, below is an example:

root@trs-qemuarm64:~# xl console ewaol-guest-vm1

Afterwards, you could input ctrl-[ to exit from Xen DomU’s console and return back to Xen Dom0.

Known issue1: Currently Xen hypervisor is only supported for ADLink AVA platform.

Known issue2: Xen hypvervisor loads kernel image but it doesn’t load initial ramdisk.

Known issue3: TPM is not supported by Xen Dom0. If the system runs into the normal booting flow with GRUB menu entry TRS, the root file system image will be encrypted with TPM; afterwards when we switch back to Xen, it cannot reuse the root file system image due to Xen not supporting TPM at the current stage.