Skip to content

Commit b21c27e

Browse files
olethanhhoh
olethanh
and
hoh
authored
Start documentation on confidential (#655)
* Documentat on confidential * Update doc/confidential.md Co-authored-by: Hugo Herter <[email protected]> * Update doc/confidential.md Co-authored-by: Hugo Herter <[email protected]> * Update doc/confidential.md Co-authored-by: Hugo Herter <[email protected]> * Update doc/confidential.md Co-authored-by: Hugo Herter <[email protected]> * Update doc/confidential.md Co-authored-by: Hugo Herter <[email protected]> * Update doc/confidential.md Co-authored-by: Hugo Herter <[email protected]> * Update doc/confidential.md Co-authored-by: Hugo Herter <[email protected]> * Update doc/confidential.md Co-authored-by: Hugo Herter <[email protected]> * Update doc/confidential.md Co-authored-by: Hugo Herter <[email protected]> * Update doc/confidential.md Co-authored-by: Hugo Herter <[email protected]> * Update doc/confidential.md Co-authored-by: Hugo Herter <[email protected]> * Update doc/confidential.md Co-authored-by: Hugo Herter <[email protected]> * Update doc/confidential.md Co-authored-by: Hugo Herter <[email protected]> * Review comments * Update confidential image creation doc --------- Co-authored-by: Hugo Herter <[email protected]>
1 parent bd3a735 commit b21c27e

File tree

4 files changed

+275
-6
lines changed

4 files changed

+275
-6
lines changed

‎doc/confidential.md

Lines changed: 266 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,266 @@
1+
# Confidential computing
2+
3+
Aleph-vm offers to launch confidential VM with AMD SEV. This is also known as TEE, Trusted Execution Environment.
4+
5+
This is only supported for instances using Qemu as their hypervisor.
6+
7+
## Life cycle
8+
First, a user creates a VM message and sends it with notify_allocate. This notifies the orchestrator about the creation of the new VM.
9+
The user fetches the platform certificate, validates its chain again AMD root certificate.
10+
The user must then upload so-called Guest Owner certificates (created with sevctl) to create an encrypted channel between the user and the Security Processor.
11+
12+
Once uploaded, the VM is started in Qemu in stopped mode: Qemu will allocate the RAM for the VM, load the firmware inside it and then let the AMD Security Processor encrypt the memory. Once this is done, the SEV endpoints allow to retrieve a measure of the memory of the VM and to decide whether to inject a user secret in the VM. Upon secret injection, the VM is launched, i.e. the VM CPU is started and goes through the boot sequence of the VM.
13+
14+
The end result is a virtual machine that is accessible through SSH and is completely encrypted in RAM, making it inaccessible from the point of view of the hypervisor.
15+
16+
```mermaid
17+
flowchart TD
18+
A[Start] -->|Allocate VM on CRN| B(CRN: Check payment, download image, volume)
19+
B --> |Download certificate from CRN| C(User: Validate Certificate again CHAIN)
20+
C --> |Create session certificates| D[Certificates file created]
21+
D --> |Send certificate to CRN to init sessions | E[CRN: Launch VM with firmware with encrypted communication channel]
22+
E --> |Fetch measurement from VM| F[User: Calculate it's own measurement and verify them again the CRN's]
23+
F --> | if ok: Send secret in encrypted channel | G[CRN: Start and unlock VM]
24+
```
25+
26+
27+
# CRN Side
28+
29+
## Hardware requirement
30+
4th Generation AMD EPYC™ Processors with SEV support.
31+
32+
This includes the [9004 Series Processors and 8004 Series Processors](https://www.amd.com/en/products/processors/server/epyc/4th-generation-9004-and-8004-series.html#tabs-4380fde236-item-2130f0d757-tab).
33+
34+
Note that the [4004 Series Processors do not provide SEV](https://www.amd.com/en/products/processors/server/epyc/infinity-guard.html) and are therefore not supported.
35+
36+
> ℹ️ The 4th Generation requirement stems from security vulnerabilities discovered in SEV on Zen3 and earlier architectures.
37+
38+
## Requirements for the CRN
39+
* Support must be [enabled in the computer BIOS](https://www.amd.com/content/dam/amd/en/documents/epyc-technical-docs/tuning-guides/58207-using-sev-with-amd-epyc-processors.pdf) (see Section 2.1).
40+
* The kernel and platform must support SEV. (e.g Ubuntu 24.04 support it by default)
41+
* [sevctl](https://github.com/virtee/sevctl) must be installed. A copy is included in the aleph-vm Debian package and installed as `/opt/sevctl`.
42+
* QEMU must be installed.
43+
44+
Check with the `sevctl ok` command that the system is supporting AMD SEV properly, at least:
45+
46+
```[ PASS ] - Secure Encrypted Virtualization (SEV)```
47+
48+
49+
50+
See AMD DOC for more info on enabling SEV for your system
51+
https://www.amd.com/fr/developer/sev.html
52+
53+
54+
## Enabling the confidential computing feature in aleph-vm
55+
56+
Enable SEV in the configuration of `aleph-vm`, by default in `/etc/aleph-vm/supervisor.env`:
57+
```
58+
ALEPH_VM_ENABLE_QEMU_SUPPORT=1
59+
ALEPH_VM_ENABLE_CONFIDENTIAL_COMPUTING=1
60+
61+
```
62+
63+
After launching the server you can check the endpoint
64+
http://localhost:4020/status/config and verify that ENABLE_CONFIDENTIAL_COMPUTING is true
65+
66+
67+
# User side
68+
The user wanting to launch the VM, referred as the Guest Owner.
69+
70+
The aleph-sdk-python and the aleph-client provide way to launch , validate and start the VM.
71+
72+
## Create an encrypted VM image
73+
74+
The user must create a virtual machine disk image that has been encrypted using a password of their choice.
75+
Follow the instruction here: https://github.com/aleph-im/aleph-vm/blob/dev-confidential/examples/example_confidential_image/README.md
76+
77+
## OVMF Launcher Firmware
78+
The OMVF file, a UEFI firmware for virtual machines, handle launching the confidential VM.
79+
It receives the secret (decryption key) in a secure manner and pass it to the VM bootloader (see Boot process section).
80+
81+
Aleph.im provide a default one, destined to work with confidential image created following the procedure described above.
82+
83+
84+
In the usual case a user would just create an encrypted VM image but they might also provide a customised firmware in the `firmare` field of `trusted_execution`.
85+
86+
See [the instructions on how the Firmware is built](runtimes/ovmf/README.md)
87+
88+
The hash from the Firmware is needed to validate if it's the one launched the CRN.
89+
90+
91+
# Implementation details
92+
## Aleph-message
93+
on Instance type message, we check if the `content.environment.trusted_execution` is set
94+
95+
```
96+
"trusted_execution": {
97+
"policy": 1,
98+
"firmware": "e258d248fda94c63753607f7c4494ee0fcbe92f1a76bfdac795c9d84101eb317"
99+
}
100+
```
101+
102+
* Firmware is an [IPFS CID](https://docs.ipfs.tech/concepts/content-addressing/) reference to the OVMF firmware file (see OVMF firmware section)
103+
* policy is an AMD SEV Policy (for now we only expose if AMD SEV and SEV-ES are supported)
104+
105+
106+
## Boot process
107+
The following diagram describes the different pieces of the VM boot process.
108+
109+
![Boot process](./images/boot_process.drawio.png)
110+
111+
* OVMF: UEFI firmware (see section above), finds the bootloader and launches it
112+
* GRUB, the boot loader, decrypts the VM image and jumps to it.
113+
* GRUB configuration files: the unencrypted script looks for the user disk decryption password injected during
114+
the SEV boot process, then jumps to a complete Grub configuration file provided by the user inside the VM
115+
image.
116+
* Kernel + initrd + root filesystem: The OS of the VM.
117+
118+
OVMF and Grub must be unencrypted. This means that the VM supervisor can alter these pieces at will.
119+
It is therefore crucial that these pieces are part of the launch measurement retrieved during the SEV
120+
sequence.
121+
122+
The process documented in `runtimes/ovmf/README.md` can be used to generate a firmware image that combines OVMF and Grub
123+
into one binary.
124+
125+
126+
## Detailed sequence with endpoints
127+
```mermaid
128+
sequenceDiagram
129+
participant Qemu
130+
participant CRN
131+
actor User
132+
CRN->>User: Fetch platform certificate (GET /about/certificates/)
133+
Note right of User: Generate via sevctl using the platfom certificate:<br> TIK, TEK, GODH, Session
134+
User->>CRN:Upload certificates POST /control/machine/{ref}/confidential/inialize
135+
Note over CRN,User:session.b64, godh.b64
136+
CRN->>Qemu: Run qemu process (pass session, godh, image, ovmf)
137+
Note left of Qemu: Emulator is in stopped state
138+
User->>CRN: Fetch measurement (GET /control/machine/{ref}/confidential/measurement)
139+
Qemu->>CRN: Retrieve launch measurement (via qmp)
140+
CRN->>User: Measurements (SEV version, policy, firmware hash, signature)
141+
Note right of User: Verify measuremetn signature
142+
Note right of User: Encrypt secret using TEK key
143+
User->>CRN: Pass encoded secrets (POST /control/machine/{ref}/confidential/inject_secret)
144+
CRN->>Qemu: Inject secret (via qmp)
145+
CRN->>Qemu: Start VM (via qmp)
146+
Note left of Qemu: Emulator is in started state, VM Boot
147+
User->>Qemu: SSH or other interaction
148+
```
149+
150+
# Development and debugging
151+
152+
See QEMU.md in general for QEMU related developement
153+
154+
## Note on systemd in dev
155+
If you use a local copy of aleph-vm, for example a version you are developping on, by default systemd will still use the system version of the aleph controller. It is necessary to modify
156+
`/etc/systemd/system/[email protected]` to point to your version.
157+
158+
For example here is what I use
159+
```
160+
[Unit]
161+
Description=Aleph VM %i Controller Olivier
162+
After=network.target
163+
164+
[Service]
165+
Type=simple
166+
RestartSec=5s
167+
PrivateTmp=yes
168+
NoNewPrivileges=true
169+
WorkingDirectory=/home/olivier/pycharm-aleph-vm/src
170+
Environment=PYTHONPATH=/home/olivier/pycharm-aleph-vm/src:$PYTHONPATH
171+
ExecStart=/home/olivier/.virtualenvs/aleph-vm/bin/python3 -m aleph.vm.controllers --config=/var/lib/aleph/vm/%i-controller.json
172+
Restart=no
173+
174+
[Install]
175+
WantedBy=multi-user.target
176+
```
177+
178+
After modification use the following command to have the modification taken into account
179+
```shell
180+
sudo systemctl daemon-reload
181+
```
182+
183+
# Testing
184+
185+
After initializing the VM you can check it's status with:
186+
`sudo systemctl status aleph-vm-controller@decadecadecadecadecadecadecadecadecadecadecadecadecadecadecadeca.service`
187+
188+
and see the logs with
189+
` sudo journalctl -u aleph-vm-controller@decadecadecadecadecadecadecadecadecadecadecadecadecadecadecadeca.service`
190+
191+
**Important**
192+
193+
If you modify your base image between tests, you will need to delete the image file on disk (which is a delta of the base image)
194+
For example using :
195+
`sudo rm /var/lib/aleph/vm/volumes/persistent/decadecadecadecadecadecadecadecadecadecadecadecadecadecadecadeca/rootfs.qcow2`
196+
197+
Ensure the VM controller is stopped before!
198+
`sudo systemctl stop aleph-vm-controller@decadecadecadecadecadecadecadecadecadecadecadecadecadecadecadeca.service`
199+
200+
Between your test you can also stop the execution using
201+
```http
202+
### Stop all VMs
203+
POST http://localhost:4020/control/allocations
204+
Content-Type: application/json
205+
X-Auth-Signature: test
206+
Accept: application/json
207+
208+
209+
{
210+
"persistent_vms": [],
211+
"instances": [
212+
]
213+
}
214+
215+
```
216+
217+
## Sevctl
218+
Most operations done by `sevctl` are implemented in [aleph-sdk-python](https://github.com/aleph-im/aleph-sdk-python), either by calling it, calling the relevant endpoint
219+
or by reimplementing the functionality in python. Here is a primer in case you need to call it manually.
220+
221+
### Install `sevctl`
222+
If you are not taking the version from the debian package, you can install sevctl manually with cargo
223+
224+
Requirements:
225+
* `cargo`
226+
227+
On Ubuntu/ Debian install it via `apt install cargo` (as root)
228+
229+
To build and install sevctl
230+
```cargo install sevctl```
231+
232+
Ensure $HOME/.cargo/bin is in your PATH to launch it manually.
233+
234+
To configure which bin aleph-vm use, set the environment variable
235+
```
236+
ALEPH_VM_SEV_CTL_PATH=/home/olivier/.cargo/bin/sevctl
237+
```
238+
239+
Alternatively, `sevctl` can be build from `git` : ```cargo install --git https://github.com/virtee/sevctl```
240+
241+
242+
## Example Commands
243+
## Generate session key
244+
You can generate the sessions keys using sevctl
245+
1. Export the platform key
246+
`sudo sevctl export platform.pem`
247+
2. Create the sessions files
248+
`sevctl session platform.pem 0x1 dwdw`
249+
250+
This will create the files `vm_godh.b64`, `vm_session.b64`, `vm_tek.bin`, `vm_tik.bin` in your current directory
251+
252+
### Calculate measurement
253+
254+
```shell
255+
RUST_LOG=trace sevctl measurement build
256+
--api-major 01 --api-minor 55 --build-id 24 --policy 1
257+
--tik ~/pycharm-aleph-sdk-python/decadecadecadecadecadecadecadecadecadecadecadecadecadecadecadeca_tik.bin
258+
--firmware /usr/share/ovmf/OVMF.fd
259+
--nonce URQNqJAqh/2ep4drjx/XvA
260+
```
261+
262+
### Debug
263+
To enable debugging log, set the environment variable
264+
```env
265+
RUST_LOG=trace
266+
```

‎doc/images/boot_process.drawio.png

21.3 KB
Loading

‎examples/example_confidential_image/README.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,11 +53,14 @@ sudo rm -r /mnt/debian
5353

5454
Run the build_debian_image.sh that will create the image with the encrypted disk
5555
> This script will require sudo for certain commands
56+
57+
The password option is the *secret* password key, with which the disk will be encrypted, you will need to pass it to launch the VM.
58+
5659
```shell
57-
bash ./build_debian_image.sh -o ~/destination-image.img --password your-password -r $ROOT_DIR
60+
bash ./build_debian_image.sh --rootfs-dir $ROOT_DIR -o ~/destination-image.img --password your-password
5861
```
5962

60-
> If you need debuging you can pass the -x option to bash before the script name
63+
> Tip: To debug the image creation, pass the `-x` option to bash in front of the script name
6164
6265
## To test and further customise you image you can also boot it inside qemu
6366
```shell

‎examples/example_confidential_image/setup_debian_rootfs.sh

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -106,9 +106,9 @@ update-initramfs -u
106106
# Generate system SSH keys
107107
ssh-keygen -A
108108

109-
# Example to add a sudo user
110-
useradd -m -s /bin/bash username
111-
echo 'username:password' | chpasswd
112-
usermod -aG sudo username
109+
### Example to add a user with sudo right
110+
#useradd -m -s /bin/bash username
111+
#echo 'username:password' | chpasswd
112+
#usermod -aG sudo username
113113

114114
umount /tmp

0 commit comments

Comments
 (0)