docs-legacy/src/cloud/advanced.md

Advanced Features

This page is oriented towards power users that would like to know more about the mechanics of the DeTEE platform and would like to experiment by adding their own kernel, their own initrd or building OS templates for their own distribution or with their own apps.

Kernel and DTRFS

DeTEE VMs currently run using the latest stable version of the kernel. This is required in order to guarantee that the SNP features are enabled. As the progress moves forward, we will also offer the LTS kernel as part of our default feature set.

AMD SNP attestation is based on the following parameters:

• Number of vCPUs
• Type of the virtual CPU
• Generation of the underlying hardware
• Kernel image
• The initrd image
• Kernel parameters
• VM firmware

The CLI works together with the initrd (which is actually an initramfs) to enable remote attestation. The integration is based on virtee. As you probably imagined, the normal initramfs images offered by normal distributions do not offer the capabilities required for remote attestation, so we decided to build the DeTEE Init RAM FileSystem (DTRFS).

The code required to build DTRFS is currently not public and will get open sourced after the project grows enough to open source the entire intellectual property. You can, however check the contents of the initramfs by downloading the images listed in the CLI:

 $ detee-cli vm dtrfs 
name: dtrfs-6.12.8
vendor: gheorghe
dtrfs_url: http://registry.detee.ltd/dtrfs-payments2025-01-23.cpio.gz
dtrfs_sha: 2e95d7969a0f2ae2ee6f37acd2789a032be1653e76ba93e607477c8b1cde42ed
kernel_url: http://registry.detee.ltd/vmlinuz-linux-6.12.10-arch1-1
kernel_sha: f3a4a74b11c07efa0338c5741d44f13480727e8f2021364a64fcffe1706c6231

name: dtrfs-6.12.9
vendor: ramil
dtrfs_url: http://registry.detee.ltd/detee-constantin-6.12.9-arch1-1.cpio.gz
dtrfs_sha: f1d4d818b5f403ec84b6f1f23cbca3d29ccad6db11941fd60fef1018d9116be4
kernel_url: http://registry.detee.ltd/vmlinuz-linux-6.12.9-arch1-1
kernel_sha: 8094abfd3a2a9dfdbc19b39d7e720eb43116b885abb36fc9431f0c18cbd5938e

DTRFS is based on ArchLinux, however the kernel and the initramfs support any distribution that will work with a generic kernel. Each DTRFS image is linked to an archlinux kernel, in order to guarantee that the kernel gets the modules required to work as a SNP VM running on top of QEMU.

Distributions

Our system currently offers 3 distributions: ArchLinux, Ubuntu and Fedora. You are free to package your own OS template. It should work as long as the operating system works with a generic kernel. Please take the following things into consideration:

• The VM does not need /boot/, meaning it does not need a kernel or a initramfs
• As a consequence, the VM also does not need kernel modules
• Make sure sshd will start and will listen on 0.0.0.0:22 after boot

After creating the image, upload it to any webserver you want and provide the URL to the CLI. Don't forget to also update the sha256sum of the image, as it gets verified during the installation process.

ArchLinux example

Create an ArchLinux VM anywhere (preferably on DeTEE):

SSH into the VM and execute the following commands to build your OS template:

pacman -Syu --noconfirm
pacman -S --noconfirm arch-install-scripts fsarchiver
pacstrap /mnt base openssh
ln -s /usr/lib/systemd/system/sshd.service /mnt/etc/systemd/system/multi-user.target.wants/sshd.service
rm -rf /mnt/var/cache/pacman/pkg
fsarchiver savedir /tmp/os_template.fsa /mnt

Download the OS arch template:

scp -P 49545 root@173.234.17.2:/tmp/os_template.fsa ./detee_arch_$(date -I).fsa
# and grab sha256sum:
sha256sum ./detee_arch_$(date -I).fsa

And upload it to your registry.

Ubuntu example

Create an Ubuntu VM anywhere (preferably on DeTEE) and SSH into the VM by using the SSH. Run the following bash commands to create the template:

apt update
apt dist-upgrade -y
apt install software-properties-common -y
add-apt-repository universe -y
apt install fsarchiver debootstrap -y
debootstrap --include=openssh-server --arch=amd64 noble /mnt http://archive.ubuntu.com/ubuntu/
rm -rf /mnt/var/cache/apt/archives/
ln -s /usr/lib/systemd/system/sshd.service /mnt/etc/systemd/system/multi-user.target.wants/sshd.service
fsarchiver savedir /tmp/os_template.fsa /mnt

Logout of SSH and scp the file to your machine:

scp -P 48331 root@173.234.17.2:/tmp/os_template.fsa ./detee_ubuntu_$(date -I).fsa

Experimenting on your own

If you feel like experimenting, the CLI allows you to deploy a VM using your own kernel and your own OS template. This can be done using detee-cli vm deploy --from-yaml and passing the path to your config. Examples can be found in ~/.detee/cli/new_vm_samples/

This is how the yaml file must look:

hostname: my-specific-vm-01
price: 20000
hours: 5
location:
  country: "GB"
ipv4: !PublishPorts
public_ipv6: false
vcpus: 2
memory_mb: 2000
disk_size_gb: 20
dtrfs:
  name: myinitramfs
  vendor: My Company Ltd
  dtrfs_url: http://registry.detee.ltd/detee-constantin-6.12.9-arch1-1.cpio.gz
  dtrfs_sha: f1d4d818b5f403ec84b6f1f23cbca3d29ccad6db11941fd60fef1018d9116be4
  kernel_url: http://registry.detee.ltd/vmlinuz-linux-6.12.9-arch1-1
  kernel_sha: 8094abfd3a2a9dfdbc19b39d7e720eb43116b885abb36fc9431f0c18cbd5938e
distro:
  name: ubuntu_2025-01-14
  vendor: whoami
  template_url: http://registry.detee.ltd/ubuntu_os_template.fsa
  template_sha: dab318f58c19d31181fc09a497d26408c06fb445913809075d7be74583172205