general/occlum
4
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update init_ra_flow demo with init_grpc_ratls

Browse Source
This commit is contained in:
Zheng, Qi 2023-03-02 17:01:40 +08:00 committed by volcano
parent a096d176c9
commit d9676d52d0
9 changed files with 66 additions and 363 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
demos/remote_attestation/dcap/README.md
View File

@ -16,6 +16,18 @@ applications through ioctls on device `/dev/sgx`.
include `libsgx_quote_ex, libsgx_quote_ex_sim, libsgx_dcap_tvl,
libsgx_dcap_ql and libsgx_dcap_quoteverify`.
You can simply start a Occlum develop container to meet above two.
A valid PCCS service should be accessible in your environment. This demo is verified in Aliyun, thus `https://sgx-dcap-server.cn-shanghai.aliyuncs.com/sgx/certification/v3/` is used as the PCCS URL. For example,
* Start the Occlum develop container
```
docker run --rm -it \
--device /dev/sgx/enclave --device /dev/sgx/provision \
occlum/occlum:latest-ubuntu20.04 bash
```
In the container, update the **pccs_url** in the file `/etc/sgx_default_qcnl.conf` with the valid address.
## Run this demo on Occlum
You can run the DCAP quote generation and verification demo, including rust test demo and C test demo on Occlum via

25
demos/remote_attestation/init_ra_flow/README.md
View File

@ -4,13 +4,15 @@
Remote attestation is a key part in the confidential computing. Occlum provides a [`DCAP Library`](../../../tools/toolchains/dcap_lib/) to ease the RA application. A brief Occlum RA [`introduction`](../../../docs/remote_attestation.md) is a good entry point. But still, utilizing the RA in a real application is complicated for general developers.
Occlum provides a `Init RA` way to seperate the RA operation and the actual application. With this way, the APP developers don't need know too much about the RA and the application doesn't need to be modified for RA.
Occlum provides a `Init RA` way to separate the RA operation and the actual application. With this way, the APP developers don't need know too much about the RA and the application doesn't need to be modified for RA.
This demo shows the `Init RA` way with a sample [`Flask TLS web application`](../../python/flask/), based on [`GRPC-RATLS`](../../../tools/toolchains/grpc_ratls/ra_tls/) server/client implementation and a modified [`init`](./init_ra/) for Occlum InitFS.
This demo shows the `Init RA` way with a sample [`Flask TLS web application`](../../python/flask/), based on [`GRPC-RATLS`](../../../tools/toolchains/grpc_ratls) server/client implementation and a customized [`init`](../../../tools/init_grpc_ratls) for Occlum InitFS.
Note: users could choose different `init` by passing parameters on `occlum init` or `occlum new`. In this demo, **grpc-ratls** client is chosen.
![Arch Overview](./arch.png)
The GRPC-RATLS server holds some sensitive data thus it is usually deploed on secure environment. The application consuming the sensitive data could be deployed on general environment, such as Cloud service vendor provided SGX instance. For this demo, all are running on one SGX machine.
The GRPC-RATLS server holds some sensitive data thus it is usually deployed on secure environment. The application consuming the sensitive data could be deployed on general environment, such as Cloud service vendor provided SGX instance. For this demo, all are running on one SGX machine.
## Flow
@ -59,11 +61,24 @@ The `RA Verify Config` JSON records the secrets. Each secret has a name and its
* Starts the Flask-TLS-Infer demo. For every Occlum built application, it starts `init` process first, then starts the real application in RootFS. The default [`init`](../../../tools/init/) process just run RootFS integrity check and then load the RootFS where the real application is located.
For this demo, a modified [`init`](./init_ra/) is used. Besides the general `init` operation, it embeds the `GRPC-RATLS` client API `grpc_ratls_get_secret`, gets the secrets(base64 encoded) from the `GRPC-RATLS server`, does base64 decoding, acquires the real secrets. The `image_key` is used to decrypt the RootFS image. The other two are saved to RootFS. In this example, they are `/etc/flask.crt` and `/etc/flask.key`. Finally, when the Flask-TLS app is running, all secrets are securely obtained already in `init` thus the app runs successfully without RA involvement in this stage.
For this demo, a customized [`init`](../../../tools/init_grpc_ratls) is used. Besides the general `init` operation, it embeds the `GRPC-RATLS` client API `grpc_ratls_get_secret`, gets the secrets(base64 encoded) from the `GRPC-RATLS server`, does base64 decoding, acquires the real secrets. The `image_key` is used to decrypt the RootFS image. The other two are saved to RootFS. In this example, they are `/etc/flask.crt` and `/etc/flask.key`. Finally, when the Flask-TLS app is running, all secrets are securely obtained already in `init` thus the app runs successfully without RA involvement in this stage.
Also, to configure which KMS server (GRPC-RATLS server) is to be trusted, the default `init_ra_conf.json` needs modified accordingly. Details please refer to the function **update_client_init_ra_conf** in the script [build_content.sh](./build_content.sh).
## How-to build the demo
A valid PCCS service should be accessible in your environment. This demo is verified in Aliyun, thus `https://sgx-dcap-server.cn-shanghai.aliyuncs.com/sgx/certification/v3/` is used as the PCCS URL. For example,
* Start the Occlum develop container
```
docker run --rm -it \
--device /dev/sgx/enclave --device /dev/sgx/provision \
occlum/occlum:latest-ubuntu20.04 bash
```
In the container, update the **pccs_url** in the file `/etc/sgx_default_qcnl.conf` with the valid address.
* Just run `build_content.sh` which builds everything.
Once successful, two Occlum instances are created.
```
@ -82,7 +97,7 @@ occlum run /bin/server localhost:50051 &
* Starts the Flask-TLS web portal in backgroud.
```
cd occlum_client
occlum run /bin/rest_api.py &
occlum run --config-svn 1234 /bin/rest_api.py &
```
Above two could be executed in one script [`run.sh`](./run.sh).

67
demos/remote_attestation/init_ra_flow/build_content.sh
View File

@ -3,16 +3,7 @@ set -e
script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
export INITRA_DIR="${script_dir}/init_ra"
export FLASK_DIR="${script_dir}/../../python/flask"
export RATLS_DIR="${script_dir}/../../../tools/toolchains/grpc_ratls/"
function build_ratls()
{
pushd ${RATLS_DIR}
./build.sh
popd
}
function build_flask()
{
@ -21,12 +12,23 @@ function build_flask()
popd
}
function build_init_ra()
function update_client_init_ra_conf()
{
pushd ${INITRA_DIR}
occlum-cargo clean
occlum-cargo build --release
popd
# Fill in the keys
new_json="$(jq '.kms_keys = [ {"key": "flask_cert", "path": "/etc/flask.crt"}] |
.kms_keys += [ {"key": "flask_key", "path": "/etc/flask.key"}]' init_ra_conf.json)" && \
echo "${new_json}" > init_ra_conf.json
# Fill in the KMS server measurements.
new_json="$(jq ' .ra_config.verify_mr_enclave = "off" |
.ra_config.verify_mr_signer = "on" |
.ra_config.verify_isv_prod_id = "off" |
.ra_config.verify_isv_svn = "off" |
.ra_config.verify_config_svn = "off" |
.ra_config.verify_enclave_debuggable = "on" |
.ra_config.sgx_mrs[0].mr_signer = ''"'`get_mr client mrsigner`'" |
.ra_config.sgx_mrs[0].debuggable = true ' init_ra_conf.json)" && \
echo "${new_json}" > init_ra_conf.json
}
function build_client_instance()
@ -34,38 +36,30 @@ function build_client_instance()
# generate client image key
occlum gen-image-key image_key
rm -rf occlum_client && occlum new occlum_client
rm -rf occlum_client
# choose grpc_ratls as init ra kms client
occlum new occlum_client --init-ra grpc_ratls
pushd occlum_client
# prepare flask content
rm -rf image
copy_bom -f ../flask.yaml --root image --include-dir /opt/occlum/etc/template
# Try build first to get mrsigner
# In our case, client and server use the same sign-key thus also the same mrsigner
occlum build
new_json="$(jq '.resource_limits.user_space_size = "600MB" |
.resource_limits.kernel_space_heap_size = "128MB" |
.resource_limits.max_num_of_threads = 32 |
.metadata.debuggable = false |
.metadata.debuggable = true |
.metadata.enable_kss = true |
.metadata.version_number = 88 |
.env.default += ["PYTHONHOME=/opt/python-occlum"]' Occlum.json)" && \
.env.default += ["PYTHONHOME=/opt/python-occlum"] ' Occlum.json)" && \
echo "${new_json}" > Occlum.json
occlum build --image-key ../image_key
# Get server mrsigner.
# Here client and server use the same signer-key thus using client mrsigner directly.
jq ' .verify_mr_enclave = "off" |
.verify_mr_signer = "on" |
.verify_isv_prod_id = "off" |
.verify_isv_svn = "off" |
.verify_config_svn = "off" |
.verify_enclave_debuggable = "on" |
.sgx_mrs[0].mr_signer = ''"'`get_mr client mrsigner`'" |
.sgx_mrs[0].debuggable = false ' ../ra_config_template.json > dynamic_config.json
# prepare init-ra content
rm -rf initfs
copy_bom -f ../init_ra_client.yaml --root initfs --include-dir /opt/occlum/etc/template
# Update init_ra_conf json file accordingly before occlum build
update_client_init_ra_conf
occlum build -f --image-key ../image_key
@ -106,10 +100,10 @@ function build_server_instance()
.sgx_mrs[0].mr_signer = ''"'`get_mr client mrsigner`'" |
.sgx_mrs[0].isv_svn = 88 |
.sgx_mrs[0].config_svn = 1234 |
.sgx_mrs[0].debuggable = false ' ../ra_config_template.json > dynamic_config.json
.sgx_mrs[0].debuggable = true ' ../ra_config_template.json > dynamic_config.json
new_json="$(jq '.resource_limits.user_space_size = "500MB" |
.metadata.debuggable = false ' Occlum.json)" && \
.metadata.debuggable = true ' Occlum.json)" && \
echo "${new_json}" > Occlum.json
rm -rf image
@ -120,9 +114,6 @@ function build_server_instance()
popd
}
build_ratls
build_flask
build_init_ra
build_client_instance
build_server_instance

94
demos/remote_attestation/init_ra_flow/init_ra/Cargo.lock generated
View File

@ -1,94 +0,0 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
[[package]]
name = "init"
version = "0.0.1"
dependencies = [
"libc",
"serde",
"serde_json",
]
[[package]]
name = "itoa"
version = "0.4.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dd25036021b0de88a0aff6b850051563c6516d0bf53f8638938edbb9de732736"
[[package]]
name = "libc"
version = "0.2.84"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1cca32fa0182e8c0989459524dc356b8f2b5c10f1b9eb521b7d182c03cf8c5ff"
[[package]]
name = "proc-macro2"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e0704ee1a7e00d7bb417d0770ea303c1bccbabf0ef1667dae92b5967f5f8a71"
dependencies = [
"unicode-xid",
]
[[package]]
name = "quote"
version = "1.0.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3d0b9745dc2debf507c8422de05d7226cc1f0644216dfdfead988f9b1ab32a7"
dependencies = [
"proc-macro2",
]
[[package]]
name = "ryu"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "71d301d4193d031abdd79ff7e3dd721168a9572ef3fe51a1517aba235bd8f86e"
[[package]]
name = "serde"
version = "1.0.123"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92d5161132722baa40d802cc70b15262b98258453e85e5d1d365c757c73869ae"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.123"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9391c295d64fc0abb2c556bad848f33cb8296276b1ad2677d1ae1ace4f258f31"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "serde_json"
version = "1.0.62"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ea1c6153794552ea7cf7cf63b1231a25de00ec90db326ba6264440fa08e31486"
dependencies = [
"itoa",
"ryu",
"serde",
]
[[package]]
name = "syn"
version = "1.0.60"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c700597eca8a5a762beb35753ef6b94df201c81cca676604f547495a0d7f0081"
dependencies = [
"proc-macro2",
"quote",
"unicode-xid",
]
[[package]]
name = "unicode-xid"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f7fe0bb3479651439c9112f72b6c505038574c9fbb575ed1bf3b797fa39dd564"

11
demos/remote_attestation/init_ra_flow/init_ra/Cargo.toml
View File

@ -1,11 +0,0 @@
[package]
name = "init"
version = "0.0.1"
build = "build.rs"
authors = ["LI Qing geding.lq@antgroup.com"]
edition = "2021"
[dependencies]
libc = "0.2.84"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"

7
demos/remote_attestation/init_ra_flow/init_ra/build.rs
View File

@ -1,7 +0,0 @@
fn main() {
println!("cargo:rustc-link-search=native=/opt/occlum/toolchains/dcap_lib/musl");
println!("cargo:rustc-link-search=native=/opt/occlum/toolchains/grpc_ratls/musl");
println!("cargo:rustc-link-lib=dylib=grpc_ratls_client");
println!("cargo:rustc-link-lib=dylib=hw_grpc_proto");
println!("cargo:rustc-link-lib=dylib=occlum_dcap");
}

182
demos/remote_attestation/init_ra_flow/init_ra/src/main.rs
View File

@ -1,182 +0,0 @@
extern crate libc;
extern crate serde;
extern crate serde_json;
use libc::syscall;
use serde::Deserialize;
use std::error::Error;
use std::fs;
use std::fs::File;
use std::io::{ErrorKind, Read};
use std::ffi::CString;
use std::os::raw::{c_int, c_char};
#[link(name = "grpc_ratls_client")]
extern "C" {
fn grpc_ratls_get_secret(
server_addr: *const c_char, // grpc server address+port, such as "localhost:50051"
config_json: *const c_char, // ratls handshake config json file
name: *const c_char, // secret name to be requested
secret_file: *const c_char // secret file to be saved
) -> c_int;
}
#[link(name = "grpc_ratls_client")]
extern "C" {
fn grpc_ratls_get_secret_to_buf(
server_addr: *const c_char, // grpc server address+port, such as "localhost:50051"
config_json: *const c_char, // ratls handshake config json file
name: *const c_char, // secret name to be requested
secret_buf: *const u8, // secret buffer provided by user
buf_len: *mut u32 // buffer size
) -> c_int;
}
fn main() -> Result<(), Box<dyn Error>> {
// Load the configuration from initfs
const IMAGE_CONFIG_FILE: &str = "/etc/image_config.json";
let image_config = load_config(IMAGE_CONFIG_FILE)?;
// Get client secrets through grpc-ratls
let server_addr = CString::new("localhost:50051").unwrap();
let config_json = CString::new("dynamic_config.json").unwrap();
// Get the key of FS image if needed
let key = match &image_config.image_type[..] {
"encrypted" => {
// Get the image encrypted key through RA
let secret = CString::new("image_key").unwrap();
let mut buffer: Vec<u8> = vec![0; 256];
let buffer_ptr: *const u8 = buffer.as_ptr();
let mut buffer_len: u32 = buffer.len() as u32;
let len_ptr: *mut u32 = &mut buffer_len as *mut u32;
//Read to buffer instead of file system for better security
let ret = unsafe {
grpc_ratls_get_secret_to_buf(
server_addr.as_ptr(),
config_json.as_ptr(),
secret.as_ptr(),
buffer_ptr,
len_ptr
)
};
if ret != 0 {
println!("grpc_ratls_get_secret failed return {}", ret);
return Err(Box::new(std::io::Error::last_os_error()));
}
buffer.resize(buffer_len as usize, 0);
let key_string = String::from_utf8(buffer)
.expect("error converting to string");
let key_str = key_string
.trim_end_matches(|c| c == '\r' || c == '\n').to_string();
let mut key: sgx_key_128bit_t = Default::default();
parse_str_to_bytes(&key_str, &mut key)?;
Some(key)
}
"integrity-only" => None,
_ => unreachable!(),
};
let key_ptr = key
.as_ref()
.map(|key| key as *const sgx_key_128bit_t)
.unwrap_or(std::ptr::null());
// Get certificate
let secret = CString::new("flask_cert").unwrap();
let filename = CString::new("cert_file").unwrap();
let ret = unsafe {
grpc_ratls_get_secret(
server_addr.as_ptr(),
config_json.as_ptr(),
secret.as_ptr(),
filename.as_ptr())
};
if ret != 0 {
println!("grpc_ratls_get_secret failed return {}", ret);
return Err(Box::new(std::io::Error::last_os_error()));
}
let cert_secret = fs::read_to_string(filename.into_string().unwrap())
.expect("Something went wrong reading the file");
// Get key
let secret = CString::new("flask_key").unwrap();
let filename = CString::new("key_file").unwrap();
let ret = unsafe {
grpc_ratls_get_secret(
server_addr.as_ptr(),
config_json.as_ptr(),
secret.as_ptr(),
filename.as_ptr())
};
if ret != 0 {
println!("grpc_ratls_get_secret failed return {}", ret);
return Err(Box::new(std::io::Error::last_os_error()));
}
let key_secret = fs::read_to_string(filename.into_string().unwrap())
.expect("Something went wrong reading the file");
// Mount the image
const SYS_MOUNT_FS: i64 = 363;
// User can provide valid path for runtime mount and boot
// Otherwise, just pass null pointer to do general mount and boot
let rootfs_config: *const i8 = std::ptr::null();
let ret = unsafe { syscall(SYS_MOUNT_FS, key_ptr, rootfs_config) };
if ret < 0 {
return Err(Box::new(std::io::Error::last_os_error()));
}
// Write the secrets to rootfs
fs::write("/etc/flask.crt", cert_secret.into_bytes())?;
fs::write("/etc/flask.key", key_secret.into_bytes())?;
Ok(())
}
#[allow(non_camel_case_types)]
type sgx_key_128bit_t = [u8; 16];
#[derive(Deserialize, Debug)]
#[serde(deny_unknown_fields)]
struct ImageConfig {
image_type: String,
}
fn load_config(config_path: &str) -> Result<ImageConfig, Box<dyn Error>> {
let mut config_file = File::open(config_path)?;
let config_json = {
let mut config_json = String::new();
config_file.read_to_string(&mut config_json)?;
config_json
};
let config: ImageConfig = serde_json::from_str(&config_json)?;
Ok(config)
}
fn parse_str_to_bytes(arg_str: &str, bytes: &mut [u8]) -> Result<(), Box<dyn Error>> {
let bytes_str_vec = {
let bytes_str_vec: Vec<&str> = arg_str.split('-').collect();
if bytes_str_vec.len() != bytes.len() {
return Err(Box::new(std::io::Error::new(
ErrorKind::InvalidData,
"The length or format of Key/MAC string is invalid",
)));
}
bytes_str_vec
};
for (byte_i, byte_str) in bytes_str_vec.iter().enumerate() {
bytes[byte_i] = u8::from_str_radix(byte_str, 16)?;
}
Ok(())
}

21
demos/remote_attestation/init_ra_flow/init_ra_client.yaml
View File

@ -1,21 +0,0 @@
includes:
- base.yaml
targets:
- target: /bin/
copy:
- files:
- ${INITRA_DIR}/target/x86_64-unknown-linux-musl/release/init
- target: /lib/
copy:
- files:
- /opt/occlum/toolchains/grpc_ratls/musl/libgrpc_ratls_client.so
- /opt/occlum/toolchains/grpc_ratls/musl/libhw_grpc_proto.so
- /opt/occlum/toolchains/dcap_lib/musl/libocclum_dcap.so.0.1.0
- target: /
copy:
- files:
- dynamic_config.json
- target: /etc
copy:
- dirs:
- /etc/ssl

10
demos/remote_attestation/init_ra_flow/ra_server.yaml
View File

@ -4,13 +4,13 @@ targets:
- target: /bin/
copy:
- files:
- /opt/occlum/toolchains/grpc_ratls/musl/server
- target: /lib/
- /opt/occlum/toolchains/grpc_ratls/glibc/server
- target: /opt/occlum/glibc/lib
copy:
- files:
- /opt/occlum/toolchains/grpc_ratls/musl/libgrpc_ratls_server.so
- /opt/occlum/toolchains/grpc_ratls/musl/libhw_grpc_proto.so
- /opt/occlum/toolchains/dcap_lib/musl/libocclum_dcap.so.0.1.0
- /opt/occlum/glibc/lib/libnss_files.so.2
- /opt/occlum/toolchains/grpc_ratls/glibc/libgrpc_ratls_server.so
- /opt/occlum/toolchains/dcap_lib/glibc/libocclum_dcap.so.0.1.0
- target: /
copy:
- files: