Add tensorflow demo

.github/workflows/demo_test.yml

@@ -352,6 +352,32 @@ jobs:
         fi
 
 
+  Tensorflow_test:
+    runs-on: ubuntu-18.04
+    steps:
+    - uses: actions/checkout@v1
+      with:
+        submodules: true
+
+    - name: Get occlum version
+      run: echo "OCCLUM_VERSION=$(grep "Version =" src/pal/include/occlum_version.h |  awk '{print $4}')" >> $GITHUB_ENV
+
+    - name: Create container
+      run: docker run -itd --cpuset-cpus="0" --name=tensorflow_test -v $GITHUB_WORKSPACE:/root/occlum occlum/occlum:${{ env.OCCLUM_VERSION }}-ubuntu18.04
+
+    - name: Build dependencies
+      run: docker exec tensorflow_test bash -c "cd /root/occlum; make submodule"
+
+    - name: Make install
+      run: docker exec tensorflow_test bash -c "source /opt/intel/sgxsdk/environment; cd /root/occlum; OCCLUM_RELEASE_BUILD=1 make install"
+
+    - name: Build python and tensorflow
+      run: docker exec tensorflow_test bash -c "cd /root/occlum/demos/tensorflow/tensorflow_training; ./install_python_with_conda.sh"
+
+    - name: Run tensorflow test
+      run: docker exec tensorflow_test bash -c "cd /root/occlum/demos/tensorflow/tensorflow_training; SGX_MODE=SIM ./run_tensorflow_on_occlum.sh 2>&1 | tee /root/occlum/log"
+
+
 # Below tests needs test image to run faster
   Grpc_test:
     runs-on: ubuntu-18.04

demos/README.md

@@ -25,7 +25,8 @@ This set of demos shows how real-world apps can be easily run inside SGX enclave
 * [redis](redis/): A demo of [Redis](https://redis.io).
 * [sofaboot](sofaboot/): A demo of [SOFABoot](https://github.com/sofastack/sofa-boot), an open source Java development framework based on Spring Boot.
 * [sqlite](sqlite/) A demo of [SQLite](https://www.sqlite.org) SQL database engine.
-* [tensorflow_lite](tensorflow_lite/): A demo and benchmark of [Tensorflow Lite](https://www.tensorflow.org/lite) inference engine.
+* [tensorflow](tensorflow/): A demo of [TensorFlow](https://www.tensorflow.org/) MNIST classification training.
+* [tensorflow_lite](tensorflow_lite/): A demo and benchmark of [TensorFlow Lite](https://www.tensorflow.org/lite) inference engine.
 * [vault](golang/vault/): A demo of [HashiCorp Vault](https://github.com/hashicorp/vault).
 * [xgboost](xgboost/): A demo of [XGBoost](https://xgboost.readthedocs.io/en/latest).
 

demos/tensorflow/tensorflow_training/README.md

@@ -0,0 +1,34 @@
+# Use TensorFlow with Python and Occlum
+
+This project demonstrates how Occlum enables _unmodified_ [TensorFlow](https://www.tensorflow.org/) programs running in SGX enclaves, on the basis of _unmodified_ [Python](https://www.python.org). Actually, we have tested various _unmodified_ [TensorFlow Benchmarks](https://github.com/tensorflow/benchmarks) on occlum.
+
+## Sample Code: neural network model
+
+This short introduction uses Keras to:
+
+Build a neural network that classifies MNIST handwritten digit images.
+Train this neural network.
+And, finally, evaluate the accuracy of the model.
+
+## How to Run
+
+This tutorial is written under the assumption that you have Docker installed and use Occlum in a Docker container.
+
+Occlum is compatible with glibc-supported Python, we employ miniconda as python installation tool. You can import TensorFlow packages using conda. Here, miniconda is automatically installed by install_python_with_conda.sh script, the required python and TensorFlow package and MNIST dataset for this project are also loaded by this script. 
+
+Step 1 (on the host): Start an Occlum container
+```
+docker run -it --name=tensorflowDemo --device /dev/sgx occlum/occlum:[version]-ubuntu18.04 bash
+```
+
+Step 2 (on the host): Download miniconda and install python
+```
+cd /root/occlum/demos/tensorflow/tensorflow_training
+bash ./install_python_with_conda.sh
+```
+
+Step 3 (on the host): Run the sample code on Occlum
+```
+cd /root/occlum/demos/tensorflow/tensorflow_training
+bash ./run_tensorflow_on_occlum.sh
+```

demos/tensorflow/tensorflow_training/demo.py

@@ -0,0 +1,23 @@
+import tensorflow as tf
+mnist = tf.keras.datasets.mnist
+
+# Load and prepare the MNIST dataset. Convert the samples from integers to floating-point numbers
+(x_train, y_train), (x_test, y_test) = mnist.load_data(path='/bin/mnist.npz')
+x_train, x_test = x_train / 255.0, x_test / 255.0
+
+# Build the tf.keras.Sequential model by stacking layers. Choose an optimizer and loss function for training
+model = tf.keras.models.Sequential([
+  tf.keras.layers.Flatten(input_shape=(28, 28)),
+  tf.keras.layers.Dense(128, activation='relu'),
+  tf.keras.layers.Dropout(0.2),
+  tf.keras.layers.Dense(10, activation='softmax')
+])
+
+model.compile(optimizer='adam',
+              loss='sparse_categorical_crossentropy',
+              metrics=['accuracy'])
+
+# Train and evaluate model
+model.fit(x_train, y_train, epochs=5)
+
+model.evaluate(x_test,  y_test, verbose=2)

demos/tensorflow/tensorflow_training/install_python_with_conda.sh

@@ -0,0 +1,11 @@
+#!/bin/bash
+set -e
+script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}"  )" >/dev/null 2>&1 && pwd )"
+
+# Install python and dependencies
+[ -f Miniconda3-latest-Linux-x86_64.sh ] || wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
+[ -d miniconda ] || bash ./Miniconda3-latest-Linux-x86_64.sh -b -p $script_dir/miniconda
+$script_dir/miniconda/bin/conda create --prefix $script_dir/python-occlum -y  python=3.7 tensorflow==1.15.0
+
+# Download mnist dataset
+[ -f mnist.npz ] || wget https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz

demos/tensorflow/tensorflow_training/run_tensorflow_on_occlum.sh

@@ -0,0 +1,39 @@
+#!/bin/bash
+set -e
+
+BLUE='\033[1;34m'
+NC='\033[0m'
+
+script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}"  )" >/dev/null 2>&1 && pwd )"
+python_dir="$script_dir/python-occlum"
+
+[ -d occlum_instance ] || occlum new occlum_instance
+
+if [ ! -d $python_dir ];then
+    echo "Error: cannot stat '$python_dir' directory"
+    exit 1
+fi
+
+cd occlum_instance
+# Copy files into Occlum Workspace and build
+if [ ! -L "image/bin/python3" ];then
+    mkdir -p image/opt
+    cp -rf $python_dir image/opt/python-occlum
+    ln -s /opt/python-occlum/bin/python3 image/bin/python3
+    cp -f /opt/occlum/glibc/lib/libdl.so.2 image/opt/occlum/glibc/lib/
+    cp -f /opt/occlum/glibc/lib/libutil.so.1 image/opt/occlum/glibc/lib/
+    cp -f /opt/occlum/glibc/lib/librt.so.1 image/opt/occlum/glibc/lib/
+    cp -f ../demo.py image/bin
+    cp -f ../mnist.npz image/bin
+    new_json="$(jq '.resource_limits.user_space_size = "5400MB" |
+                    .resource_limits.kernel_space_heap_size = "512MB" |
+                    .process.default_mmap_size = "5000MB" |
+                    .resource_limits.max_num_of_threads = 64 |
+                    .env.default += ["PYTHONHOME=/opt/python-occlum", "OMP_NUM_THREADS=1"]' Occlum.json)" && \
+    echo "${new_json}" > Occlum.json
+    occlum build
+fi
+
+# Run the tensorflow demo
+echo -e "${BLUE}occlum run /bin/python3 demo.py${NC}"
+occlum run /bin/python3 /bin/demo.py