Add DEB installer for Occlum and C/C++ toolchains

README.md

@@ -265,7 +265,7 @@ To cut off the complexity of self-hosted infrastructure, one can deploy Occlum-p
 
 ## How to Run Occlum Without Docker?
 
-One easy way to run Occlum without Docker is to use the Occlum Installer. We have prepared the RPM installer for Occlum, which can be downloaded with every release of Occlum. For more info about the installer, see [here](tools/installer/README.md).
+One easy way to run Occlum without Docker is to use the Occlum Installer. We have prepared the RPM and DEB installers for Occlum, which can be downloaded with every release of Occlum. For more info about the installer, see [here](tools/installer/README.md).
 
 ## What is the Implementation Status?
 

tools/installer/README.md

@@ -1,9 +1,19 @@
 # Occlum Installer
 
+To evaluate Occlum in non-docker environment, installers are needed. Occlum provides a variety of installers to support different scenarios. Users can choose to install different minimum subsets of Occlum installers to meet their needs.
+
+- **occlum-runtime**: necessary binaries for `occlum run/exec`. For deployment scenarios, this must be installed.
+- **occlum-pal**: only contains the Occlum PAL library (a thin layer to hide details of libOS and provide API for applications)
+- **occlum-sgx-tools**: minimum dependencies from Intel SGX SDK e.g. sgx-gdb, sgx_sign
+- **occlum-toolchains-\<language\>**: toolchain components for specific language
+- **occlum**: complete package to support all Occlum commands. `occlum-toolchains-gcc` is also installed by default. Please install packages of other programming languages based on your need.
+
 ## RPM Installer
 
 ### How to Build
 
+Normally, Occlum installers should be provided together with release. However, users can also build them on their own.
+
 To build RPM packages, a docker container with Occlum CentOS image (based on CentOS 8.1) is needed. Execute below commands under the occlum directory:
 ```
 cd tools/installer/rpm
@@ -42,9 +52,8 @@ rpm -i libsgx-uae-service-2.9.101.2-1.el7.x86_64.rpm
 **Step 3. Install Occlum Installer and Toolchains Installer**
 ```
 rpm -i occlum-sgx-tools-*.rpm
-rpm -i occlum-"$occlum_version"-*.rpm
 rpm -i occlum-pal-*.rpm
-rpm -i occlum-platform-*.rpm
+rpm -i occlum-runtime-*.rpm
 ```
 
 Toolchains are needed when compile applications and also during runtime. Choose to install the toolchain installer based on the application's language. Currently, we only supports `C/C++`. More language toolchain installers are on the way. To install `C/C++` toolchain, just run the command:
@@ -52,6 +61,11 @@ Toolchains are needed when compile applications and also during runtime. Choose
 rpm -i occlum-toolchains-gcc-*.rpm
 ```
 
+At last, install `occlum` package to get complete support of Occlum:
+```
+rpm -i occlum_*.rpm
+```
+
 To make the new installed binaries and libraries work, this command must be executed:
 ```
 source /etc/profile
@@ -61,4 +75,78 @@ Finally, you are good to go!
 
 
 ## DEB Installer
-TBD
+
+### How to Build
+
+Normally, Occlum installers should be provided together with release. However, users can also build them on their own.
+
+To build deb packages, a docker container with Occlum Ubuntu image (based on Ubuntu 18.04) is needed. Execute below commands under the occlum directory:
+```
+cd tools/installer/deb
+make
+```
+and occlum deb installer can be found under `build/debs`.
+
+If a user wants to build his application on a platform installed with Occlum installer, toolchain installers are also needed. To build language specific toolchain installer, just run the command:
+```
+cd tools/installer/deb
+make <language option>
+```
+Now, only `c/c++` option is supported. And the installer can be found under `build/debs`.
+
+### How to Use
+
+DEB installer should be found together with Occlum release package at [this page](https://github.com/occlum/occlum/releases).
+To run Occlum on clean Ubuntu 18.04, please follow below steps:
+
+**Step 1. Install Prerequisites**
+```
+apt-get update
+apt-get install -y --no-install-recommends libcurl4-openssl-dev libssl-dev libprotobuf-dev libfuse-dev autoconf automake make cmake libtool gdb python jq ca-certificates gnupg wget vim
+```
+
+**Step 2. Install Intel® SGX driver and Intel® SGX PSW**
+Please follow [Intel SGX Installation Guide](https://download.01.org/intel-sgx/sgx-linux/2.9.1/docs/Intel_SGX_Installation_Guide_Linux_2.9.1_Open_Source.pdf) to install SGX driver and SGX PSW. SGX SDK is not required. Using PSW installer is recommanded.
+
+To install PSW, follow the guide to add Intel® SGX repository to apt source. And then run:
+```
+apt-get update
+apt-get install -y libsgx-epid libsgx-urts libsgx-quote-ex libsgx-uae-service
+```
+
+After installing PSW, please make sure that the aesm service is running:
+```
+service status aesmd
+```
+
+**Step 3. Install enable_RDFSBASE Kernel Module**
+Please follow [this README](https://github.com/occlum/enable_rdfsbase/blob/master/README.md) to install `enable_rdfsbase` kernel module.
+
+**Step 4. Install Occlum Installer and Toolchains Installer**
+```
+cd <path to installer>
+apt install -y ./occlum-runtime*.deb
+apt install -y ./occlum-pal*.deb
+apt install -y ./occlum-sgx-tools*.deb
+```
+
+Currently, we only supports `C/C++`. More language toolchain installers are on the way. To install `C/C++` toolchain, just run the command:
+```
+apt install -y ./occlum-toolchains-gcc*.deb
+```
+
+At last, install `occlum` package to get complete support of Occlum:
+```
+apt install -y ./occlum_*.deb
+```
+
+To make the new installed binaries and libraries work, this command must be executed:
+```
+source /etc/profile
+```
+
+**Step 5. Install Debug Symbol Packages (OPTIONAL)**
+If users want to debug the application running inside the libos, debug symbol packages are also needed. Just run:
+```
+apt install -y ./occlum-dbgsym*.ddeb ./occlum-pal-dbgsym*.ddeb ./occlum-runtime-dbgsym*.ddeb ./occlum-toolchains-gcc-dbgsym*.ddeb ./occlum-sgx-tools-dbgsym*.ddeb
+```

tools/installer/deb/Makefile

@@ -0,0 +1,59 @@
+CUR_DIR := $(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
+PROJECT_DIR := $(CUR_DIR)/../../..
+TARGET_DIR := $(abspath $(PROJECT_DIR)/build/debs)
+
+MAJOR_VER_NUM = $(shell grep '\#define OCCLUM_MAJOR_VERSION' $(PROJECT_DIR)/src/pal/include/occlum_version.h | awk '{print $$3}')
+MINOR_VER_NUM = $(shell grep '\#define OCCLUM_MINOR_VERSION' $(PROJECT_DIR)/src/pal/include/occlum_version.h | awk '{print $$3}')
+PATCH_VER_NUM = $(shell grep '\#define OCCLUM_PATCH_VERSION' $(PROJECT_DIR)/src/pal/include/occlum_version.h | awk '{print $$3}')
+VERSION_NUM = $(MAJOR_VER_NUM).$(MINOR_VER_NUM).$(PATCH_VER_NUM)
+
+DEB_BUILD_DIR := /tmp/deb_build
+PACKAGE_DIR := /tmp
+
+.PHONY: all main config_files main_deps c/c++ deps_c clean clean-build
+
+all: main
+
+main: config_files
+	cd $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM); \
+		export VERSION=$(VERSION_NUM); dpkg-buildpackage -us -uc
+	cp -t $(TARGET_DIR) $(DEB_BUILD_DIR)/occlum-runtime*.deb $(DEB_BUILD_DIR)/occlum-runtime-dbgsym*.ddeb $(DEB_BUILD_DIR)/occlum-pal*.deb \
+		$(DEB_BUILD_DIR)/occlum-pal-dbgsym*.ddeb $(DEB_BUILD_DIR)/occlum_$(VERSION_NUM)*.deb $(DEB_BUILD_DIR)/occlum-dbgsym_$(VERSION_NUM)*.ddeb \
+		$(DEB_BUILD_DIR)/occlum-sgx-tools*.deb $(DEB_BUILD_DIR)/occlum-sgx-tools-dbgsym*.ddeb
+
+config_files: main_deps
+	echo " echo 'Please execute command \"source /etc/profile\" to validate envs immediately' " > $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM)/debian/occlum-pal.postinst
+	cp $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM)/debian/occlum-pal.postinst $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM)/debian/occlum-runtime.postinst
+	cp $(PROJECT_DIR)/tools/installer/rpm/occlum*filelist $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM)/debian
+	cp $(PROJECT_DIR)/tools/installer/rpm/occlum-*.sh $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM)/debian
+	sed -e 's/^\///g' -i $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM)/debian/occlum*filelist
+	sed -i -e '$$aetc/profile.d/occlum-pal.sh' $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM)/debian/occlum-pal-filelist
+	sed -i -e '$$aetc/profile.d/occlum-runtime.sh' $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM)/debian/occlum-runtime-filelist
+	sed -i -e 's/$$VERSION/$(VERSION_NUM)/g' $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM)/debian/changelog
+
+main_deps:
+	mkdir -p $(TARGET_DIR)
+	mkdir -p $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM)
+	cp -r occlum $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM)/debian
+	@cp -r $(PROJECT_DIR) $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM)/occlum-src
+	@cd $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM)/occlum-src; \
+		git submodule deinit -f . ; \
+		rm -rf $(DEB_BUILD_DIR)/occlum-$(VERSION_NUM)/occlum-src/{build,demos}
+
+c/c++: deps_c
+	cd $(DEB_BUILD_DIR)/occlum-toolchains-gcc ; \
+		dpkg-buildpackage -us -uc
+	cp -t $(TARGET_DIR) $(DEB_BUILD_DIR)/occlum-toolchains-gcc*.deb $(DEB_BUILD_DIR)/occlum-toolchains-gcc-dbgsym*.ddeb
+
+deps_c:
+	mkdir -p $(TARGET_DIR)
+	mkdir -p $(DEB_BUILD_DIR)/occlum-toolchains-gcc
+	cp -r toolchains/gcc $(DEB_BUILD_DIR)/occlum-toolchains-gcc/debian
+	cp $(PROJECT_DIR)/tools/toolchains/gcc/0014-libgomp-futex-occlum.diff $(DEB_BUILD_DIR)/occlum-toolchains-gcc/debian
+	cp $(PROJECT_DIR)/tools/installer/rpm/toolchains/gcc/occlum-gcc.sh $(DEB_BUILD_DIR)/occlum-toolchains-gcc/debian
+	echo " echo 'Please execute command \"source /etc/profile\" to validate envs immediately' " > $(DEB_BUILD_DIR)/occlum-toolchains-gcc/debian/occlum-gcc.postinst
+
+clean-build:
+	rm -rf $(DEB_BUILD_DIR)
+clean: clean-build
+	rm -rf $(TARGET_DIR)

tools/installer/deb/occlum/changelog

@@ -0,0 +1,5 @@
+occlum ($VERSION-1) unstable; urgency=medium
+
+  * Initial release
+
+ -- Chunyang Hui <sanqian.hcy@antfin.com>  Tue, 18 Aug 2020 14:37:39 +0000

tools/installer/deb/occlum/compat

@@ -0,0 +1 @@
+10

tools/installer/deb/occlum/control

@@ -0,0 +1,30 @@
+Source: occlum
+Section: libs
+Priority: optional
+Maintainer: Chunyang Hui <sanqian.hcy@antfin.com>
+Build-Depends: debhelper (>=9), autoconf, automake, cmake, libtool, ocaml, ocamlbuild, python, libssl-dev, libcurl4-openssl-dev, libprotobuf-dev, git, protobuf-compiler
+Standards-Version: 4.1.2
+Homepage: https://github.com/occlum/occlum
+
+Package: occlum
+Architecture: amd64
+Depends: ${misc:Depends}, occlum-sgx-tools, occlum-pal, occlum-runtime, occlum-toolchains-gcc
+Description: Occlum is a memory-safe, multi-process library OS (LibOS) for Intel SGX.
+ As a LibOS, it enables legacy applications to run on SGX with little or even no modifications of source code,
+ thus protecting the confidentiality and integrity of user workloads transparently.
+
+Package: occlum-runtime
+Architecture: amd64
+Depends: ${misc:Depends}
+Description: occlum-runtime contains command for occlum run/exec
+
+Package: occlum-pal
+Architecture: amd64
+Depends: ${misc:Depends}
+Description: occlum-pal is the Platform Abstraction Layer of Occlum enclave.
+ It provides interfaces to execute trused applications inside enclave.
+
+Package: occlum-sgx-tools
+Architecture: amd64
+Depends: ${misc:Depends}
+Description: Occlum sgx tools used during `occlum build` and `occlum gdb`

tools/installer/deb/occlum/rules

@@ -0,0 +1,51 @@
+#!/usr/bin/make -f
+export DH_VERBOSE = 1
+export deb_build_dir = /tmp/deb_build
+export name = occlum
+
+# Generate all files need by three packages
+export buildroot := $(deb_build_dir)/occlum-$(VERSION)/debian/buildroot
+
+# For package
+export occlum-buildroot := $(deb_build_dir)/occlum-$(VERSION)/debian/occlum
+export occlum-pal-buildroot := $(deb_build_dir)/occlum-$(VERSION)/debian/occlum-pal
+export occlum-runtime-buildroot := $(deb_build_dir)/occlum-$(VERSION)/debian/occlum-runtime
+export occlum-sgx-tools-buildroot := $(deb_build_dir)/occlum-$(VERSION)/debian/occlum-sgx-tools
+
+SHELL = /bin/bash
+
+%:
+	dh $@
+
+override_dh_auto_build:
+	cd $(deb_build_dir)/occlum-$(VERSION)/occlum-src; make submodule
+
+override_dh_install:
+	# install Occlum
+	cd $(deb_build_dir)/occlum-$(VERSION)/occlum-src; OCCLUM_RELEASE_BUILD=1 OCCLUM_PREFIX=$(buildroot)/opt/occlum make install
+	mkdir -p $(buildroot)/etc/profile.d
+	cp $(deb_build_dir)/occlum-$(VERSION)/debian/{occlum-pal.sh,occlum-runtime.sh} $(buildroot)/etc/profile.d
+	chmod 644 $(buildroot)/etc/profile.d/{occlum-pal.sh,occlum-runtime.sh}
+
+	# generate occlum-runtime
+	mkdir -p $(occlum-runtime-buildroot)
+	cd $(buildroot) && cat $(deb_build_dir)/occlum-$(VERSION)/debian/occlum-runtime-filelist | xargs -I files bash -c "cp --parents --target-directory=$(occlum-runtime-buildroot) files"
+
+	# generate occlum-pal
+	mkdir -p $(occlum-pal-buildroot)
+	cd $(buildroot) && cat $(deb_build_dir)/occlum-$(VERSION)/debian/occlum-pal-filelist | xargs -I files bash -c "cp --parents --target-directory=$(occlum-pal-buildroot) files"
+
+	# generate occlum-sgx-tools
+	mkdir -p $(occlum-sgx-tools-buildroot)/opt/occlum
+	cp -r $(buildroot)/opt/occlum/sgxsdk-tools $(occlum-sgx-tools-buildroot)/opt/occlum
+
+	# generate occlum package
+	mkdir -p $(occlum-buildroot)
+	cd $(buildroot) && cat $(deb_build_dir)/occlum-$(VERSION)/debian/occlum-filelist | xargs -I files bash -c "cp --parents --target-directory=$(occlum-buildroot) files"
+
+clean:
+	dh_clean
+	rm -f $(buildroot) $(occlum-buildroot) $(occlum-pal-buildroot) $(occlum-runtime-buildroot)
+
+override_dh_shlibdeps:
+	echo "skip ..."

tools/installer/deb/occlum/source/format

@@ -0,0 +1 @@
+1.0

tools/installer/deb/toolchains/gcc/changelog

@@ -0,0 +1,5 @@
+occlum-toolchains-gcc (0.15.0-1) unstable; urgency=medium
+
+  * Initial release
+
+ -- Chunyang Hui <sanqian.hcy@antfin.com>  Tue, 18 Aug 2020 14:37:39 +0000

tools/installer/deb/toolchains/gcc/compat

@@ -0,0 +1 @@
+10

tools/installer/deb/toolchains/gcc/control

@@ -0,0 +1,12 @@
+Source: occlum-toolchains-gcc
+Section: libs
+Priority: optional
+Maintainer: Chunyang Hui <sanqian.hcy@antfin.com>
+Build-Depends: debhelper (>=9), autoconf, automake, cmake, libtool, ocaml, ocamlbuild, python, libssl-dev, libcurl4-openssl-dev, libprotobuf-dev, git, protobuf-compiler
+Standards-Version: 4.1.2
+Homepage: https://github.com/occlum/musl
+
+Package: occlum-toolchains-gcc
+Architecture: amd64
+Depends: ${misc:Depends}, ${shlibs:Depends}
+Description: Occlum gcc toolchains

tools/installer/deb/toolchains/gcc/rules

@@ -0,0 +1,100 @@
+#!/usr/bin/make -f
+export DH_VERBOSE = 1
+export deb_build_dir = /tmp/deb_build
+export name = occlum-toolchains-gcc
+export buildroot := $(deb_build_dir)/$(name)/debian/$(name)
+
+export tmp_build_dir = $(deb_build_dir)/$(name)/tmp_build
+export install_dir = $(buildroot)/opt/occlum/toolchains/gcc
+
+SHELL = /bin/bash
+
+# Needed by musl-cross-make config
+TARGET = x86_64-linux-musl
+MUSL_REPO = https://github.com/occlum/musl
+MUSL_VER = 1.1.24
+GCC_VER = 8.3.0
+
+.ONESHELL: prepare_musl override_dh_install
+
+%:
+	dh $@
+
+# All rules executed in one shell
+prepare_musl:
+	rm -rf $(tmp_build_dir)
+	rm -rf $(install_dir)
+	mkdir -p $(tmp_build_dir)
+	cd $(tmp_build_dir)
+	git clone https://github.com/occlum/musl-cross-make
+	cd musl-cross-make
+	git checkout 0.9.9.hotfix
+	cp $(deb_build_dir)/$(name)/debian/0014-libgomp-*.diff patches/gcc-$(GCC_VER)
+	cat > config.mak <<EOF
+		TARGET = $(TARGET)
+		OUTPUT = $(install_dir)
+		COMMON_CONFIG += CFLAGS="-fPIC" CXXFLAGS="-fPIC" LDFLAGS="-pie"
+		GCC_VER = $(GCC_VER)
+		MUSL_VER = git-$(MUSL_VER)
+		MUSL_REPO = $(MUSL_REPO)
+	EOF
+
+override_dh_auto_build: prepare_musl
+	cd $(tmp_build_dir)/musl-cross-make
+	cd $(tmp_build_dir)/musl-cross-make; \
+		make -j$(nproc)
+
+# We create symlinks in /usr/local
+override_dh_usrlocal:
+	echo "Skip ..."
+
+override_dh_fixperms:
+	dh_fixperms
+	# Occlum need this to be executable
+	chmod 755 $(install_dir)/$(TARGET)/lib/libc.so
+
+override_dh_install:
+	cd $(tmp_build_dir)/musl-cross-make
+	make install
+	cat > $(install_dir)/bin/occlum-gcc <<EOF
+		#!/bin/bash
+		/opt/occlum/toolchains/gcc/bin/$(TARGET)-gcc -fPIC -pie -Wl,-rpath,/opt/occlum/toolchains/gcc/$(TARGET)/lib "\$$@"
+	EOF
+
+	cat > $(install_dir)/bin/occlum-g++ <<EOF
+		#!/bin/bash
+		/opt/occlum/toolchains/gcc/bin/$(TARGET)-g++ -fPIC -pie -Wl,-rpath,/opt/occlum/toolchains/gcc/$(TARGET)/lib "\$$@"
+	EOF
+
+	cat > $(install_dir)/bin/occlum-ld <<EOF
+		#!/bin/bash
+		/opt/occlum/toolchains/gcc/bin/$(TARGET)-ld -pie -rpath /opt/occlum/toolchains/gcc/$(TARGET)/lib "\$$@"
+	EOF
+
+	chmod +x $(install_dir)/bin/occlum-gcc
+	chmod +x $(install_dir)/bin/occlum-g++
+	chmod +x $(install_dir)/bin/occlum-ld
+
+	mkdir -p $(buildroot)/lib
+	pushd $(buildroot)/lib
+	ln -sf /opt/occlum/toolchains/gcc/$(TARGET)/lib/libc.so ld-musl-x86_64.so.1
+	popd
+	mkdir -p $(buildroot)/usr/local
+	pushd $(buildroot)/usr/local
+	ln -sf /opt/occlum/toolchains/gcc occlum
+	popd
+	pushd $(install_dir)/bin
+	ln -sf /opt/occlum/toolchains/gcc/bin/x86_64-linux-musl-gcc-ar occlum-ar
+	ln -sf /opt/occlum/toolchains/gcc/bin/x86_64-linux-musl-strip occlum-strip
+	popd
+
+	mkdir -p $(buildroot)/etc/profile.d
+	cp $(deb_build_dir)/$(name)/debian/occlum-gcc.sh $(buildroot)/etc/profile.d
+	chmod 644 $(buildroot)/etc/profile.d/occlum-gcc.sh
+
+clean:
+	dh_clean
+	rm -f $(buildroot)
+
+override_dh_shlibdeps:
+	echo "skip ..."

tools/installer/deb/toolchains/gcc/source/format

@@ -0,0 +1 @@
+1.0

tools/installer/rpm/occlum.spec

@@ -56,7 +56,7 @@ occlum-platform contains command for occlum enclave.
 %build
 # build occlum
 cd %{?_builddir}/%{name}-%{_version}/occlum-%{_version}
-NOT_GIT=true make submodule
+make submodule
 
 %install
 # set sgxsdk env