Commit Graph

393 Commits

Author SHA1 Message Date
zongmin.gu
b28aee68b7 Upgrade Intel SGX SDK to 2.11 and Rust SGX SDK accordingly 2020-09-18 15:39:58 +08:00
Hui, Chunyang
4031216f13 Add timerslack concept for libos and enbale prctl PR_GET_TIMERSLACK option 2020-09-18 12:39:12 +08:00
LI Qing
1a11655169 Fix two bugs related to open directories
1. Support O_DIRECTORY flag for open syscall
2. Disallow to open a directory in write mode
2020-09-18 12:39:12 +08:00
LI Qing
292fc28340 Add "*at()" system calls
The syscalls implemented in this submission are as follows:
fchmodat, fchownat, linkat, mkdirat, renameat, readlinkat, unlinkat
2020-09-18 12:39:12 +08:00
He Sun
9c390923a5 Narrow the focus of exec client/server build
Only change of the proto file should trigger the build script.
2020-09-18 12:39:12 +08:00
He Sun
83fcd47cd0 Reorganize the header files of pal 2020-09-18 12:39:12 +08:00
He Sun
5d5e8d44ec Pass host-generated SIGPIPE to libos
Socket-related ocalls, e.g, sendto, sendmsg and write, may cause SIGPIPE
in host. Since the ocall is called by libos, this kind of signal should
be handled in libos. We ignore SIGPIPE in host and raise the same signal
in libos if the return value of the above ocalls is EPIPE. In this way
the signal is handled by libos.
2020-09-18 12:39:12 +08:00
Hui, Chunyang
7b882bd368 Add support for pal library to run libos outside occlum instance 2020-09-18 12:39:12 +08:00
He Sun
e13242e7e5 Add netdevice ioctl command 2020-09-18 12:39:12 +08:00
Zongmin
edd0980dc3 Fix exec mode occlum server crashes issue (if the client unexpected exit) 2020-09-18 12:39:12 +08:00
Tate, Hongliang Tian
ccb5e6bef4 Bump version to 0.15.1 2020-08-21 00:19:57 +08:00
LI Qing
70d7d10eeb Add support to mkdir & rmdir in hostfs 2020-08-20 17:03:30 +08:00
LI Qing
7d31cb743c Add support to read directory in hostfs 2020-08-20 17:03:30 +08:00
zongmin.gu
85f5bc7ccc Fix a potential memory issue in fpregs' free 2020-08-20 17:01:08 +08:00
duanbing
000cd88756 Get quote size dynamiclly 2020-08-17 01:30:24 +08:00
Tate, Hongliang Tian
3e3a1955af Bump version to 0.15.0 2020-08-15 19:19:53 +08:00
Hui, Chunyang
9435b1a196 Add RPM installer for Occlum and C/C++ toolchains 2020-08-15 19:12:40 +08:00
zongmin.gu
2ca5629b3d Save floating point registers in exception/interrupt flow 2020-08-15 19:12:40 +08:00
zongmin.gu
68c8cc100b Fix the syscall interface bug 2020-08-15 19:12:40 +08:00
Hui, Chunyang
7ac917aa1a Fix PAL library command arguments not follow convention 2020-08-15 19:12:39 +08:00
Hui, Chunyang
66e5cefec2 Add "occlum new" command 2020-08-15 19:12:39 +08:00
Hui, Chunyang
85501d8993 Improve implementation for occlum build
This commit mainly accomplish two things:
1. Use makefile to manage dependencies for `occlum build`, which can save lots of time
2. Take dirs `build`, `run` outside from `.occlum`. Remove env var "OCCLUM_INSTANCE_DIR"
2020-08-15 19:12:39 +08:00
LI Qing
3f6bcec1c5 Substitute ramFS with a temporary SEFS at "/tmp" 2020-08-15 19:12:39 +08:00
He Sun
b04aa2d7ea Fix the dequeuing order of items in a FutexBucket 2020-08-15 19:12:39 +08:00
LI Qing
6d39587c40 Add getcpu syscall 2020-08-15 19:12:39 +08:00
zongmin.gu
c67bdd9a23 Update the syscall interface 2020-08-11 17:35:03 +08:00
Hui, Chunyang
259c485427 Use new build directory arch 2020-08-11 13:47:17 +08:00
He Sun
9b17ac1847 Improve futex performance
1. Enlarge the size of the futex buckets;
2. Wake up the waiting threads in one ocall.
2020-07-28 13:14:27 +08:00
He Sun
2400cc4baa Fix the signal action examination of SIGKILL and SIGSTOP
Sigaction() can be called with a NULL second argument to query the
current signal handler.
2020-07-28 12:20:34 +08:00
LI Qing
572873d9a4 Enable UnionFS 2020-07-23 21:40:34 +08:00
Tate, Hongliang Tian
55eaae8810 Bump version to 0.14.0 2020-07-18 06:36:59 +08:00
Hui, Chunyang
ebd7dc056b Add occlum specific EDL file to decouple with official linux-sgx-sdk psw
In this case, sefs-fuse can run with official linux-sgx-sdk psw.
2020-07-18 06:36:59 +08:00
Tate, Hongliang Tian
8c7b59ad17 Fix a bug in sigaction
The bug would allow the user to block non-blockable signals (SIGKILL and SIGSTOP)
using sigaction syscall. This commit fixes this bug.
2020-07-18 01:39:04 +08:00
Tate, Hongliang Tian
a6dbce21cc Add msync system call and flush updates to shared file memory mappings
1. Add msync-related methods to VMManager;
2. Add msync system call;
3. Fix munmap, fsync, and fdatasync system calls;
4. Add test cases.
2020-07-17 16:57:00 +00:00
He Sun
c85163ec0a Add notification mechanism for basic IO events
1. Add notification mechanism for select, poll, epoll and blocking IO
2. Add pipe support for select, poll and blocking IO
2020-07-18 00:27:26 +08:00
Hui, Chunyang
9b1d694830 Add support for per-process memory size configuration with rlimit
Rlimit are now on the same page of memory space limits defined in Occlum.json. Specific
memory size configuration can be set to child process with `prlimit` syscall or using `ulimit`
command in shell script.
2020-07-17 22:07:56 +08:00
Hui, Chunyang
c43fbfea7f Fix signal testcase and a bug in sig_set 2020-07-17 18:02:09 +08:00
Hui, Chunyang
4f96790cf3 Fallback to old way for exit process in simulation mode 2020-07-16 21:35:47 +08:00
zongmin.gu
45b799d99f Stop the server within the given seconds.
If a new command comes when the timer is not triggered, stop the timer.
2020-07-16 20:12:12 +08:00
He Sun
3d70ca9355 Substitute SgxRwLock with RwLock 2020-07-16 14:35:38 +08:00
He Sun
4f965fd8db Apply RwLock to the sig_queues of Process and Thread
The new RwLock has better performance than SgxMutex and SgxRwLock.
2020-07-16 14:35:38 +08:00
He Sun
83637d7938 Add a new kind of readers-writer lock 2020-07-16 14:35:32 +08:00
Tate, Hongliang Tian
0db804d131 Support sa_mask for sigaction syscall
Struct sigaction has a field named sa_mask, which specifies the blocked
signals while executing the signal handler. Previously, this field is not
supported. This commit adds this missing feature.
2020-07-15 10:20:36 +08:00
He Sun
cfda47b316 Check the input buffer size against the available CPUs in sched_get/setaffinity
There are scenarios where the available CPUs are less than all the CPUs
on the machine. Therefore, sched_get/setaffinity should be allowed when
the input buffer size is no less than the available CPUs but less than
all the CPUs.
2020-07-15 09:58:23 +08:00
He Sun
b4750c0fcd Revert "Zeroize memory in munmap"
This reverts commit 1e456f025d6b4e34a726180e7a27a04424fe79d1.
This commit results in segmentation fault when the application munmaps
its own stack. Should be committed back after removing the dependency of
sysret on the user space stack.
2020-07-14 10:32:12 +08:00
Tate, Hongliang Tian
518ff76228 Bump version to 0.13.1 2020-07-11 14:19:41 +00:00
Tate, Hongliang Tian
a3ad465ce3 Fix SGX simulation mode broken by the interrupt subsystem
The new interrupt subsystem breaks the simulation mode in two ways:

1. The signal 64 is not handled by Intel SGX SDK in simulation mode. A
handled real-time signal crashes the process.

2. The newly-enabled test case exit_group depends on interrupts. But
enclave interrupts, like enclave exceptions, are not supported in
simulation mode.

This commit ensures signal 64 is ignored by default and exit_group test
case is not enabled in simulation mode.
2020-07-11 14:19:41 +00:00
Tate, Hongliang Tian
b585fce65d Add the interrupt subsystem
Before this commit, events like signals and exit_group are handled by
LibOS threads in a cooperative fashion: if the user code executed by a
LibOS thread does not invoke system calls (e.g., a busy loop), then the LibOS
won't have any opportunity to take control and handle events.

With the help from the POSIX signal-based interrupt mechanism of
Occlum's version of Intel SGX SDK, the LibOS can now interrupt the
execution of arbitrary user code in a LibOS thread by sending real-time
POSIX signals (the signal number is 64) to it. These signals are sent by
a helper thread spawn by Occlum PAL. The helper thread periodically
enters into the enclave to check if there are any LibOS threads with
pending events. If any, the helper thread broadcast POSIX signals to
them. When interrupted by a signal, the receiver LibOS thread may be in
one of the two previously problematic states in terms of event handling:

1. Executing non-cooperative user code (e.g., a busy loop). In this
case, the signal will trigger an interrupt handler inside the enclave,
which can then enter the LibOS kernel to deal with any pending events.

2. Executing an OCall that invokes blocking system calls (e.g., futex,
nanosleep, or blocking I/O). In this case, the signal will interrupt the
blocking system call so that the OCall can return back to the enclave.

Thanks to the new interrupt subsystem, some event-based system calls
are made robust. One such example is exit_group. We can now guarantee
that exit_group can force any thread in a process to exit.
2020-07-10 11:52:01 +00:00
zongmin.gu
f52a732d50 Remove the occlum server auto destory policy 2020-07-10 19:23:57 +08:00
Tate, Hongliang Tian
35787be29d Fix two bugs on process exit
The first bug is a race condition when acquiring the lock of a process's
parent. An example code with race condition looks like below:

```rust
let process : ProessRef = current!().process();
let parent : ProcessRef = process.parent();
let parent_guard : SgxMutexGuard<ProesssInner> = parent.inner();
// This assertion may fail because the process's parent may change to another
// process before the lock is acquired
assert!(parent.pid() == process.parent().pid());
```

The second bug is that when a process exits, its children processes are
not transfered to the idle process correctly.
2020-07-10 19:08:26 +08:00
LI Qing
686ec343b2 Fix the dirfd issue when path is absolute 2020-07-10 18:08:03 +08:00
LI Qing
1ad8f22170 Add support to handle symbolic link file 2020-07-10 18:08:03 +08:00
Hui, Chunyang
3cd46fd224 Add prctl support of PR_SET/GET_NAME options 2020-07-10 08:31:54 +00:00
Zongmin
b86d8ed490 Fix occlum pal build issue 2020-07-09 19:12:02 +08:00
Hui, Chunyang
406f30ec7a Polish build and install process
Remove redundent files and make processes for SGX simulation mode and hardware mode.
2020-07-08 11:51:33 +00:00
He Sun
1e456f025d Zeroize memory in munmap
1. Move the memory zeroization of mmap to munmap to increase mmap
performance
2. Do memory zeroizaiton during the drop of VMManager to guarentee all
allocated memory is zeroized before the next allocation
2020-07-01 20:23:54 +08:00
Hui, Chunyang
92207d5535 Add support for executing a script that begins with a shebang 2020-06-27 07:56:06 +00:00
He Sun
ec970f0e76 Refactor performance profiler
1. Add the thread profiler to Thread structure
2. Fix bugs introduced by previous commits
2020-06-23 22:17:41 +08:00
Hui, Chunyang
6909629241 Add Occlum version for dynamic libraries 2020-06-22 07:36:03 +00:00
He Sun
f854950416 Optimize ProcessVM to be interiorly mutable 2020-06-21 16:22:03 +08:00
Tate, Hongliang Tian
bca0663972 Add mprotect system call 2020-06-19 07:32:31 +00:00
He Sun
b9b9b1032c Avoid locking for checking if a process has been forced to exit
It turns out taking a lock in every system call is a significant
performance bottleneck. In light of this finding, we replace a mutex in
a critical path of system call with an atomic boolean.
2020-06-17 22:54:33 +08:00
He Sun
3a5793be4f Fix set_tid_address not supporting null pointers 2020-06-17 07:44:47 +08:00
Tate, Hongliang Tian
9cefcb08b6 Bump version to 0.13.0 2020-06-13 04:29:50 +00:00
Zongmin
898967554c Link uRTS static library 2020-06-13 04:19:45 +00:00
Hui, Chunyang
6e57937b45 Add support for sysinfo syscall 2020-06-13 03:33:32 +00:00
Hui, Chunyang
bddb87a11b Fix stat for symbolic file 2020-06-13 03:33:32 +00:00
LI Qing
c8e3e0ffc6 Add clock_getres syscall 2020-06-13 03:33:32 +00:00
Tate, Hongliang Tian
70d3991ff5 Rewrite mremap system call
This rewrite serves three purposes:
1. Fix some subtle bugs in the old implementation;
2. Implement mremap using mmap and munmap so that mremap can automatically
enjoy new features (e.g., mprotect and memory permissions) once mmap and
munmap support the feature.
3. Write down the invariants hold by VMManager explictly so that the correctness
of the new implementation can be reason more easily.
2020-06-13 03:33:32 +00:00
He Sun
28440b0d69 Build in the MAC of the occlum configuration file with objcopy
1. Objcopy the MAC of Occlum.json to libocclum-libos.so before signature
during occlum build.
2. Remove the files and codes no longer used.
2020-06-13 03:33:32 +00:00
He Sun
f020fed2ae Use Intel SGX SDK reserved memory as the user space memory 2020-06-13 03:33:32 +00:00
jack.wxz
cfbab68a9d Use pal api interface without prefix of libos type.
Signed-off-by: jack.wxz <wangxiaozhe@linux.alibaba.com>
2020-06-12 09:02:07 +08:00
zongmin.gu
4e02db367e Update occlum exec to support kill 2020-06-05 14:52:10 +08:00
zongmin.gu
942321363d Combine the enclave configuration into the occlum configuration file
Update the occlum.json to align with the gen_enclave_conf design.
Below is the two updated structures:
   "metadata": {
        "product_id": 0,
        "version_number": 0,
        "debuggable": true
    },
    "resource_limits": {
        "max_num_of_threads": 32,
        "kernel_space_heap_size": "32MB",
        "kernel_space_stack_size": "1MB",
        "user_space_size": "256MB"
    }
2020-06-05 11:03:47 +08:00
LI Qing
f54abc78a2 Support to execute a symlink file 2020-06-04 20:46:50 +08:00
Tate, Hongliang Tian
655869711a Support page fault 2020-06-04 20:36:29 +08:00
Tate, Hongliang Tian
8c113eb8a2 Fix a minor format issue 2020-06-04 11:47:48 +00:00
Tianjia Zhang
aed88145f2 pal: Fix crash caused by getpwuid() returning NULL
On lightweight Linux distribution, like alpine, getpwuid()
returns NULL, and errno is ENOENT, this patch fix crash
caused by this situation.

Signed-off-by: Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
2020-06-04 19:36:15 +08:00
Tianjia Zhang
e8c48161b5 pal: Fix typo error in saving token file
The address of the `token` should be used by fwrite().

Signed-off-by: Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
2020-06-04 19:33:57 +08:00
He Sun
eb4bb860ce Add the unix socket support for sendto system call 2020-06-03 17:57:38 +08:00
He Sun
987e06a458 Move networking system call interfaces to net module 2020-06-03 17:57:38 +08:00
Hui, Chunyang
ed664d1143 Fix sched set/get affinity wrong behaviour
1. Fix child process not inherit affinity from parent process
2. Add warning for buffer length gap with kernel when getaffinity
2020-06-02 17:36:36 +08:00
Hui, Chunyang
73b4f8b08c Add support for PAL API v2 2020-05-29 05:26:38 +00:00
LI Qing
e6996f3c45 Add mremap system call 2020-05-29 03:38:49 +00:00
Hui, Chunyang
5b695c9539 Format c/c++ files in src, tools and test 2020-05-27 07:09:18 +00:00
Hui, Chunyang
03ba13aec7 Add "make format" and "make format-check" to check format for c/c++ files 2020-05-27 07:08:59 +00:00
He Sun
cd2f13ae54 Refactor select syscall
1. Substitute the underlying poll OCall to select OCall to update the
timeout argument correctly.
2. Add more checks for the inputs.
2020-05-26 11:44:46 +08:00
Hui, Chunyang
c14ee62678 Add untrusted environment variable override
Add "untrusted" sections for environment variables defined in Occlum.json. Environment
variable defined in "default" will be shown in libos directly. Environment variable
defined in "untrusted" can be passed from occlum run or PAL layer and can override
the value in "default" and thus is considered "untrusted".
2020-05-25 03:14:54 +00:00
Hui, Chunyang
dadffb9f17 Fix sendfile syscall write buffer length 2020-05-21 17:20:12 +08:00
Hui, Chunyang
c6e0e06768 Clean up the build files generated in src/exec 2020-05-21 08:54:42 +00:00
Zongmin
17b4912055 Send SIGCHLD to parent process 2020-05-18 20:24:14 +08:00
Tate, Hongliang Tian
e1789dc89f Bump version to 0.12.0
Also, update the docs accordingly.
2020-05-15 11:32:44 +00:00
He Sun
9252a1a993 Fix two bugs of ioctl
1. Add length of the argument to the ioctl ocall to guard sgx sdk to do
the copy.
2. Handle non-zero return value of ioctl.
2020-05-15 12:50:31 +08:00
Zongmin
0c3466f4ad Add three new occlum commands: start, exec and stop
Usage:
//start the occlum server
occlum start

//execute the command inside occlum
occlum exec [cmd] [-- <args>]

//stop the occlum server
occlum stop
2020-05-15 03:02:42 +00:00
LI Qing
2d34ee349e Add the sigaltstack system call 2020-05-15 03:02:42 +00:00
Hui, Chunyang
0bb1cbfbb5 Add PAL version api 2020-05-15 03:02:42 +00:00
He Sun
f9486bf7a9 Add resource limit check for the number of the fds in poll 2020-05-15 03:02:42 +00:00
zongmin.gu
eca27408be Support user manage stack
Go/Java/JIT code manage their own stack
So we need to help them to handle exception
2020-05-15 03:02:42 +00:00
Zongmin
a6d97d5b4f Update the sdk version to 2.9.1
Fix std::alloc::Alloc not found
The lastest Rust changes the trait to std::alloc::AllocRef.

Update the docker files to support sgx 2.9.1

Remove the compilerRT dependency for rust sdk update
2020-05-15 03:02:42 +00:00
Tate, Hongliang Tian
1d1330772c Add new API occlum_pal_kill
This API enables sending signals to one or multiple LibOS processes from
outside the enclave.
2020-05-15 03:02:42 +00:00