Commit Graph

349 Commits

Author SHA1 Message Date
Hui, Chunyang
7ac917aa1a Fix PAL library command arguments not follow convention 2020-08-15 19:12:39 +08:00
LI Qing
3f6bcec1c5 Substitute ramFS with a temporary SEFS at "/tmp" 2020-08-15 19:12:39 +08:00
He Sun
b04aa2d7ea Fix the dequeuing order of items in a FutexBucket 2020-08-15 19:12:39 +08:00
LI Qing
6d39587c40 Add getcpu syscall 2020-08-15 19:12:39 +08:00
zongmin.gu
c67bdd9a23 Update the syscall interface 2020-08-11 17:35:03 +08:00
Hui, Chunyang
259c485427 Use new build directory arch 2020-08-11 13:47:17 +08:00
He Sun
9b17ac1847 Improve futex performance
1. Enlarge the size of the futex buckets;
2. Wake up the waiting threads in one ocall.
2020-07-28 13:14:27 +08:00
He Sun
2400cc4baa Fix the signal action examination of SIGKILL and SIGSTOP
Sigaction() can be called with a NULL second argument to query the
current signal handler.
2020-07-28 12:20:34 +08:00
LI Qing
572873d9a4 Enable UnionFS 2020-07-23 21:40:34 +08:00
Tate, Hongliang Tian
55eaae8810 Bump version to 0.14.0 2020-07-18 06:36:59 +08:00
Tate, Hongliang Tian
8c7b59ad17 Fix a bug in sigaction
The bug would allow the user to block non-blockable signals (SIGKILL and SIGSTOP)
using sigaction syscall. This commit fixes this bug.
2020-07-18 01:39:04 +08:00
Tate, Hongliang Tian
a6dbce21cc Add msync system call and flush updates to shared file memory mappings
1. Add msync-related methods to VMManager;
2. Add msync system call;
3. Fix munmap, fsync, and fdatasync system calls;
4. Add test cases.
2020-07-17 16:57:00 +00:00
He Sun
c85163ec0a Add notification mechanism for basic IO events
1. Add notification mechanism for select, poll, epoll and blocking IO
2. Add pipe support for select, poll and blocking IO
2020-07-18 00:27:26 +08:00
Hui, Chunyang
9b1d694830 Add support for per-process memory size configuration with rlimit
Rlimit are now on the same page of memory space limits defined in Occlum.json. Specific
memory size configuration can be set to child process with `prlimit` syscall or using `ulimit`
command in shell script.
2020-07-17 22:07:56 +08:00
Hui, Chunyang
c43fbfea7f Fix signal testcase and a bug in sig_set 2020-07-17 18:02:09 +08:00
He Sun
3d70ca9355 Substitute SgxRwLock with RwLock 2020-07-16 14:35:38 +08:00
He Sun
4f965fd8db Apply RwLock to the sig_queues of Process and Thread
The new RwLock has better performance than SgxMutex and SgxRwLock.
2020-07-16 14:35:38 +08:00
He Sun
83637d7938 Add a new kind of readers-writer lock 2020-07-16 14:35:32 +08:00
Tate, Hongliang Tian
0db804d131 Support sa_mask for sigaction syscall
Struct sigaction has a field named sa_mask, which specifies the blocked
signals while executing the signal handler. Previously, this field is not
supported. This commit adds this missing feature.
2020-07-15 10:20:36 +08:00
He Sun
cfda47b316 Check the input buffer size against the available CPUs in sched_get/setaffinity
There are scenarios where the available CPUs are less than all the CPUs
on the machine. Therefore, sched_get/setaffinity should be allowed when
the input buffer size is no less than the available CPUs but less than
all the CPUs.
2020-07-15 09:58:23 +08:00
He Sun
b4750c0fcd Revert "Zeroize memory in munmap"
This reverts commit 1e456f025d6b4e34a726180e7a27a04424fe79d1.
This commit results in segmentation fault when the application munmaps
its own stack. Should be committed back after removing the dependency of
sysret on the user space stack.
2020-07-14 10:32:12 +08:00
Tate, Hongliang Tian
518ff76228 Bump version to 0.13.1 2020-07-11 14:19:41 +00:00
Tate, Hongliang Tian
b585fce65d Add the interrupt subsystem
Before this commit, events like signals and exit_group are handled by
LibOS threads in a cooperative fashion: if the user code executed by a
LibOS thread does not invoke system calls (e.g., a busy loop), then the LibOS
won't have any opportunity to take control and handle events.

With the help from the POSIX signal-based interrupt mechanism of
Occlum's version of Intel SGX SDK, the LibOS can now interrupt the
execution of arbitrary user code in a LibOS thread by sending real-time
POSIX signals (the signal number is 64) to it. These signals are sent by
a helper thread spawn by Occlum PAL. The helper thread periodically
enters into the enclave to check if there are any LibOS threads with
pending events. If any, the helper thread broadcast POSIX signals to
them. When interrupted by a signal, the receiver LibOS thread may be in
one of the two previously problematic states in terms of event handling:

1. Executing non-cooperative user code (e.g., a busy loop). In this
case, the signal will trigger an interrupt handler inside the enclave,
which can then enter the LibOS kernel to deal with any pending events.

2. Executing an OCall that invokes blocking system calls (e.g., futex,
nanosleep, or blocking I/O). In this case, the signal will interrupt the
blocking system call so that the OCall can return back to the enclave.

Thanks to the new interrupt subsystem, some event-based system calls
are made robust. One such example is exit_group. We can now guarantee
that exit_group can force any thread in a process to exit.
2020-07-10 11:52:01 +00:00
Tate, Hongliang Tian
35787be29d Fix two bugs on process exit
The first bug is a race condition when acquiring the lock of a process's
parent. An example code with race condition looks like below:

```rust
let process : ProessRef = current!().process();
let parent : ProcessRef = process.parent();
let parent_guard : SgxMutexGuard<ProesssInner> = parent.inner();
// This assertion may fail because the process's parent may change to another
// process before the lock is acquired
assert!(parent.pid() == process.parent().pid());
```

The second bug is that when a process exits, its children processes are
not transfered to the idle process correctly.
2020-07-10 19:08:26 +08:00
LI Qing
686ec343b2 Fix the dirfd issue when path is absolute 2020-07-10 18:08:03 +08:00
LI Qing
1ad8f22170 Add support to handle symbolic link file 2020-07-10 18:08:03 +08:00
Hui, Chunyang
3cd46fd224 Add prctl support of PR_SET/GET_NAME options 2020-07-10 08:31:54 +00:00
Hui, Chunyang
406f30ec7a Polish build and install process
Remove redundent files and make processes for SGX simulation mode and hardware mode.
2020-07-08 11:51:33 +00:00
He Sun
1e456f025d Zeroize memory in munmap
1. Move the memory zeroization of mmap to munmap to increase mmap
performance
2. Do memory zeroizaiton during the drop of VMManager to guarentee all
allocated memory is zeroized before the next allocation
2020-07-01 20:23:54 +08:00
Hui, Chunyang
92207d5535 Add support for executing a script that begins with a shebang 2020-06-27 07:56:06 +00:00
He Sun
ec970f0e76 Refactor performance profiler
1. Add the thread profiler to Thread structure
2. Fix bugs introduced by previous commits
2020-06-23 22:17:41 +08:00
Hui, Chunyang
6909629241 Add Occlum version for dynamic libraries 2020-06-22 07:36:03 +00:00
He Sun
f854950416 Optimize ProcessVM to be interiorly mutable 2020-06-21 16:22:03 +08:00
Tate, Hongliang Tian
bca0663972 Add mprotect system call 2020-06-19 07:32:31 +00:00
He Sun
b9b9b1032c Avoid locking for checking if a process has been forced to exit
It turns out taking a lock in every system call is a significant
performance bottleneck. In light of this finding, we replace a mutex in
a critical path of system call with an atomic boolean.
2020-06-17 22:54:33 +08:00
He Sun
3a5793be4f Fix set_tid_address not supporting null pointers 2020-06-17 07:44:47 +08:00
Tate, Hongliang Tian
9cefcb08b6 Bump version to 0.13.0 2020-06-13 04:29:50 +00:00
Hui, Chunyang
6e57937b45 Add support for sysinfo syscall 2020-06-13 03:33:32 +00:00
Hui, Chunyang
bddb87a11b Fix stat for symbolic file 2020-06-13 03:33:32 +00:00
LI Qing
c8e3e0ffc6 Add clock_getres syscall 2020-06-13 03:33:32 +00:00
Tate, Hongliang Tian
70d3991ff5 Rewrite mremap system call
This rewrite serves three purposes:
1. Fix some subtle bugs in the old implementation;
2. Implement mremap using mmap and munmap so that mremap can automatically
enjoy new features (e.g., mprotect and memory permissions) once mmap and
munmap support the feature.
3. Write down the invariants hold by VMManager explictly so that the correctness
of the new implementation can be reason more easily.
2020-06-13 03:33:32 +00:00
He Sun
28440b0d69 Build in the MAC of the occlum configuration file with objcopy
1. Objcopy the MAC of Occlum.json to libocclum-libos.so before signature
during occlum build.
2. Remove the files and codes no longer used.
2020-06-13 03:33:32 +00:00
He Sun
f020fed2ae Use Intel SGX SDK reserved memory as the user space memory 2020-06-13 03:33:32 +00:00
zongmin.gu
942321363d Combine the enclave configuration into the occlum configuration file
Update the occlum.json to align with the gen_enclave_conf design.
Below is the two updated structures:
   "metadata": {
        "product_id": 0,
        "version_number": 0,
        "debuggable": true
    },
    "resource_limits": {
        "max_num_of_threads": 32,
        "kernel_space_heap_size": "32MB",
        "kernel_space_stack_size": "1MB",
        "user_space_size": "256MB"
    }
2020-06-05 11:03:47 +08:00
LI Qing
f54abc78a2 Support to execute a symlink file 2020-06-04 20:46:50 +08:00
Tate, Hongliang Tian
655869711a Support page fault 2020-06-04 20:36:29 +08:00
He Sun
eb4bb860ce Add the unix socket support for sendto system call 2020-06-03 17:57:38 +08:00
He Sun
987e06a458 Move networking system call interfaces to net module 2020-06-03 17:57:38 +08:00
Hui, Chunyang
ed664d1143 Fix sched set/get affinity wrong behaviour
1. Fix child process not inherit affinity from parent process
2. Add warning for buffer length gap with kernel when getaffinity
2020-06-02 17:36:36 +08:00
LI Qing
e6996f3c45 Add mremap system call 2020-05-29 03:38:49 +00:00
He Sun
cd2f13ae54 Refactor select syscall
1. Substitute the underlying poll OCall to select OCall to update the
timeout argument correctly.
2. Add more checks for the inputs.
2020-05-26 11:44:46 +08:00
Hui, Chunyang
c14ee62678 Add untrusted environment variable override
Add "untrusted" sections for environment variables defined in Occlum.json. Environment
variable defined in "default" will be shown in libos directly. Environment variable
defined in "untrusted" can be passed from occlum run or PAL layer and can override
the value in "default" and thus is considered "untrusted".
2020-05-25 03:14:54 +00:00
Hui, Chunyang
dadffb9f17 Fix sendfile syscall write buffer length 2020-05-21 17:20:12 +08:00
Zongmin
17b4912055 Send SIGCHLD to parent process 2020-05-18 20:24:14 +08:00
Tate, Hongliang Tian
e1789dc89f Bump version to 0.12.0
Also, update the docs accordingly.
2020-05-15 11:32:44 +00:00
He Sun
9252a1a993 Fix two bugs of ioctl
1. Add length of the argument to the ioctl ocall to guard sgx sdk to do
the copy.
2. Handle non-zero return value of ioctl.
2020-05-15 12:50:31 +08:00
Zongmin
0c3466f4ad Add three new occlum commands: start, exec and stop
Usage:
//start the occlum server
occlum start

//execute the command inside occlum
occlum exec [cmd] [-- <args>]

//stop the occlum server
occlum stop
2020-05-15 03:02:42 +00:00
LI Qing
2d34ee349e Add the sigaltstack system call 2020-05-15 03:02:42 +00:00
He Sun
f9486bf7a9 Add resource limit check for the number of the fds in poll 2020-05-15 03:02:42 +00:00
zongmin.gu
eca27408be Support user manage stack
Go/Java/JIT code manage their own stack
So we need to help them to handle exception
2020-05-15 03:02:42 +00:00
Zongmin
a6d97d5b4f Update the sdk version to 2.9.1
Fix std::alloc::Alloc not found
The lastest Rust changes the trait to std::alloc::AllocRef.

Update the docker files to support sgx 2.9.1

Remove the compilerRT dependency for rust sdk update
2020-05-15 03:02:42 +00:00
Tate, Hongliang Tian
1d1330772c Add new API occlum_pal_kill
This API enables sending signals to one or multiple LibOS processes from
outside the enclave.
2020-05-15 03:02:42 +00:00
Tate, Hongliang Tian
6e140a0d38 Add errno info for ECalls
Before this commit, the three ECalls of the LibOS enclave do not give
the exact reason on error. In this commit, we modify the enclave entry code
to return the errno and list all possible values of errno in Enclave.edl.
2020-05-15 02:59:16 +00:00
He Sun
1c707eda30 Add support for monitoring epoll fds with epoll 2020-05-15 02:59:16 +00:00
Hui, Chunyang
25350b0e85 Fix sched agent failure for offline core 2020-05-15 02:59:16 +00:00
He Sun
7a87d77509 Fix error when PF_UNIX is passed as protocol to create unix socket
Zero and PF_UNIX are both supported as protocol when creating unix
socket.
2020-05-15 02:59:16 +00:00
Tate, Hongliang Tian
e166382923 Add the signal subsystem
In this commit, we add eight signal-related syscalls
* kill
* tkill
* tgkill
* rt_sigaction
* rt_sigreturn
* rt_sigprocmask
* rt_sigpending
* exit_group

We implement the following major features for signals:
* Generate, mask, and deliver signals
* Support user-defined signal handlers
    * Support nested invocation of signal handlers
    * Support passing arguments: signum, sigaction, and ucontext
* Support both process-directed and thread-directed signals
* Capture hardware exceptions and convert them to signals
* Deliver fatal signals (like SIGKILL) to kill processes gracefully

But we still have gaps, including but not limited to the points below:
* Convert #PF (page fault) and #GP (general protection) exceptions to signals
* Force delivery of signals via interrupt
* Support simulation mode
2020-05-15 02:59:16 +00:00
LI Qing
1172c25677 Modify occlum_entry to be compatible with Linux syscall ABI 2020-05-15 02:59:16 +00:00
Hui, Chunyang
72f2a33e2a Add log level alignment with OCI 2020-05-15 02:59:16 +00:00
He Sun
71475c3e96 Fix wrong assert in recvmsg for MSG_TRUNC flag 2020-05-15 02:59:16 +00:00
He Sun
bbb52f6990 Refine check for pointers from user space and outside enclave
Implement the check functions that are empty before and do some
adjustment where they are called.
2020-05-15 02:59:16 +00:00
LI Qing
1dcabb09cd Fix the access and faccessat system calls 2020-05-15 02:59:16 +00:00
LI Qing
6d27595195 Fix the negative offset check for pread/pwrite 2020-05-15 02:59:16 +00:00
He Sun
48d7f8df3e Fix the invalid epoll_event pointer introduced by compilation optimization
That pointer does not point to the libc::epoll_event after release
build. Explicitly declaring the libc::epoll_event avoids the
invalidation.
2020-05-15 02:59:16 +00:00
Hui, Chunyang
6a17e6292c Add support for user specified instance dir name
The default instance dir of Occlum is ".occlum". User now can specify the name
by declaring environment variable "OCCLUM_INSTANCE_DIR"
2020-05-15 02:59:16 +00:00
LI Qing
03bb09abdf Add the emulation of SYSCALL instruction
1. Refactor the exception handling process
2. Support to emulate the SYSCALL instruction in the exception handler
2020-05-15 02:59:16 +00:00
Hui, Chunyang
79fd6570f5 Change log level to warning for ioctl TIOCGWINSZ 2020-05-15 02:59:16 +00:00
He Sun
aa10d96f89 Fix u32::MAX not found error
Using std::u32 instead of u32 passes the compilation.
2020-05-15 02:59:16 +00:00
He Sun
e651be4403 Fix not passing null pointers to ocalls for empty slices
It gets a non-null fat pointer when an empty slice simply calls as_ptr
or as_mut_ptr.
2020-05-15 02:59:16 +00:00
He Sun
58403f8415 Fix panic when dropping unix socket
When a unix socket only calls function listen, its object is not created
but its status becomes listening. At this time closing the socket would
cause a panic before this commit.
2020-05-15 02:59:16 +00:00
He Sun
9815523a95 Fix panic when there is no enough memory for a new ringbuffer 2020-05-15 02:59:16 +00:00
Hui, Chunyang
53f8ade4de Fix a typo 2020-05-15 02:59:16 +00:00
yuanwu
ed7f83562a Check the MPX status of platform before enabling
Signed-off-by: yuanwu <yuan.wu@intel.com>
2020-04-26 15:02:26 +08:00
Tate, Hongliang Tian
5b4b72a81a Bump the version number to 0.11.0 2020-04-17 17:52:17 +08:00
Tate, Hongliang Tian
df71872f76 Disable Intel MPX by default
The next generation of Intel CPUs does not support Intel MPX. Enabling MPX
by default crashes the LibOS on startup. So we disable MPX by default. The
long term plan is to turn on/off MPX via compiling options.
2020-04-15 21:25:24 +08:00
Tate, Hongliang Tian
2b556f8de9 Refactor the CPU scheduling subsystem
This commits improves both readability and correctness of the scheduling-related
system calls. In terms of readability, it extracts all scheduling-related code
ouf of the process/ directory and put it in a sched/ directory. In terms
of correctness, the new scheduling subsystem introduces CpuSet and SchedAgent
types to maintain and manipulate CPU scheduler settings in a secure and robust way.
2020-04-15 09:43:38 +00:00
Tate, Hongliang Tian
2a1d3d98c5 Refactor the process/thread subsystem
As a major rewrite to the process/thread subsystem, this commits:
1. Implements threads as a first-class object, which represents a group of OS resources
and a thread of execution;
2. Implements processes as a first-class object that manages threads and maintains
the parent-child relationship between processes;
3. Refactors the code in process subsystem to follow the improved coding style and
conventions emerged in recent commits;
4. Refactors the code in other subsystems to use the new process/thread subsystem.
2020-04-15 06:22:41 +00:00
LI Qing
bbc3b8a467 Fix RDTSC emulation's changing app's TLS by mistake
This commit switches from user mode to kernel mode before handling
the exception caused by RDTSC instruction.
2020-04-10 09:53:39 +00:00
He Sun
52cb897436 Fix F_DUPFD crash when there is no space in file table 2020-04-03 12:47:04 +08:00
LI Qing
978edf8a17 Add chmod and chown system calls 2020-04-03 03:39:57 +00:00
LI Qing
e1648fc870 Add the redirection of standard I/O for process 2020-04-02 11:07:24 +00:00
He Sun
221f5b78e8 Rewrite epoll implementation and the test 2020-04-02 17:43:24 +08:00
Tate, Hongliang Tian
a7400ca6dc Fix thread exiting but not being freed 2020-03-31 14:47:31 +00:00
LI Qing
23329efe14 Fix the broken dependency on SEFS
SEFS depends on version 0.9 of bitvec crate, which has been yanked on crates.io
by the crate author for some reasons. To fix this, we upgrade to the latest
version of bitvec crate.
2020-03-30 08:58:57 +00:00
Tate, Hongliang Tian
1eefa0a123 Fix poll when given negative fds 2020-03-27 18:00:49 +08:00
Tate, Hongliang Tian
6dbdbfdcf0 Fix potential deadlocks caused by epoll/poll/select 2020-03-27 05:42:20 +00:00
Tate, Hongliang Tian
1a35188212 Simplify the frequent code of getting a file reference 2020-03-26 10:34:14 +00:00
Tate, Hongliang Tian
5933499f9b Refactor LibOS to conform with logging strategy
This commit introduces a unified logging strategy, summarized as below:

1. Use `error!` to mark errors or unexpected conditions, e.g., a
 `Result::Err` returned from a system call.
2. Use `warn!` to warn about potentially problematic issues, e.g.,
 executing a workaround or fake implementation.
3. Use `info!` to show important events (from users' perspective) in
 normal execution, e.g., creating/exiting a process/thread.
4. Use `debug!` to track major events in normal execution, e.g., the
 high-level arguments of a system call.
5. Use `trace!` to record the most detailed info, e.g., when a system
 call enters and exits the LibOS.
2020-03-25 02:53:31 +00:00
Tate, Hongliang Tian
9713e74ed9 Add the support for setting log level at runtime
Now one can specify the log level of the LibOS by setting `OCCLUM_LOG_LEVEL`
environment variable. The possible values are "off", "error", "warn",
"info", and "trace".

However, for the sake of security, the log level of a release enclave
(DisableDebug = 1 in Enclave.xml) is always "off" (i.e., no log) regardless of
the log level specified by the untrusted environment.
2020-03-25 01:40:47 +00:00
Tate, Hongliang Tian
6d7cf7b9f6 Add more info to log messages (e.g., timestamp, thread, and round) 2020-03-25 01:34:41 +00:00