Commit Graph

277 Commits

Author SHA1 Message Date
LI Qing
33876e122f Fix the compiler's warnings in make test with glibc 2021-08-04 12:00:31 +08:00
Hui, Chunyang
b2b86b796a Add support for /dev/fd 2021-08-03 21:30:15 +08:00
LI Qing
dc37995bf0 Add getdents support for procfs's inode 2021-08-03 20:40:07 +08:00
LI Qing
42bed8d338 Remove the mount config for tmp dir 2021-08-03 19:45:56 +08:00
LI Qing
215e8ffbdf Add support for robust futex syscalls 2021-07-30 10:25:24 +08:00
ClawSeven
d35d98d551 Add resolv-conf parser 2021-07-28 10:52:46 +08:00
LI Qing
a54de67431 Fix the wrong value returned from getcwd
`getcwd` should return the length of buffer filled
2021-07-27 13:05:35 +08:00
LI Qing
17e4810d3e Add support for "/proc/[pid]/comm" and fix "/proc/[pid]/cmdline" 2021-07-20 19:56:12 +08:00
ClawSeven
eb07b01ac0 Support timerfd-related syscall 2021-07-16 14:36:27 +08:00
Hui, Chunyang
09b7828f44 Disable busyloop test case due to signal handling constraints in SIM mode 2021-06-25 21:12:42 +08:00
Hui, Chunyang
c62b6d4091 Add support for execve
RFC: https://github.com/occlum/occlum/issues/429
2021-06-10 20:59:06 +08:00
Hui, Chunyang
0dc85f8229 Add support for indexing unix domain socket file with inode 2021-06-10 20:50:30 +08:00
zongmin.gu
2cedafeacb Add getgroups syscall 2021-06-10 20:14:37 +08:00
Hui, Chunyang
883f7b259f Add support for ioctl FIONREAD, FIOCLEX, FIONCLEX for pipe and uds file 2021-06-07 08:28:51 +08:00
LI Qing
eb046d4241 Add O_TRUNC support in open syscall and rewrite the truncate test 2021-06-04 11:47:38 +08:00
LI Qing
789b57c6f7 Add statfs and fstatfs syscall 2021-06-04 11:36:47 +08:00
zongmin.gu
070bdf6f39 Add sendmmsg syscall 2021-06-03 16:24:34 +08:00
Hui, Chunyang
d15a75fafb Add support for POSIX_SPAWN_SETSIGMASK and POSIX_SPAWN_SETSIGDEF 2021-06-01 20:34:51 +08:00
LI Qing
69c79d8252 Fix a potential deadlock when dereference procfs's symlink 2021-06-01 18:40:41 +08:00
LI Qing
9a76ca1888 Add support for '/proc/[pid]/root' 2021-05-25 12:04:50 +08:00
Hui, Chunyang
6d37dd2d90 Add support for mremap with file-backed memory 2021-05-23 15:14:06 +08:00
Hui, Chunyang
a95e26df42 Add support for WNOHANG of waitpid option 2021-05-10 10:11:24 +08:00
LI Qing
ba720dc346 Add fstat support for pipe 2021-05-08 11:25:08 +08:00
Hui, Chunyang
f77e2c5e89 Add support for ioctl TCGETS/TCSETS 2021-04-29 14:23:01 +08:00
LI Qing
8db6a55696 Mount ramfs at '/dev/shm' 2021-04-01 14:16:07 +08:00
LI Qing
64a980f529 Add time syscall and default localtime support 2021-03-26 16:56:57 +08:00
zongmin.gu
4b1b6158fb Workaround the server and server_epoll test case
In some cases, the main thread would be interrupted by
SIGCHID, so it needs retry more times or check the status.
2021-03-03 16:07:51 +08:00
zongmin.gu
0252f98d60 Enable exception test cases under simulation mode 2021-03-03 16:07:51 +08:00
LI Qing
be62e2c65d Add posix_fallocate 2021-03-03 11:06:09 +08:00
LI Qing
3dbf050904 Add sync_all, sync_data and poll for hostfs file 2021-03-02 13:47:02 +08:00
LI Qing
c3a02ffc28 Add support for the encrypted fs image 2021-03-01 16:45:01 +08:00
Hui, Chunyang
f0fa35b004 Add errno info when return error in make test 2021-02-24 14:47:29 +08:00
Hui, Chunyang
79b264a6c8 Add support for Address Space Layout Randomization
ASLR is only enabled on release mode. Also manage the whole process
VM with VMManager.
2021-02-24 14:46:41 +08:00
LI Qing
d7b994bc7d Fix getdents cannot output all entries in a directory 2021-02-24 13:24:54 +08:00
Hui, Chunyang
c4c3315c06 Fix false alarm on non-page-aligned size for mprotect 2021-02-23 20:59:17 +08:00
Hui, Chunyang
58bddcb89d Fix sysinfo test false alarm on uptime 2021-02-04 17:48:43 +08:00
He Sun
2ea7fc1ad6 Add Unix socket support for poll
A relay notifier that observes the underlying endpoint is added as the
notifier for the socket. It broadcasts to its observers when either end
of the channel has IoEvents.
2021-01-21 23:45:36 +08:00
LI Qing
d4e7c906c0 Fix the sched test when available cpu is less than online cpu 2021-01-20 16:00:33 +08:00
LI Qing
0b51d83811 Add ProcFS 2021-01-20 12:42:00 +08:00
LI Qing
d6cd89f03b Add DevFS for device files 2021-01-20 11:40:39 +08:00
He Sun
f711dcdf6d Use pthread_join to wait for the async thread to exit 2021-01-14 16:17:32 +08:00
zongmin.gu
6954808fd9 Enlarge the user_space_size as a temp solution to make the CI pass 2021-01-14 08:22:35 +08:00
He Sun
f1e5f574ca Add support for DCAP
1. Five new ioctl commands of /dev/sgx are added for occlum
applications to securely get and verify DCAP quote;
2. Not all the functions of the intel DCAP package are open to
developers to simplify the DCAP usage;
3. The test may only run on the platform with DCAP driver installed;
4. A macro OCCLUM_DISABLE_DCAP is used to separate the DCAP code from
the other code.
5. Skip DCAP test when DCAP driver is not detected or in simulation mode
2020-12-19 19:53:31 +08:00
LI Qing
22b02850a3 Add Glibc as an optional libc and fix test cases 2020-12-16 19:21:22 +08:00
He Sun
3b915db774 Refactor Unix socket
1. Implement type-safe functions;
2. Improve the correctness of nearly all the functions;
3. Improve the readability by introducing Listener and Endpoint for StreamUnix;
4. Substitue RingBuf with Channel in Unix socket.
2020-12-16 01:28:53 +08:00
zongmin.gu
a09c01819b Enlarge the kernel heap size in make test 2020-12-16 01:03:15 +08:00
He Sun
5285e3b55d Fix the insufficient output buffer in SIOCGIFCONF ioctl test 2020-12-02 14:56:14 +08:00
Tate, Hongliang Tian
1de089ac7d Rewrite the select syscall using the new poll implementation 2020-11-18 19:35:04 +08:00
LI Qing
9bf2a77e16 Add getdents syscall 2020-11-18 16:44:11 +08:00
Tate, Hongliang Tian
71df1cf2c8 Add the new poll implementation 2020-11-12 15:49:20 +08:00
He Sun
4260a8defc Fix two cpuid bugs
1. >> has higher precedence than &. Use parentheses to conduct & first;
2. In the latest Intel software developer's manual, cpuid leaf 06H EDX
is related to the logical processor.
2020-11-12 15:48:04 +08:00
Tate, Hongliang Tian
6fdfa57a14 Add the new epoll implementation
Before this commit, the epoll implementation works by simply delegating to the
host OS through OCall. One major problem with this implementation is
that it can only handle files that are backed by a file of the host OS
(e.g., sockets), but not those are are mainly implemented by the LibOS
(e.g., pipes). Therefore, a new epoll implementation that can handle all
kinds of files is needed.

This commit completely rewrites the epoll implementation by leveraging
the new event subsystem. Now the new epoll can handle all file types:

1. Host files, e.g., sockets, eventfd;
2. LibOS files, e.g., pipes;
3. Hybrid files, e.g., epoll files.

For a new file type to support epoll, it only neends to implement no
more than four methods of the File trait:

* poll (required for all file types);
* notifier (required for all file files);
* host_fd (only required for host files);
* recv_host_events (only required for host files).
2020-11-10 14:34:40 +08:00
He Sun
dd63f0194f Refine the data comparison in server_epoll test
Verify received data with the length of the data sent from client.
Otherwise, it may fail when there was dirty data in the buffer for
receiving.
2020-10-22 15:51:07 +08:00
LI Qing
28f47dacce Add ioctl support for FIONBIO command 2020-10-21 12:51:18 +08:00
Tate, Hongliang Tian
f5ae00895e Refactor pipe with the new event subsystem
1. Introduce channels, which provide an efficient means for IPC;
2. Leverage channels to rewrite pipe, improving the performance (3X),
robustness, and readability.

This pipe rewrite is not done: some more commits will be added to
implement poll and epoll for pipe.
2020-10-09 16:37:11 +08:00
Tate, Hongliang Tian
567e965eae Add sigtimedwait syscall 2020-09-29 18:10:30 +08:00
Hui, Chunyang
4031216f13 Add timerslack concept for libos and enbale prctl PR_GET_TIMERSLACK option 2020-09-18 12:39:12 +08:00
LI Qing
1a11655169 Fix two bugs related to open directories
1. Support O_DIRECTORY flag for open syscall
2. Disallow to open a directory in write mode
2020-09-18 12:39:12 +08:00
LI Qing
292fc28340 Add "*at()" system calls
The syscalls implemented in this submission are as follows:
fchmodat, fchownat, linkat, mkdirat, renameat, readlinkat, unlinkat
2020-09-18 12:39:12 +08:00
He Sun
e13242e7e5 Add netdevice ioctl command 2020-09-18 12:39:12 +08:00
LI Qing
70d7d10eeb Add support to mkdir & rmdir in hostfs 2020-08-20 17:03:30 +08:00
LI Qing
7d31cb743c Add support to read directory in hostfs 2020-08-20 17:03:30 +08:00
zongmin.gu
2ca5629b3d Save floating point registers in exception/interrupt flow 2020-08-15 19:12:40 +08:00
Hui, Chunyang
85501d8993 Improve implementation for occlum build
This commit mainly accomplish two things:
1. Use makefile to manage dependencies for `occlum build`, which can save lots of time
2. Take dirs `build`, `run` outside from `.occlum`. Remove env var "OCCLUM_INSTANCE_DIR"
2020-08-15 19:12:39 +08:00
LI Qing
3f6bcec1c5 Substitute ramFS with a temporary SEFS at "/tmp" 2020-08-15 19:12:39 +08:00
LI Qing
6d39587c40 Add getcpu syscall 2020-08-15 19:12:39 +08:00
Hui, Chunyang
32fa9fe4aa Fix cpuid test case to check SGX support accurately 2020-08-11 12:07:23 +08:00
LI Qing
572873d9a4 Enable UnionFS 2020-07-23 21:40:34 +08:00
Tate, Hongliang Tian
a6dbce21cc Add msync system call and flush updates to shared file memory mappings
1. Add msync-related methods to VMManager;
2. Add msync system call;
3. Fix munmap, fsync, and fdatasync system calls;
4. Add test cases.
2020-07-17 16:57:00 +00:00
He Sun
c85163ec0a Add notification mechanism for basic IO events
1. Add notification mechanism for select, poll, epoll and blocking IO
2. Add pipe support for select, poll and blocking IO
2020-07-18 00:27:26 +08:00
Hui, Chunyang
c43fbfea7f Fix signal testcase and a bug in sig_set 2020-07-17 18:02:09 +08:00
Tate, Hongliang Tian
a3ad465ce3 Fix SGX simulation mode broken by the interrupt subsystem
The new interrupt subsystem breaks the simulation mode in two ways:

1. The signal 64 is not handled by Intel SGX SDK in simulation mode. A
handled real-time signal crashes the process.

2. The newly-enabled test case exit_group depends on interrupts. But
enclave interrupts, like enclave exceptions, are not supported in
simulation mode.

This commit ensures signal 64 is ignored by default and exit_group test
case is not enabled in simulation mode.
2020-07-11 14:19:41 +00:00
Tate, Hongliang Tian
b585fce65d Add the interrupt subsystem
Before this commit, events like signals and exit_group are handled by
LibOS threads in a cooperative fashion: if the user code executed by a
LibOS thread does not invoke system calls (e.g., a busy loop), then the LibOS
won't have any opportunity to take control and handle events.

With the help from the POSIX signal-based interrupt mechanism of
Occlum's version of Intel SGX SDK, the LibOS can now interrupt the
execution of arbitrary user code in a LibOS thread by sending real-time
POSIX signals (the signal number is 64) to it. These signals are sent by
a helper thread spawn by Occlum PAL. The helper thread periodically
enters into the enclave to check if there are any LibOS threads with
pending events. If any, the helper thread broadcast POSIX signals to
them. When interrupted by a signal, the receiver LibOS thread may be in
one of the two previously problematic states in terms of event handling:

1. Executing non-cooperative user code (e.g., a busy loop). In this
case, the signal will trigger an interrupt handler inside the enclave,
which can then enter the LibOS kernel to deal with any pending events.

2. Executing an OCall that invokes blocking system calls (e.g., futex,
nanosleep, or blocking I/O). In this case, the signal will interrupt the
blocking system call so that the OCall can return back to the enclave.

Thanks to the new interrupt subsystem, some event-based system calls
are made robust. One such example is exit_group. We can now guarantee
that exit_group can force any thread in a process to exit.
2020-07-10 11:52:01 +00:00
LI Qing
686ec343b2 Fix the dirfd issue when path is absolute 2020-07-10 18:08:03 +08:00
LI Qing
1ad8f22170 Add support to handle symbolic link file 2020-07-10 18:08:03 +08:00
Hui, Chunyang
3cd46fd224 Add prctl support of PR_SET/GET_NAME options 2020-07-10 08:31:54 +00:00
Hui, Chunyang
406f30ec7a Polish build and install process
Remove redundent files and make processes for SGX simulation mode and hardware mode.
2020-07-08 11:51:33 +00:00
Tate, Hongliang Tian
bca0663972 Add mprotect system call 2020-06-19 07:32:31 +00:00
Hui, Chunyang
6e57937b45 Add support for sysinfo syscall 2020-06-13 03:33:32 +00:00
LI Qing
c8e3e0ffc6 Add clock_getres syscall 2020-06-13 03:33:32 +00:00
Tate, Hongliang Tian
70d3991ff5 Rewrite mremap system call
This rewrite serves three purposes:
1. Fix some subtle bugs in the old implementation;
2. Implement mremap using mmap and munmap so that mremap can automatically
enjoy new features (e.g., mprotect and memory permissions) once mmap and
munmap support the feature.
3. Write down the invariants hold by VMManager explictly so that the correctness
of the new implementation can be reason more easily.
2020-06-13 03:33:32 +00:00
Tate, Hongliang Tian
ffbae880f2 Reorder the config entries in Occlum.json
Not all config entries are created equal: some are more likely to be
customized by users, some are not so often. This commit reorders the
config entries in descending order of expected popularity.
2020-06-05 04:16:16 +00:00
zongmin.gu
942321363d Combine the enclave configuration into the occlum configuration file
Update the occlum.json to align with the gen_enclave_conf design.
Below is the two updated structures:
   "metadata": {
        "product_id": 0,
        "version_number": 0,
        "debuggable": true
    },
    "resource_limits": {
        "max_num_of_threads": 32,
        "kernel_space_heap_size": "32MB",
        "kernel_space_stack_size": "1MB",
        "user_space_size": "256MB"
    }
2020-06-05 11:03:47 +08:00
Tate, Hongliang Tian
655869711a Support page fault 2020-06-04 20:36:29 +08:00
He Sun
eb4bb860ce Add the unix socket support for sendto system call 2020-06-03 17:57:38 +08:00
Hui, Chunyang
ed664d1143 Fix sched set/get affinity wrong behaviour
1. Fix child process not inherit affinity from parent process
2. Add warning for buffer length gap with kernel when getaffinity
2020-06-02 17:36:36 +08:00
LI Qing
e6996f3c45 Add mremap system call 2020-05-29 03:38:49 +00:00
Hui, Chunyang
5b695c9539 Format c/c++ files in src, tools and test 2020-05-27 07:09:18 +00:00
Hui, Chunyang
03ba13aec7 Add "make format" and "make format-check" to check format for c/c++ files 2020-05-27 07:08:59 +00:00
He Sun
cd2f13ae54 Refactor select syscall
1. Substitute the underlying poll OCall to select OCall to update the
timeout argument correctly.
2. Add more checks for the inputs.
2020-05-26 11:44:46 +08:00
Hui, Chunyang
c14ee62678 Add untrusted environment variable override
Add "untrusted" sections for environment variables defined in Occlum.json. Environment
variable defined in "default" will be shown in libos directly. Environment variable
defined in "untrusted" can be passed from occlum run or PAL layer and can override
the value in "default" and thus is considered "untrusted".
2020-05-25 03:14:54 +00:00
Hui, Chunyang
bc6002f6dd Add counters for passes and failures in test
Also exit with error if the fail number is not 0.
2020-05-22 14:23:20 +08:00
Zongmin
17b4912055 Send SIGCHLD to parent process 2020-05-18 20:24:14 +08:00
He Sun
459ca45ab9 Fix reading extra message in neogotiate_msg 2020-05-15 21:49:26 +08:00
He Sun
a0b34c4450 Remove unintentionally added comments 2020-05-15 11:32:14 +00:00
Zongmin
0c3466f4ad Add three new occlum commands: start, exec and stop
Usage:
//start the occlum server
occlum start

//execute the command inside occlum
occlum exec [cmd] [-- <args>]

//stop the occlum server
occlum stop
2020-05-15 03:02:42 +00:00
LI Qing
2d34ee349e Add the sigaltstack system call 2020-05-15 03:02:42 +00:00
He Sun
aed572064b Fix potential address-in-use errors in epoll_server test
Wait for all the children to exit before server exits to release all the
resource used by the test.
2020-05-15 03:02:42 +00:00
Hui, Chunyang
25350b0e85 Fix sched agent failure for offline core 2020-05-15 02:59:16 +00:00
He Sun
7a87d77509 Fix error when PF_UNIX is passed as protocol to create unix socket
Zero and PF_UNIX are both supported as protocol when creating unix
socket.
2020-05-15 02:59:16 +00:00