general/occlum
4
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[demos] Update readme of demo llm

Browse Source
This commit is contained in:
Qi Zheng 2023-10-18 11:28:39 +08:00 committed by volcano
parent 34c1ff7922
commit 192a1a2f02
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
demos/bigdl-llm/README.md
View File

@ -76,9 +76,9 @@ In order to load models using BigDL-LLM, the model name should include "bigdl".
To serve using the Web UI, you need three main components: web servers that interface with users, model workers that host one or more models, and a controller to coordinate the web server and model workers. To serve using the Web UI, you need three main components: web servers that interface with users, model workers that host one or more models, and a controller to coordinate the web server and model workers.
#### Launch the Controller #### Launch the Controller in non-TEE env
```bash ```bash
./python-occlum/bin/python -m fastchat.serve.controller ./python-occlum/bin/python -m fastchat.serve.controller --host 0.0.0.0
``` ```
This controller manages the distributed workers. This controller manages the distributed workers.