Skip to content

Commit 9bff3a7

Browse files
authored
docs: update ascend docs for docker running (#3266)
* docs: update ascend docs for docker running * ci: fix mdformat linting
1 parent 9958b89 commit 9bff3a7

File tree

2 files changed

+30
-2
lines changed

2 files changed

+30
-2
lines changed

docs/en/get_started/ascend/get_started.md

+15-1
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,13 @@ Add `--device ascend` in the serve command.
109109
lmdeploy serve api_server --backend pytorch --device ascend --eager-mode internlm/internlm2_5-7b-chat
110110
```
111111

112+
Run the following commands to launch docker container for lmdeploy LLM serving:
113+
114+
```bash
115+
docker exec -it --net=host crpi-4crprmm5baj1v8iv.cn-hangzhou.personal.cr.aliyuncs.com/lmdeploy_dlinfer/ascend:latest \
116+
    bash -i -c "lmdeploy serve api_server --backend pytorch --device ascend --eager-mode internlm/internlm2_5-7b-chat"
117+
```
118+
112119
### Serve a VLM model
113120

114121
Add `--device ascend` in the serve command
@@ -117,6 +124,13 @@ Add `--device ascend` in the serve command
117124
lmdeploy serve api_server --backend pytorch --device ascend --eager-mode OpenGVLab/InternVL2-2B
118125
```
119126

127+
Run the following commands to launch docker container for lmdeploy VLM serving:
128+
129+
```bash
130+
docker exec -it --net=host crpi-4crprmm5baj1v8iv.cn-hangzhou.personal.cr.aliyuncs.com/lmdeploy_dlinfer/ascend:latest \
131+
    bash -i -c "lmdeploy serve api_server --backend pytorch --device ascend --eager-mode OpenGVLab/InternVL2-2B"
132+
```
133+
120134
## Inference with Command line Interface
121135

122136
Add `--device ascend` in the serve command.
@@ -128,7 +142,7 @@ lmdeploy chat internlm/internlm2_5-7b-chat --backend pytorch --device ascend --e
128142
Run the following commands to launch lmdeploy chatting after starting container:
129143

130144
```bash
131-
docker exec -it lmdeploy_ascend_demo \
145+
docker exec -it crpi-4crprmm5baj1v8iv.cn-hangzhou.personal.cr.aliyuncs.com/lmdeploy_dlinfer/ascend:latest \
132146
    bash -i -c "lmdeploy chat --backend pytorch --device ascend --eager-mode internlm/internlm2_5-7b-chat"
133147
```
134148

docs/zh_cn/get_started/ascend/get_started.md

+15-1
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,13 @@ if __name__ == "__main__":
105105
lmdeploy serve api_server --backend pytorch --device ascend --eager-mode internlm/internlm2_5-7b-chat
106106
```
107107

108+
也可以运行以下命令启动容器运行LLM模型服务。
109+
110+
```bash
111+
docker exec -it --net=host crpi-4crprmm5baj1v8iv.cn-hangzhou.personal.cr.aliyuncs.com/lmdeploy_dlinfer/ascend:latest \
112+
    bash -i -c "lmdeploy serve api_server --backend pytorch --device ascend --eager-mode internlm/internlm2_5-7b-chat"
113+
```
114+
108115
### VLM 模型服务
109116

110117
`--device ascend`加入到服务启动命令中。
@@ -113,6 +120,13 @@ lmdeploy serve api_server --backend pytorch --device ascend --eager-mode internl
113120
lmdeploy serve api_server --backend pytorch --device ascend --eager-mode OpenGVLab/InternVL2-2B
114121
```
115122

123+
也可以运行以下命令启动容器运行VLM模型服务。
124+
125+
```bash
126+
docker exec -it --net=host crpi-4crprmm5baj1v8iv.cn-hangzhou.personal.cr.aliyuncs.com/lmdeploy_dlinfer/ascend:latest \
127+
    bash -i -c "lmdeploy serve api_server --backend pytorch --device ascend --eager-mode OpenGVLab/InternVL2-2B"
128+
```
129+
116130
## 使用命令行与LLM模型对话
117131

118132
`--device ascend`加入到服务启动命令中。
@@ -124,7 +138,7 @@ lmdeploy chat internlm/internlm2_5-7b-chat --backend pytorch --device ascend --e
124138
也可以运行以下命令使启动容器后开启lmdeploy聊天
125139

126140
```bash
127-
docker exec -it lmdeploy_ascend_demo \
141+
docker exec -it crpi-4crprmm5baj1v8iv.cn-hangzhou.personal.cr.aliyuncs.com/lmdeploy_dlinfer/ascend:latest \
128142
    bash -i -c "lmdeploy chat --backend pytorch --device ascend --eager-mode internlm/internlm2_5-7b-chat"
129143
```
130144

0 commit comments

Comments
 (0)