Update README.md
Browse files
README.md
CHANGED
|
@@ -24,7 +24,7 @@ recipe = [
|
|
| 24 |
### vllm
|
| 25 |
Please load the model into vllm and sglang as float16 data type for AWQ support and use `tensor_parallel_size <= 2` i.e.,
|
| 26 |
```
|
| 27 |
-
vllm serve cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ --dtype float16 --tensor-parallel-size 2 --pipeline-parallel-size 2
|
| 28 |
```
|
| 29 |
# Qwen3-Coder-30B-A3B-Instruct
|
| 30 |
<a href="https://chat.qwen.ai/" target="_blank" style="margin: 2px;">
|
|
|
|
| 24 |
### vllm
|
| 25 |
Please load the model into vllm and sglang as float16 data type for AWQ support and use `tensor_parallel_size <= 2` i.e.,
|
| 26 |
```
|
| 27 |
+
vllm serve cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ-4bit --dtype float16 --tensor-parallel-size 2 --pipeline-parallel-size 2
|
| 28 |
```
|
| 29 |
# Qwen3-Coder-30B-A3B-Instruct
|
| 30 |
<a href="https://chat.qwen.ai/" target="_blank" style="margin: 2px;">
|