linzhao-amd commited on
Commit
df6afb4
·
verified ·
1 Parent(s): a94dc97

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -1
README.md CHANGED
@@ -129,9 +129,17 @@ for i in $(seq 1 10); do
129
  2>&1 | tee -a "$LOG"
130
  ```
131
 
132
- The result of GSM8K was obtained using [lm-eval-harness](https://github.com/EleutherAI/lm-evaluation-harness) and the following commands.
133
 
134
  ```
 
 
 
 
 
 
 
 
135
  MODEL_ARGS="model=amd/DeepSeek-R1-0528-MXFP4-ASQ,base_url=http://localhost:8000/v1/completions,num_concurrent=999999,timeout=999999,tokenized_requests=False,max_length=38768,temperature=0.6,top_p=0.95,add_bos_token=True,seed=$SEED"
136
  lm_eval \
137
  --model local-completions \
 
129
  2>&1 | tee -a "$LOG"
130
  ```
131
 
132
+ The result of GSM8K was obtained using [lm-eval-harness](https://github.com/EleutherAI/lm-evaluation-harness) and [SGLang](https://docs.sglang.ai/), running with [docker](https://hub.docker.com/layers/lmsysorg/sglang/v0.5.3.post3-rocm700-mi35x-srt/images/sha256-8c7281fcd4adc7942c7e674d464fee322d1775d7b546596ab4cc7edd258517fc).
133
 
134
  ```
135
+ # Launching server
136
+ SGLANG_USE_AITER=1 python -m sglang.launch_server \
137
+ --model-path $MODEL_DIR \
138
+ --tp 8 \
139
+ --port 8000 \
140
+ --attention-backend aiter
141
+
142
+ #
143
  MODEL_ARGS="model=amd/DeepSeek-R1-0528-MXFP4-ASQ,base_url=http://localhost:8000/v1/completions,num_concurrent=999999,timeout=999999,tokenized_requests=False,max_length=38768,temperature=0.6,top_p=0.95,add_bos_token=True,seed=$SEED"
144
  lm_eval \
145
  --model local-completions \