Update README.md
Browse files
README.md
CHANGED
@@ -177,11 +177,6 @@ print(f"Peak Memory Usage: {mem:.02f} GB")
|
|
177 |
|
178 |
# Model Performance
|
179 |
|
180 |
-
Need to install vllm nightly to get some recent changes
|
181 |
-
```
|
182 |
-
pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
|
183 |
-
```
|
184 |
-
|
185 |
## Results (H100 machine)
|
186 |
| Benchmark | | |
|
187 |
|----------------------------------|----------------|--------------------------|
|
@@ -199,6 +194,11 @@ Download sharegpt dataset: `wget https://huggingface.co/datasets/anon8231489123/
|
|
199 |
Other datasets can be found in: https://github.com/vllm-project/vllm/tree/main/benchmarks
|
200 |
## benchmark_latency
|
201 |
|
|
|
|
|
|
|
|
|
|
|
202 |
Run the following under `vllm` source code root folder:
|
203 |
|
204 |
### baseline
|
|
|
177 |
|
178 |
# Model Performance
|
179 |
|
|
|
|
|
|
|
|
|
|
|
180 |
## Results (H100 machine)
|
181 |
| Benchmark | | |
|
182 |
|----------------------------------|----------------|--------------------------|
|
|
|
194 |
Other datasets can be found in: https://github.com/vllm-project/vllm/tree/main/benchmarks
|
195 |
## benchmark_latency
|
196 |
|
197 |
+
Need to install vllm nightly to get some recent changes
|
198 |
+
```
|
199 |
+
pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
|
200 |
+
```
|
201 |
+
|
202 |
Run the following under `vllm` source code root folder:
|
203 |
|
204 |
### baseline
|