Update README.md
Browse files
README.md
CHANGED
@@ -85,6 +85,9 @@ vllm serve pytorch/Phi-4-mini-instruct-float8dq --tokenizer microsoft/Phi-4-mini
|
|
85 |
|
86 |
# Model Quality
|
87 |
We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the quality of the quantized model.
|
|
|
|
|
|
|
88 |
|
89 |
## baseline
|
90 |
```
|
@@ -120,6 +123,11 @@ lm_eval --model hf --model_args pretrained=pytorch/Phi-4-mini-instruct-float8dq
|
|
120 |
|
121 |
# Model Performance
|
122 |
|
|
|
|
|
|
|
|
|
|
|
123 |
## Results (H100 machine)
|
124 |
| Benchmark | | |
|
125 |
|----------------------------------|----------------|--------------------------|
|
|
|
85 |
|
86 |
# Model Quality
|
87 |
We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the quality of the quantized model.
|
88 |
+
Need to install lm-eval from source:
|
89 |
+
https://github.com/EleutherAI/lm-evaluation-harness#install
|
90 |
+
|
91 |
|
92 |
## baseline
|
93 |
```
|
|
|
123 |
|
124 |
# Model Performance
|
125 |
|
126 |
+
Need to install vllm nightly to get some recent changes
|
127 |
+
```
|
128 |
+
pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
|
129 |
+
```
|
130 |
+
|
131 |
## Results (H100 machine)
|
132 |
| Benchmark | | |
|
133 |
|----------------------------------|----------------|--------------------------|
|