Update README.md
Browse files
README.md
CHANGED
@@ -57,7 +57,7 @@ if __name__ == '__main__':
|
|
57 |
print("-" * 60)
|
58 |
```
|
59 |
|
60 |
-
Note: please use `VLLM_DISABLE_COMPILE_CACHE=1` when running this code, e.g. `VLLM_DISABLE_COMPILE_CACHE=1 python example.py`, since there are some issues with the composability of compile in vLLM and torchao,
|
61 |
this is expected be resolved in pytorch 2.8.
|
62 |
|
63 |
## Serving
|
|
|
57 |
print("-" * 60)
|
58 |
```
|
59 |
|
60 |
+
Note: please use `VLLM_DISABLE_COMPILE_CACHE=1` to disable compile cache when running this code, e.g. `VLLM_DISABLE_COMPILE_CACHE=1 python example.py`, since there are some issues with the composability of compile in vLLM and torchao,
|
61 |
this is expected be resolved in pytorch 2.8.
|
62 |
|
63 |
## Serving
|