Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -48,13 +48,20 @@ tags:
|
|
48 |
- Run as LlamaEdge service
|
49 |
|
50 |
```bash
|
51 |
-
wasmedge --dir .:. --nn-preload default:GGML:AUTO:Qwen2-7B-Instruct-Q5_K_M.gguf
|
|
|
|
|
|
|
|
|
52 |
```
|
53 |
|
54 |
- Run as LlamaEdge command app
|
55 |
|
56 |
```bash
|
57 |
-
wasmedge --dir .:. --nn-preload default:GGML:AUTO:Qwen2-7B-Instruct-Q5_K_M.gguf
|
|
|
|
|
|
|
58 |
```
|
59 |
|
60 |
## Quantized GGUF Models
|
|
|
48 |
- Run as LlamaEdge service
|
49 |
|
50 |
```bash
|
51 |
+
wasmedge --dir .:. --nn-preload default:GGML:AUTO:Qwen2-7B-Instruct-Q5_K_M.gguf \
|
52 |
+
llama-api-server.wasm \
|
53 |
+
--model-name Qwen2-7B-Instruct \
|
54 |
+
--prompt-template chatml \
|
55 |
+
--ctx-size 131072
|
56 |
```
|
57 |
|
58 |
- Run as LlamaEdge command app
|
59 |
|
60 |
```bash
|
61 |
+
wasmedge --dir .:. --nn-preload default:GGML:AUTO:Qwen2-7B-Instruct-Q5_K_M.gguf \
|
62 |
+
llama-chat.wasm \
|
63 |
+
--prompt-template chatml \
|
64 |
+
--ctx-size 131072
|
65 |
```
|
66 |
|
67 |
## Quantized GGUF Models
|