|
|
--- |
|
|
license: apache-2.0 |
|
|
base_model: |
|
|
- google/madlad400-3b-mt |
|
|
pipeline_tag: translation |
|
|
tags: |
|
|
- text2text-generation |
|
|
- text-generation-inference |
|
|
--- |
|
|
|
|
|
## ONNX version of [google/madlad400-3b-mt](https://huggingface.co/google/madlad400-3b-mt) |
|
|
|
|
|
## Converted and quantized with [optimum-cli](https://github.com/huggingface/optimum) |
|
|
|
|
|
- Convert to ONNX: |
|
|
```sh |
|
|
optimum-cli onnxruntime export --model google/madlad400-3b-mt <output_path> --legacy |
|
|
``` |
|
|
|
|
|
- Quantization: |
|
|
```sh |
|
|
optimum-cli onnxruntime quantize --onnx_model <input_model_path> -o <output_model_path> --avx512_vnni |
|
|
``` |
|
|
|
