Please note: This model is not the official implementation from AutoAWQ. Instead, it is a Qwen3 implementation featuring scales assigned per expert.
To run this model, you will need to install and use the kishizaki-sci version of AutoAWQ in your environment.
git clone -b qwen3_moe https://github.com/kIshizaki-sci/AutoAWQ.git
pip install -e ./AutoAWQ
quantization : https://colab.research.google.com/drive/1FRWZBiWDYeNrVx6pOQEB24tJK8EGkZoR?usp=sharing
inference : https://colab.research.google.com/drive/1yN21HjT7v695voETVYa6qYsraeNW-8Cy?usp=sharing
- Downloads last month
- 25
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support