TheBloke

/

WizardCoder-Python-34B-V1.0-GPTQ

like 61

Here: https://github.com/TheBlokeAI/AIScripts/blob/main/quant_autogptq.py

@TheBloke thank u so much, Did u runned data calibration on quantize with Wizardcode? Which data used here?

If my model trained with Chinese dataset, which data should I use for calibration?

For coding models I use Evol Instruct Code dataset - you will see code in the script I linked for using this dataset. Just pass --dataset code. Check my "Provided Files" table in the README and you will see it mentions which dataset is used and links to it.

Yes, for a Chinese model I would recommend using a Chinese dataset for calibration.

I don't think it makes a big difference to quantize quality, but it does make some difference. So it is worth using a dataset that is somewhat suitable for your model and its expected use.

@TheBloke what kind of calibration data is suitable for quantize model.

is that SFT data ok? how many does it need to calibrate good