Link is broken
Please refer to this chart for a basic overview of quantization types.
but https://github.com/ggerganov/llama.cpp/blob/master/examples/perplexity/README.md#llama-3-8b-scoreboard refers to 404 error page. I am currently trying to quantize this original model to f32, upload to hf and inform users how they can merge in into one file to use in comfyui. But llama.cpp/convert_hf_to_gguf.py --outtype f32 --outfile Wan2.1-I2V-14B-720P-f32.gguf models--Wan-AI--Wan2.1-I2V-14B-720P/snapshots/8823af45fcc58a8aa999a54b04be9abc7d2aac98/ gives me KeyError: 'architectures' so i wanted to read documentation. Unfortunely, your (this hf) repo contains no information about commands you used.
Looks like this is what I need: https://www.reddit.com/r/StableDiffusion/comments/1gxcivc/do_your_own_gguf_script/
now I see ComfyUI-GGUF/tools/convert.py --help
Generate F16 GGUF files from single UNET
it can only f16? Seems I need to see the code of this python script.
because it is better to have bf16 or f32 as base gguf and only then quantize
I see that it uses bf16 if original type is bfloat16 data_qtype = gguf.GGMLQuantizationType.BF16. But what if it is f32? have to figure out
for key, data in tqdm(state_dict.items()): it processes every key individualy. I think i just need to uncomment this:
# elif old_dtype == torch.float32:
# data_qtype = gguf.GGMLQuantizationType.F32
hm python ComfyUI-GGUF/tools/convert.py --src models--Wan-AI--Wan2.1-I2V-14B-720P/snapshots/8823af45fcc58a8aa999a54b04be9abc7d2aac98/diffusion_pytorch_model-00001-of-00007.safetensors --dst Wan2.1-I2V-14B-720P-f32.ggufgives AssertionError: Unknown model architecture!
Maybe I need to set arch to wan manualy?
I am currently reading the issues here https://github.com/city96/ComfyUI-GGUF/issues to understand what I have to do.
https://github.com/city96/ComfyUI-GGUF/pull/274/commits/a17aa52e0c2527e9cd9d7f95a3058925171b015c aready has an option to keep stock data type. I will try that.
I found that gguf I want, there are i2v, flf2v and t2v there already: https://huggingface.co/calcuis/wan-gguf