---
license: apache-2.0
base_model:
- THUDM/CogView4-6B
base_model_relation: quantized
tags:
- quanto
---
## Quantization settings

- `vae.`: `torch.bfloat16`. No quantization.
- `text_encoder.layers.`:
  - Int8 with [Optimum Quanto](https://github.com/huggingface/optimum-quanto)
  - Target layers:`["q_proj", "k_proj", "v_proj", "o_proj", "mlp.down_proj", "mlp.gate_up_proj"]`
- `diffusion_model.`:
  - Int8 with [Optimum Quanto](https://github.com/huggingface/optimum-quanto)
  - Target layers: `["to_q", "to_k", "to_v", "to_out.0", "ff.net.0.proj", "ff.net.2"]`

## VRAM cosumption

- Text encoder (`text_encoder.`): about 11 GB
- Denoiser (`diffusion_model.`): about 10 GB

## Samples


|`torch.bfloat16` | Quanto Int8 |
| - | - |
| <img src="./images/sample_bf16_01.jpg" width="320px" /> | <img src="./images/sample_quanto_01.jpg" width="320px" /> |
| VRAM 40GB (without offloading) | VRAM 28GB (without offloading) |


<details><summary>Generation parameters</summary>

- prompt: `""" A photo of a nendoroid figure of hatsune miku holding a sign that says "CogView4" """"`
- negative_prompt: `"blurry, low quality, horror"`
- height: `1152`
- width: `1152`
- cfg_scale: `3.5`
- num_inference_steps: `20`
</details>