MOSS-Image-SFT · Nunchaku Lite INT4 r128

Quantized from OpenMOSS-Team/MOSS-Image-SFT. The transformer uses INT4 W4A4 SVDQuant, rank 128, with GPTQ. The bundled text encoder uses serialized BitsAndBytes NF4, BF16 compute and no double quantization. The original FP32 VAE, tokenizer and scheduler are included.

Usage

Requires the Nunchaku Lite-capable Diffusers build, Transformers 5.19.0, BitsAndBytes 0.50.2 and the compatible kernels package. The bundled custom pipeline supports packed linear layers.

import os
os.environ["DIFFUSERS_TRUST_REMOTE_KERNELS"] = "true"
import torch
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(
    "lite-infer/moss-image-sft-nunchaku-lite-int4-r128",
    custom_pipeline="pipeline_moss_image",
    trust_remote_code=True,
    torch_dtype={"default": torch.bfloat16, "vae": torch.float32},
).to("cuda")

image = pipe(
    "A watercolor painting of a lighthouse on a rocky island at sunrise",
    width=1024,
    height=1024,
    num_inference_steps=50,
    guidance_scale=3.0,
    negative_prompt="",
    uncond_skip_layers=[22],
    slg_scale=0.0,
    generator=torch.Generator("cpu").manual_seed(42),
).images[0]
image.save("output.png")

The scheduler uses shift 5.0. For lower resident GPU memory, use pipe.enable_model_cpu_offload() in place of .to("cuda").

General-case comparison

The examples below are the first six prompts in the 32-case general set, in their original order. Both columns use matching prompts, seeds, 1024×1024 resolution, 50 steps, CFG 3 and shift 5. The negative prompt is empty, block index 22 is skipped in the negative branch, and additional SLG is disabled. BF16 uses the original transformer and original encoder; INT4 r128 uses the quantized transformer and bundled NF4 encoder. The comparison images were generated with model CPU offload and the FP32 VAE.

General case BF16 reference INT4 r128 + NF4
0000 bf16 r128
0001 bf16 r128
0002 bf16 r128
0003 bf16 r128
0004 bf16 r128
0005 bf16 r128
Final pipeline General cases LPIPS ↓ PSNR ↑ (dB)
INT4 r128 + NF4 32 0.2993 17.78

Mean per-image, full-resolution AlexNet LPIPS v0.1 and RGB PSNR against the saved BF16 references. These measure similarity to those reference outputs. Fixed seeds do not guarantee identical pixels across reruns. See general-case settings and metric results.

Example prompts and seeds

0000 · seed 42 · 1024×1024

A candid portrait of a woman with silver hair in a blue wool sweater beside a large window, soft morning light, realistic skin texture and natural colors.

0001 · seed 43 · 1024×1024

Product photograph of a transparent amber perfume bottle with a brushed gold cap on pale limestone, a sprig of lavender beside it, soft side lighting, no text.

0002 · seed 44 · 1024×1024

A red fox curled up asleep beneath a snow-covered pine tree, fine fur, winter morning, wildlife photography.

0003 · seed 45 · 1024×1024

A ceramic bowl of ramen with an egg, mushrooms and scallions on a dark wooden counter, warm steam, food photography.

0004 · seed 46 · 1024×1024

A narrow street in an old Mediterranean town with pale walls, climbing flowers and blue doors, afternoon sunlight.

0005 · seed 47 · 1024×1024

A watercolor painting of a small lighthouse on a rocky island beneath a cloudy lavender sky.

Peak VRAM

Final pipeline Peak PyTorch allocated VRAM
INT4 r128 + NF4 12.00 GiB

Measured at 1024×1024, 50 steps and CFG 3, with all pipeline components resident on GPU and no CPU offload. See measurement settings.

Quantization and provenance

  • Source revision: f972bd4b4b9e3852133cc5da954d87bc32cd8fb9.
  • 262 SVDQ W4A4 linear layers, group size 64, rank 128; two AWQ INT4 modulation layers.
  • Eight original qdiff calibration prompts at 1024×1024, 50 steps and CFG 3; all 800 guidance calls retained.
  • 32,768 sampled activation rows per target, processed in chunks of 4,096.
  • GPTQ: damping 0.01, block size 128, Hessian batch 512, up to 250 inversion retries.
  • Randomized SVD: oversampling 10, four iterations, seed 0.

See quantization metadata, modifications and notice. The source model and custom pipeline are distributed under Apache License 2.0.

Downloads last month
20
Safetensors
Model size
3B params
Tensor type
F32
·
I32
·
BF16
·
I8
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for lite-infer/moss-image-sft-nunchaku-lite-int4-r128

Quantized
(55)
this model

Collection including lite-infer/moss-image-sft-nunchaku-lite-int4-r128