Instructions to use lite-infer/moss-image-sft-nunchaku-lite-int4-r128 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use lite-infer/moss-image-sft-nunchaku-lite-int4-r128 with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("lite-infer/moss-image-sft-nunchaku-lite-int4-r128", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
import torch
from diffusers import DiffusionPipeline
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("lite-infer/moss-image-sft-nunchaku-lite-int4-r128", dtype=torch.bfloat16, device_map="cuda")
prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]MOSS-Image-SFT · Nunchaku Lite INT4 r128
Quantized from OpenMOSS-Team/MOSS-Image-SFT. The transformer uses INT4 W4A4 SVDQuant, rank 128, with GPTQ. The bundled text encoder uses serialized BitsAndBytes NF4, BF16 compute and no double quantization. The original FP32 VAE, tokenizer and scheduler are included.
Usage
Requires the Nunchaku Lite-capable Diffusers build,
Transformers 5.19.0, BitsAndBytes 0.50.2 and the compatible kernels package.
The bundled custom pipeline supports packed linear layers.
import os
os.environ["DIFFUSERS_TRUST_REMOTE_KERNELS"] = "true"
import torch
from diffusers import DiffusionPipeline
pipe = DiffusionPipeline.from_pretrained(
"lite-infer/moss-image-sft-nunchaku-lite-int4-r128",
custom_pipeline="pipeline_moss_image",
trust_remote_code=True,
torch_dtype={"default": torch.bfloat16, "vae": torch.float32},
).to("cuda")
image = pipe(
"A watercolor painting of a lighthouse on a rocky island at sunrise",
width=1024,
height=1024,
num_inference_steps=50,
guidance_scale=3.0,
negative_prompt="",
uncond_skip_layers=[22],
slg_scale=0.0,
generator=torch.Generator("cpu").manual_seed(42),
).images[0]
image.save("output.png")
The scheduler uses shift 5.0. For lower resident GPU memory, use pipe.enable_model_cpu_offload()
in place of .to("cuda").
General-case comparison
The examples below are the first six prompts in the 32-case general set, in their original order. Both columns use matching prompts, seeds, 1024×1024 resolution, 50 steps, CFG 3 and shift 5. The negative prompt is empty, block index 22 is skipped in the negative branch, and additional SLG is disabled. BF16 uses the original transformer and original encoder; INT4 r128 uses the quantized transformer and bundled NF4 encoder. The comparison images were generated with model CPU offload and the FP32 VAE.
| Final pipeline | General cases | LPIPS ↓ | PSNR ↑ (dB) |
|---|---|---|---|
| INT4 r128 + NF4 | 32 | 0.2993 | 17.78 |
Mean per-image, full-resolution AlexNet LPIPS v0.1 and RGB PSNR against the saved BF16 references. These measure similarity to those reference outputs. Fixed seeds do not guarantee identical pixels across reruns. See general-case settings and metric results.
Example prompts and seeds
0000 · seed 42 · 1024×1024
A candid portrait of a woman with silver hair in a blue wool sweater beside a large window, soft morning light, realistic skin texture and natural colors.
0001 · seed 43 · 1024×1024
Product photograph of a transparent amber perfume bottle with a brushed gold cap on pale limestone, a sprig of lavender beside it, soft side lighting, no text.
0002 · seed 44 · 1024×1024
A red fox curled up asleep beneath a snow-covered pine tree, fine fur, winter morning, wildlife photography.
0003 · seed 45 · 1024×1024
A ceramic bowl of ramen with an egg, mushrooms and scallions on a dark wooden counter, warm steam, food photography.
0004 · seed 46 · 1024×1024
A narrow street in an old Mediterranean town with pale walls, climbing flowers and blue doors, afternoon sunlight.
0005 · seed 47 · 1024×1024
A watercolor painting of a small lighthouse on a rocky island beneath a cloudy lavender sky.
Peak VRAM
| Final pipeline | Peak PyTorch allocated VRAM |
|---|---|
| INT4 r128 + NF4 | 12.00 GiB |
Measured at 1024×1024, 50 steps and CFG 3, with all pipeline components resident on GPU and no CPU offload. See measurement settings.
Quantization and provenance
- Source revision:
f972bd4b4b9e3852133cc5da954d87bc32cd8fb9. - 262 SVDQ W4A4 linear layers, group size 64, rank 128; two AWQ INT4 modulation layers.
- Eight original qdiff calibration prompts at 1024×1024, 50 steps and CFG 3; all 800 guidance calls retained.
- 32,768 sampled activation rows per target, processed in chunks of 4,096.
- GPTQ: damping 0.01, block size 128, Hessian batch 512, up to 250 inversion retries.
- Randomized SVD: oversampling 10, four iterations, seed 0.
See quantization metadata, modifications and notice. The source model and custom pipeline are distributed under Apache License 2.0.
- Downloads last month
- 20
Model tree for lite-infer/moss-image-sft-nunchaku-lite-int4-r128
Base model
OpenMOSS-Team/MOSS-Image-Base










