Hugging Face's logo

DFloat11
/
OmniGen2-mllm-DF11

Any-to-Any
Safetensors
qwen2_5_vl
dfloat11
df11
lossless compression
70% size, 100% accuracy
Model card Files
xet
Community
LeanQuant commited on
Commit
f830167
·
verified ·
1 Parent(s): a189419

Add files using upload-large-folder tool

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +43 -0
  2. config.json +96 -0
  3. lm_head.safetensors +3 -0
  4. model.safetensors +3 -0
  5. model_embed_tokens.safetensors +3 -0
  6. model_layers_0.safetensors +3 -0
  7. model_layers_1.safetensors +3 -0
  8. model_layers_10.safetensors +3 -0
  9. model_layers_11.safetensors +3 -0
  10. model_layers_12.safetensors +3 -0
  11. model_layers_13.safetensors +3 -0
  12. model_layers_14.safetensors +3 -0
  13. model_layers_15.safetensors +3 -0
  14. model_layers_16.safetensors +3 -0
  15. model_layers_17.safetensors +3 -0
  16. model_layers_18.safetensors +3 -0
  17. model_layers_19.safetensors +3 -0
  18. model_layers_2.safetensors +3 -0
  19. model_layers_20.safetensors +3 -0
  20. model_layers_21.safetensors +3 -0
  21. model_layers_22.safetensors +3 -0
  22. model_layers_23.safetensors +3 -0
  23. model_layers_24.safetensors +3 -0
  24. model_layers_25.safetensors +3 -0
  25. model_layers_26.safetensors +3 -0
  26. model_layers_27.safetensors +3 -0
  27. model_layers_28.safetensors +3 -0
  28. model_layers_29.safetensors +3 -0
  29. model_layers_3.safetensors +3 -0
  30. model_layers_30.safetensors +3 -0
  31. model_layers_31.safetensors +3 -0
  32. model_layers_32.safetensors +3 -0
  33. model_layers_33.safetensors +3 -0
  34. model_layers_34.safetensors +3 -0
  35. model_layers_35.safetensors +3 -0
  36. model_layers_4.safetensors +3 -0
  37. model_layers_5.safetensors +3 -0
  38. model_layers_6.safetensors +3 -0
  39. model_layers_7.safetensors +3 -0
  40. model_layers_8.safetensors +3 -0
  41. model_layers_9.safetensors +3 -0
  42. visual_blocks_0.safetensors +3 -0
  43. visual_blocks_1.safetensors +3 -0
  44. visual_blocks_10.safetensors +3 -0
  45. visual_blocks_11.safetensors +3 -0
  46. visual_blocks_12.safetensors +3 -0
  47. visual_blocks_13.safetensors +3 -0
  48. visual_blocks_14.safetensors +3 -0
  49. visual_blocks_15.safetensors +3 -0
  50. visual_blocks_16.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - OmniGen2/OmniGen2
4
+ base_model_relation: quantized
5
+ pipeline_tag: any-to-any
6
+ tags:
7
+ - dfloat11
8
+ - df11
9
+ - lossless compression
10
+ - 70% size, 100% accuracy
11
+ ---
12
+
13
+ # DFloat11 Compressed Model: `OmniGen2/OmniGen2` MLLM
14
+
15
+ This is a **DFloat11 losslessly compressed** version of the original `OmniGen2/OmniGen2` model. It reduces model size by **32%** compared to the original BFloat16 model, while maintaining **bit-identical outputs** and supporting **efficient GPU inference**.
16
+
17
+ ### 📊 Performance Comparison
18
+
19
+ | Metric | OmniGen2 (BFloat16) | OmniGen2 (DFloat11) |
20
+ | ----------------------------------------------- | ------------------- | ------------------- |
21
+ | Model Size | 16.23 GB | 11.11 GB |
22
+ | Peak GPU Memory<br>(1024×1024 image generation) | 18.41 GB | 14.36 GB |
23
+ | Generation Time<br>(A100 GPU) | 25 seconds | 27 seconds |
24
+
25
+ ### 🔧 How to Use
26
+
27
+ A complete usage guide is available in our GitHub repository (forked from the official OmniGen2 repository).
28
+
29
+ 👉 [https://github.com/LeanModels/OmniGen2-DFloat11](https://github.com/LeanModels/OmniGen2-DFloat11) 👈
30
+
31
+ ### 🔍 How It Works
32
+
33
+ We apply **Huffman coding** to losslessly compress the exponent bits of BFloat16 model weights, which are highly compressible (their 8 bits carry only ~2.6 bits of actual information). To enable fast inference, we implement a highly efficient CUDA kernel that performs on-the-fly weight decompression directly on the GPU.
34
+
35
+ The result is a model that is **~32% smaller**, delivers **bit-identical outputs**, and achieves performance **comparable to the original** BFloat16 model.
36
+
37
+ Learn more in our [research paper](https://arxiv.org/abs/2504.11651).
38
+
39
+ ### 📄 Learn More
40
+
41
+ * **Paper**: [70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float](https://arxiv.org/abs/2504.11651)
42
+ * **GitHub**: [https://github.com/LeanModels/DFloat11](https://github.com/LeanModels/DFloat11)
43
+ * **HuggingFace**: [https://huggingface.co/DFloat11](https://huggingface.co/DFloat11)
config.json ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen2_5_VLForConditionalGeneration"
4
+ ],
5
+ "attention_dropout": 0.0,
6
+ "bos_token_id": 151643,
7
+ "dfloat11_config": {
8
+ "bytes_per_thread": 8,
9
+ "pattern_dict": {
10
+ "lm_head": [],
11
+ "model\\.embed_tokens": [],
12
+ "model\\.layers\\.\\d+": [
13
+ "self_attn.q_proj",
14
+ "self_attn.k_proj",
15
+ "self_attn.v_proj",
16
+ "self_attn.o_proj",
17
+ "mlp.gate_proj",
18
+ "mlp.up_proj",
19
+ "mlp.down_proj"
20
+ ],
21
+ "visual\\.blocks\\.\\d+": [
22
+ "attn.qkv",
23
+ "attn.proj",
24
+ "mlp.gate_proj",
25
+ "mlp.up_proj",
26
+ "mlp.down_proj"
27
+ ],
28
+ "visual\\.merger\\.mlp": [
29
+ "0",
30
+ "2"
31
+ ]
32
+ },
33
+ "threads_per_block": [
34
+ 512
35
+ ],
36
+ "version": "0.2.0"
37
+ },
38
+ "eos_token_id": 151645,
39
+ "hidden_act": "silu",
40
+ "hidden_size": 2048,
41
+ "image_token_id": 151655,
42
+ "initializer_range": 0.02,
43
+ "intermediate_size": 11008,
44
+ "max_position_embeddings": 128000,
45
+ "max_window_layers": 70,
46
+ "model_type": "qwen2_5_vl",
47
+ "num_attention_heads": 16,
48
+ "num_hidden_layers": 36,
49
+ "num_key_value_heads": 2,
50
+ "rms_norm_eps": 1e-06,
51
+ "rope_scaling": {
52
+ "mrope_section": [
53
+ 16,
54
+ 24,
55
+ 24
56
+ ],
57
+ "rope_type": "default",
58
+ "type": "default"
59
+ },
60
+ "rope_theta": 1000000.0,
61
+ "sliding_window": 32768,
62
+ "tie_word_embeddings": true,
63
+ "torch_dtype": "bfloat16",
64
+ "transformers_version": "4.51.3",
65
+ "use_cache": true,
66
+ "use_sliding_window": false,
67
+ "video_token_id": 151656,
68
+ "vision_config": {
69
+ "depth": 32,
70
+ "fullatt_block_indexes": [
71
+ 7,
72
+ 15,
73
+ 23,
74
+ 31
75
+ ],
76
+ "hidden_act": "silu",
77
+ "hidden_size": 1280,
78
+ "in_channels": 3,
79
+ "in_chans": 3,
80
+ "intermediate_size": 3420,
81
+ "model_type": "qwen2_5_vl",
82
+ "num_heads": 16,
83
+ "out_hidden_size": 2048,
84
+ "patch_size": 14,
85
+ "spatial_merge_size": 2,
86
+ "spatial_patch_size": 14,
87
+ "temporal_patch_size": 2,
88
+ "tokens_per_second": 2,
89
+ "torch_dtype": "bfloat16",
90
+ "window_size": 112
91
+ },
92
+ "vision_end_token_id": 151653,
93
+ "vision_start_token_id": 151652,
94
+ "vision_token_id": 151654,
95
+ "vocab_size": 151936
96
+ }
lm_head.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5ee2b05fafaf667b34e93e4cf25cf64ffebc48b52fb52f28ebb925e9849275dd
3
+ size 421848828
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e99ce9449bc6c3d25e329d8c956c74a3d5f1976380a13f218c78729391f10d03
3
+ size 3017528
model_embed_tokens.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6af7b236bd01b4d92535ef352a8875a4ee8ec10f3c2fa41fccb5f3d7d52f138d
3
+ size 421848900
model_layers_0.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:94a3f8b126696e4862e7e9f92f1b7c8df6e251cb7546b75910b050a33406017b
3
+ size 104867965
model_layers_1.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0764c17a3ded3c2f46c52e4a9389a88de14c988eacfbb6e96673b1b231422bd0
3
+ size 114532097
model_layers_10.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:74bb0aeb912d4a311032c961c2b585262dfc666c48d69954945ce8986022e356
3
+ size 104389786
model_layers_11.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eab2271b4a36ad95d6f29cec61025ac6314fda4bb8557de40e6640bb5caa3b2e
3
+ size 104295112
model_layers_12.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9785d49b7427d4d82e0e73534cb9ac3ce8ed60bdf5b6af67ec00a6e563473cd2
3
+ size 104374495
model_layers_13.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7d74f559861710cebfa83213bf5485c31a32986c1368901ab13e13e63d9f6515
3
+ size 104185490
model_layers_14.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4853397f139cf431b1020e75c9fc92fb2ae3c79fe9796f7727364cc7a62176bf
3
+ size 104172242
model_layers_15.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d28ff32bb616f70f7f5beff26ce610c49316fc7ebdbf59c8532fddfacf51adf
3
+ size 104204763
model_layers_16.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:16a93b6c9c19322309f6f5ba8ed28866a88b7ae3f81d541136564df41392e91e
3
+ size 104205362
model_layers_17.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e264853ccfd27b3682c06d86a963b9688680d0f9a9bfc8e08e3593e2fcc49b92
3
+ size 104283834
model_layers_18.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8dd1870dc39bd0637a2834c73d758310eb652ddca9e3f9cbb218991ef26f12c1
3
+ size 104268098
model_layers_19.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:14a1ca6dbf4cef2ff96ee573932b3c7cd9307c4e731fd4ae356a95f4542825a3
3
+ size 104390362
model_layers_2.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:116d9823dd82af1a8b6bb88fa7658a0a27fa34c0e5bdb908ede3d88fbb8bfdde
3
+ size 110756449
model_layers_20.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc88accc9445e77f8287738c2ef5871f920614be71484f9583176fb96469a539
3
+ size 104444638
model_layers_21.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a0324404ec77e045fa1ac7cbb65e7c385ceff1330a43a067fcf0bf8874bddf75
3
+ size 104302741
model_layers_22.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4ec2b3fe30a80e36595ece3e18b4e81f43d253194e337aee9dd171653ef5c894
3
+ size 104262748
model_layers_23.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:59ce6c5f78dff8b88168dcd726a4a21c42eb421c600e62bf3e2f998bab36f5fc
3
+ size 104316594
model_layers_24.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e1c2ca6a73a5796aee54442fd174b28b395bd1122b08b2588d7ad4855bc2c594
3
+ size 104254665
model_layers_25.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:04d548a9758c6950fc1ac823e2ed00a7bd1c365d2fb68442b9ec0c1a9db44573
3
+ size 104300929
model_layers_26.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:980e5fbf4be192e0599dfb39971b44d0366e7581e54811bb2332b99744986b42
3
+ size 104325999
model_layers_27.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:38a715e8b75cd35eb79d8a0b6f4516c18a5c55b25efb96588a71d0187ab4a051
3
+ size 104316162
model_layers_28.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9406182b47a760ae03b7a29b2ca2ab7bb66eaf152580d19ccb1fb91e389595bb
3
+ size 104232334
model_layers_29.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:603274fbdeeb1f61f8d615113e34586b3361c083896c768bfd62098c3d68e395
3
+ size 104208350
model_layers_3.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8c8618a465181270899e58b8b54dd3b2297b2d0f5d498daaa0a0369d33b93aed
3
+ size 111185649
model_layers_30.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:989c73a7bab175d6f6ebd7005273f6364b2a576c068c4a47d8790e66b60c8de2
3
+ size 104353517
model_layers_31.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7eb95ecdb6d0dd6bb3821766fcd5ea027df59625a83ef549916a202f71aa3b38
3
+ size 104445087
model_layers_32.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2f0fefdac9a9b425926fbc2d791c0247d95948620856b9f474835c617a652e46
3
+ size 104250896
model_layers_33.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:09e7f3598ca31e7ff319e8fc499582e25d34a37ce2acd82b4e4103b6cd06b30b
3
+ size 104274857
model_layers_34.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a0967de9d8bbd3cc7066ca24652058abdcfb7ea814854564a3c9011187af67a9
3
+ size 104038970
model_layers_35.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec805b8a8d29f3f418f89e5eb43b0a0eb9cd0f20bdb3de48192e6b3446523e46
3
+ size 104150728
model_layers_4.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4fed6b2ca6f16c70bf4e3553bcf2bb7bad53281dd39a391c3e92e513afee0b76
3
+ size 109932213
model_layers_5.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1ee3b8e2d4b5a14be4004f381283ad42d28a1608654eb3ed4b48107a268fd0eb
3
+ size 107847499
model_layers_6.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63ed00821908e785f62f1811139d59740f40f44883b31f4c4e866b0eaf15193b
3
+ size 106401889
model_layers_7.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2cfcb049cba76153afe2e0dfe992f8d8b5828dd4fcd728ae13c8a7e7d49baa9f
3
+ size 106318999
model_layers_8.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:922a41b9e85ff9415669d8b8e612defcbc1061b410fcf4044078117aaec8f8f4
3
+ size 104341050
model_layers_9.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9c9e0bfcab2200c46b5c42d5376bf7319dd76445469309840e76b22b08ee7024
3
+ size 104509953
visual_blocks_0.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:28a0b0b0816a8828817ae602a919a74a3fc2d9fe50848f6c9358e28b2001ac68
3
+ size 26935509
visual_blocks_1.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f615034265525973e25a2c8f921638537d35b5fe3e8b669d13905bde61deb8f3
3
+ size 26802722
visual_blocks_10.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6bb91be9f1627767f62b8d2559ddeb7bf2140c6c321295b107558ab0a043c578
3
+ size 26621040
visual_blocks_11.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:98f4575eef32a8f4246016f26097b3959a8e7e095ceb6c1ad948252dc048a67b
3
+ size 26623814
visual_blocks_12.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8ed89e8a9f3967f699d216d73e31f847b2b953a2965fc6e51eeea39b60408b23
3
+ size 26627436
visual_blocks_13.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec3c5d7c96fcc89f99c5404215559d2a4a8954000f6b3d40c7a7084270da51e0
3
+ size 26634249
visual_blocks_14.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bd11bd40b87fdefa700b947eec1b2bbe5f22b01c09179f15a1a7a9a932d9e032
3
+ size 26632005
visual_blocks_15.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3da195bb62cb4b675440c1aa66b198a26e58f31dd8a898a99a525b8e2a740094
3
+ size 26643365
visual_blocks_16.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:743a2a4d5d224ba31b43ab14c008d9c216c8939b25dcba0bab5c94519884b5e6
3
+ size 26632836