nielsr HF Staff commited on
Commit
1d6d989
·
verified ·
1 Parent(s): 62f4c93

Add project page link

Browse files

This PR adds a link to the project page, improving discoverability for the AnyStory model.

Files changed (1) hide show
  1. README.md +129 -5
README.md CHANGED
@@ -1,10 +1,134 @@
1
  ---
2
- license: apache-2.0
3
- language:
4
- - en
5
  base_model:
6
  - black-forest-labs/FLUX.1-dev
7
- pipeline_tag: text-to-image
 
8
  library_name: diffusers
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  ---
10
- This model repo is for [AnyStory](https://github.com/junjiehe96/AnyStory).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
 
 
2
  base_model:
3
  - black-forest-labs/FLUX.1-dev
4
+ language:
5
+ - en
6
  library_name: diffusers
7
+ license: apache-2.0
8
+ pipeline_tag: text-to-image
9
+ project_page: https://aigcdesigngroup.github.io/AnyStory/
10
+ ---
11
+
12
+ This model repo is for [AnyStory](https://github.com/junjiehe96/AnyStory).
13
+
14
+ <div align="center">
15
+ <h1>AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generation</h1>
16
+
17
+ <a href='https://aigcdesigngroup.github.io/AnyStory/'><img src='https://img.shields.io/badge/Project-Page-green'></a>
18
+ <a href='https://arxiv.org/pdf/2501.09503'><img src='https://img.shields.io/badge/arXiv-2501.09503-red'></a>
19
+ <a href='https://huggingface.co/spaces/modelscope/AnyStory'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-yellow'></a>
20
+ <a href='https://modelscope.cn/studios/iic/AnyStory'><img src='https://img.shields.io/badge/ModelScope-Spaces-blue'></a>
21
+
22
+ </div>
23
+
24
+ <img src='assets/examples-sdxl.jpg'>
25
+
26
+ AnyStory is a unified approach for personalized subject generation. It not only achieves high-fidelity personalization for single subjects, but also for multiple subjects, without sacrificing subject fidelity.
27
+
28
  ---
29
+
30
+ ## News
31
+
32
+ - [2025/05/01] 🚀 We release the code and demo for the `FLUX.1-dev` version of AnyStory.
33
+
34
+ ## Usage
35
+
36
+ ```python
37
+ import torch
38
+ from PIL import Image
39
+
40
+ from huggingface_hub import hf_hub_download
41
+ from anystory.generate import AnyStoryFluxPipeline
42
+
43
+ anystory_path = hf_hub_download(repo_id="Junjie96/AnyStory", filename="anystory_flux.bin")
44
+ story_pipe = AnyStoryFluxPipeline(
45
+ hf_flux_pipeline_path="black-forest-labs/FLUX.1-dev",
46
+ hf_flux_redux_path="black-forest-labs/FLUX.1-Redux-dev",
47
+ anystory_path=anystory_path,
48
+ device="cuda",
49
+ torch_dtype=torch.bfloat16
50
+ )
51
+ # you can add lora here
52
+ # story_pipe.flux_pipeline.load_lora_weights(lora_path, adapter_name="...")
53
+
54
+ # single-subject
55
+ subject_image = Image.open("assets/examples/1.webp").convert("RGB")
56
+ subject_mask = Image.open("assets/examples/1_mask.webp").convert("L")
57
+ prompt = "Cartoon style. A sheep is riding a skateboard and gliding through the city," \
58
+ " holding a wooden sign that says \"hello\"."
59
+ image = story_pipe.generate(prompt=prompt, images=[subject_image], masks=[subject_mask], seed=2025,
60
+ num_inference_steps=25, height=512, width=512,
61
+ guidance_scale=3.5)
62
+ image.save("output_1.png")
63
+
64
+ # multi-subject
65
+ subject_image_1 = Image.open("assets/examples/6_1.webp").convert("RGB")
66
+ subject_mask_1 = Image.open("assets/examples/6_1_mask.webp").convert("L")
67
+ subject_image_2 = Image.open("assets/examples/6_2.webp").convert("RGB")
68
+ subject_mask_2 = Image.open("assets/examples/6_2_mask.webp").convert("L")
69
+ prompt = "Two men are sitting by a wooden table, which is laden with delicious food and a pot of wine. " \
70
+ "One of the men holds a wine glass, drinking heartily with a bold expression; " \
71
+ "the other smiles as he pours wine for his companion, both of them engaged in cheerful conversation. " \
72
+ "In the background is an ancient pavilion surrounded by emerald bamboo groves, with sunlight filtering " \
73
+ "through the leaves to cast dappled shadows."
74
+
75
+ image = story_pipe.generate(prompt=prompt,
76
+ images=[subject_image_1, subject_image_2],
77
+ masks=[subject_mask_1, subject_mask_2],
78
+ seed=2025,
79
+ enable_router=True, ref_start_at=0.09,
80
+ num_inference_steps=25, height=512, width=512,
81
+ guidance_scale=3.5)
82
+ image.save("output_2.png")
83
+ ```
84
+
85
+ ### Storyboard generation
86
+
87
+ ```python
88
+ import json
89
+ from storyboard import StoryboardPipeline
90
+
91
+ storyboard_pipe = StoryboardPipeline()
92
+
93
+ script_dict = json.load(open("assets/scripts/013420.json"))
94
+ print(script_dict)
95
+ results = storyboard_pipe(script_dict, style_name="Comic book")
96
+ for key, result in results.items():
97
+ result.save(f"output_1_{key}.png")
98
+
99
+ # 狮子王辛巴成长
100
+ script_dict = json.load(open("assets/scripts/014933.json"))
101
+ print(script_dict)
102
+ results = storyboard_pipe(script_dict, style_name="Japanese Anime")
103
+ for key, result in results.items():
104
+ result.save(f"output_2_{key}.png")
105
+ ```
106
+
107
+ Example output:
108
+
109
+ <img src='assets/scripts/013420_result.jpg'>
110
+ <img src='assets/scripts/014933_result.jpg'>
111
+
112
+ ## Applications
113
+
114
+ Intelligent creation of AI story pictures with [Qwen](https://github.com/QwenLM/Qwen3) Agent (please refer to `storyboard.py`)
115
+
116
+ <img src='assets/storyboard_en.png'>
117
+
118
+ AI Animation Video Production with [Wan](https://github.com/Wan-Video/Wan2.1) Image-to-Video
119
+
120
+ ## **Acknowledgements**
121
+
122
+ This code is built on [diffusers](https://github.com/huggingface/diffusers)
123
+ and [OminiControl](https://github.com/Yuanshi9815/OminiControl). Highly appreciate their great work!
124
+
125
+ ## Cite
126
+
127
+ ```bibtex
128
+ @article{he2025anystory,
129
+ title={AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generation},
130
+ author={He, Junjie and Tuo, Yuxiang and Chen, Binghui and Zhong, Chongyang and Geng, Yifeng and Bo, Liefeng},
131
+ journal={arXiv preprint arXiv:2501.09503},
132
+ year={2025}
133
+ }
134
+ ```