Add project page link
Browse filesThis PR adds a link to the project page, improving discoverability for the AnyStory model.
    	
        README.md
    CHANGED
    
    | 
         @@ -1,10 +1,134 @@ 
     | 
|
| 1 | 
         
             
            ---
         
     | 
| 2 | 
         
            -
            license: apache-2.0
         
     | 
| 3 | 
         
            -
            language:
         
     | 
| 4 | 
         
            -
            - en
         
     | 
| 5 | 
         
             
            base_model:
         
     | 
| 6 | 
         
             
            - black-forest-labs/FLUX.1-dev
         
     | 
| 7 | 
         
            -
             
     | 
| 
         | 
|
| 8 | 
         
             
            library_name: diffusers
         
     | 
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 9 | 
         
             
            ---
         
     | 
| 10 | 
         
            -
             
     | 
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 
         | 
| 
         | 
|
| 1 | 
         
             
            ---
         
     | 
| 
         | 
|
| 
         | 
|
| 
         | 
|
| 2 | 
         
             
            base_model:
         
     | 
| 3 | 
         
             
            - black-forest-labs/FLUX.1-dev
         
     | 
| 4 | 
         
            +
            language:
         
     | 
| 5 | 
         
            +
            - en
         
     | 
| 6 | 
         
             
            library_name: diffusers
         
     | 
| 7 | 
         
            +
            license: apache-2.0
         
     | 
| 8 | 
         
            +
            pipeline_tag: text-to-image
         
     | 
| 9 | 
         
            +
            project_page: https://aigcdesigngroup.github.io/AnyStory/
         
     | 
| 10 | 
         
            +
            ---
         
     | 
| 11 | 
         
            +
             
     | 
| 12 | 
         
            +
            This model repo is for [AnyStory](https://github.com/junjiehe96/AnyStory).
         
     | 
| 13 | 
         
            +
             
     | 
| 14 | 
         
            +
            <div align="center">
         
     | 
| 15 | 
         
            +
            <h1>AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generation</h1>
         
     | 
| 16 | 
         
            +
             
     | 
| 17 | 
         
            +
            <a href='https://aigcdesigngroup.github.io/AnyStory/'><img src='https://img.shields.io/badge/Project-Page-green'></a>
         
     | 
| 18 | 
         
            +
            <a href='https://arxiv.org/pdf/2501.09503'><img src='https://img.shields.io/badge/arXiv-2501.09503-red'></a>
         
     | 
| 19 | 
         
            +
            <a href='https://huggingface.co/spaces/modelscope/AnyStory'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-yellow'></a>
         
     | 
| 20 | 
         
            +
            <a href='https://modelscope.cn/studios/iic/AnyStory'><img src='https://img.shields.io/badge/ModelScope-Spaces-blue'></a>
         
     | 
| 21 | 
         
            +
             
     | 
| 22 | 
         
            +
            </div>
         
     | 
| 23 | 
         
            +
             
     | 
| 24 | 
         
            +
            <img src='assets/examples-sdxl.jpg'>
         
     | 
| 25 | 
         
            +
             
     | 
| 26 | 
         
            +
            AnyStory is a unified approach for personalized subject generation. It not only achieves high-fidelity personalization for single subjects, but also for multiple subjects, without sacrificing subject fidelity.
         
     | 
| 27 | 
         
            +
             
     | 
| 28 | 
         
             
            ---
         
     | 
| 29 | 
         
            +
             
     | 
| 30 | 
         
            +
            ## News
         
     | 
| 31 | 
         
            +
             
     | 
| 32 | 
         
            +
            - [2025/05/01] 🚀 We release the code and demo for the `FLUX.1-dev` version of AnyStory.
         
     | 
| 33 | 
         
            +
             
     | 
| 34 | 
         
            +
            ## Usage
         
     | 
| 35 | 
         
            +
             
     | 
| 36 | 
         
            +
            ```python
         
     | 
| 37 | 
         
            +
            import torch
         
     | 
| 38 | 
         
            +
            from PIL import Image
         
     | 
| 39 | 
         
            +
             
     | 
| 40 | 
         
            +
            from huggingface_hub import hf_hub_download
         
     | 
| 41 | 
         
            +
            from anystory.generate import AnyStoryFluxPipeline
         
     | 
| 42 | 
         
            +
             
     | 
| 43 | 
         
            +
            anystory_path = hf_hub_download(repo_id="Junjie96/AnyStory", filename="anystory_flux.bin")
         
     | 
| 44 | 
         
            +
            story_pipe = AnyStoryFluxPipeline(
         
     | 
| 45 | 
         
            +
                hf_flux_pipeline_path="black-forest-labs/FLUX.1-dev",
         
     | 
| 46 | 
         
            +
                hf_flux_redux_path="black-forest-labs/FLUX.1-Redux-dev",
         
     | 
| 47 | 
         
            +
                anystory_path=anystory_path,
         
     | 
| 48 | 
         
            +
                device="cuda",
         
     | 
| 49 | 
         
            +
                torch_dtype=torch.bfloat16
         
     | 
| 50 | 
         
            +
            )
         
     | 
| 51 | 
         
            +
            # you can add lora here
         
     | 
| 52 | 
         
            +
            # story_pipe.flux_pipeline.load_lora_weights(lora_path, adapter_name="...")
         
     | 
| 53 | 
         
            +
             
     | 
| 54 | 
         
            +
            # single-subject
         
     | 
| 55 | 
         
            +
            subject_image = Image.open("assets/examples/1.webp").convert("RGB")
         
     | 
| 56 | 
         
            +
            subject_mask = Image.open("assets/examples/1_mask.webp").convert("L")
         
     | 
| 57 | 
         
            +
            prompt = "Cartoon style. A sheep is riding a skateboard and gliding through the city," \
         
     | 
| 58 | 
         
            +
                     " holding a wooden sign that says \"hello\"."
         
     | 
| 59 | 
         
            +
            image = story_pipe.generate(prompt=prompt, images=[subject_image], masks=[subject_mask], seed=2025,
         
     | 
| 60 | 
         
            +
                                        num_inference_steps=25, height=512, width=512,
         
     | 
| 61 | 
         
            +
                                        guidance_scale=3.5)
         
     | 
| 62 | 
         
            +
            image.save("output_1.png")
         
     | 
| 63 | 
         
            +
             
     | 
| 64 | 
         
            +
            # multi-subject
         
     | 
| 65 | 
         
            +
            subject_image_1 = Image.open("assets/examples/6_1.webp").convert("RGB")
         
     | 
| 66 | 
         
            +
            subject_mask_1 = Image.open("assets/examples/6_1_mask.webp").convert("L")
         
     | 
| 67 | 
         
            +
            subject_image_2 = Image.open("assets/examples/6_2.webp").convert("RGB")
         
     | 
| 68 | 
         
            +
            subject_mask_2 = Image.open("assets/examples/6_2_mask.webp").convert("L")
         
     | 
| 69 | 
         
            +
            prompt = "Two men are sitting by a wooden table, which is laden with delicious food and a pot of wine. " \
         
     | 
| 70 | 
         
            +
                     "One of the men holds a wine glass, drinking heartily with a bold expression; " \
         
     | 
| 71 | 
         
            +
                     "the other smiles as he pours wine for his companion, both of them engaged in cheerful conversation. " \
         
     | 
| 72 | 
         
            +
                     "In the background is an ancient pavilion surrounded by emerald bamboo groves, with sunlight filtering " \
         
     | 
| 73 | 
         
            +
                     "through the leaves to cast dappled shadows."
         
     | 
| 74 | 
         
            +
             
     | 
| 75 | 
         
            +
            image = story_pipe.generate(prompt=prompt,
         
     | 
| 76 | 
         
            +
                                        images=[subject_image_1, subject_image_2],
         
     | 
| 77 | 
         
            +
                                        masks=[subject_mask_1, subject_mask_2],
         
     | 
| 78 | 
         
            +
                                        seed=2025,
         
     | 
| 79 | 
         
            +
                                        enable_router=True, ref_start_at=0.09,
         
     | 
| 80 | 
         
            +
                                        num_inference_steps=25, height=512, width=512,
         
     | 
| 81 | 
         
            +
                                        guidance_scale=3.5)
         
     | 
| 82 | 
         
            +
            image.save("output_2.png")
         
     | 
| 83 | 
         
            +
            ```
         
     | 
| 84 | 
         
            +
             
     | 
| 85 | 
         
            +
            ### Storyboard generation
         
     | 
| 86 | 
         
            +
             
     | 
| 87 | 
         
            +
            ```python
         
     | 
| 88 | 
         
            +
            import json
         
     | 
| 89 | 
         
            +
            from storyboard import StoryboardPipeline
         
     | 
| 90 | 
         
            +
             
     | 
| 91 | 
         
            +
            storyboard_pipe = StoryboardPipeline()
         
     | 
| 92 | 
         
            +
             
     | 
| 93 | 
         
            +
            script_dict = json.load(open("assets/scripts/013420.json"))
         
     | 
| 94 | 
         
            +
            print(script_dict)
         
     | 
| 95 | 
         
            +
            results = storyboard_pipe(script_dict, style_name="Comic book")
         
     | 
| 96 | 
         
            +
            for key, result in results.items():
         
     | 
| 97 | 
         
            +
                result.save(f"output_1_{key}.png")
         
     | 
| 98 | 
         
            +
             
     | 
| 99 | 
         
            +
            # 狮子王辛巴成长
         
     | 
| 100 | 
         
            +
            script_dict = json.load(open("assets/scripts/014933.json"))
         
     | 
| 101 | 
         
            +
            print(script_dict)
         
     | 
| 102 | 
         
            +
            results = storyboard_pipe(script_dict, style_name="Japanese Anime")
         
     | 
| 103 | 
         
            +
            for key, result in results.items():
         
     | 
| 104 | 
         
            +
                result.save(f"output_2_{key}.png")
         
     | 
| 105 | 
         
            +
            ```
         
     | 
| 106 | 
         
            +
             
     | 
| 107 | 
         
            +
            Example output:
         
     | 
| 108 | 
         
            +
             
     | 
| 109 | 
         
            +
            <img src='assets/scripts/013420_result.jpg'>
         
     | 
| 110 | 
         
            +
            <img src='assets/scripts/014933_result.jpg'>
         
     | 
| 111 | 
         
            +
             
     | 
| 112 | 
         
            +
            ## Applications
         
     | 
| 113 | 
         
            +
             
     | 
| 114 | 
         
            +
            Intelligent creation of AI story pictures with [Qwen](https://github.com/QwenLM/Qwen3) Agent (please refer to `storyboard.py`)
         
     | 
| 115 | 
         
            +
             
     | 
| 116 | 
         
            +
            <img src='assets/storyboard_en.png'>
         
     | 
| 117 | 
         
            +
             
     | 
| 118 | 
         
            +
            AI Animation Video Production with [Wan](https://github.com/Wan-Video/Wan2.1) Image-to-Video
         
     | 
| 119 | 
         
            +
             
     | 
| 120 | 
         
            +
            ## **Acknowledgements**
         
     | 
| 121 | 
         
            +
             
     | 
| 122 | 
         
            +
            This code is built on [diffusers](https://github.com/huggingface/diffusers)
         
     | 
| 123 | 
         
            +
            and [OminiControl](https://github.com/Yuanshi9815/OminiControl). Highly appreciate their great work!
         
     | 
| 124 | 
         
            +
             
     | 
| 125 | 
         
            +
            ## Cite
         
     | 
| 126 | 
         
            +
             
     | 
| 127 | 
         
            +
            ```bibtex
         
     | 
| 128 | 
         
            +
            @article{he2025anystory,
         
     | 
| 129 | 
         
            +
              title={AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generation},
         
     | 
| 130 | 
         
            +
              author={He, Junjie and Tuo, Yuxiang and Chen, Binghui and Zhong, Chongyang and Geng, Yifeng and Bo, Liefeng},
         
     | 
| 131 | 
         
            +
              journal={arXiv preprint arXiv:2501.09503},
         
     | 
| 132 | 
         
            +
              year={2025}
         
     | 
| 133 | 
         
            +
            }
         
     | 
| 134 | 
         
            +
            ```
         
     |