Upload folder using huggingface_hub

.gitattributes

@@ -34,5 +34,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 assets/grid.gif filter=lfs diff=lfs merge=lfs -text
-assets/grid.mp4 filter=lfs diff=lfs merge=lfs -text
-assets/mochi-factory.webp filter=lfs diff=lfs merge=lfs -text
+assets/icon0.png filter=lfs diff=lfs merge=lfs -text
+assets/methods_overview.gif filter=lfs diff=lfs merge=lfs -text

README (1).md

@@ -0,0 +1,58 @@
+
+# Pusa VidGen
+
+[Codes](https://github.com/Yaofang-Liu/Pusa-VidGen) | [Paper (Coming Soon)](https://huggingface.co/RaphaelLiu/Pusa-V0.5)
+
+## Overview
+
+Pusa is built on a novel video diffusion paradigm of model that frame-level noise control, different from the conventional video diffusion models. We originally introduced that in the [FVDM](https://arxiv.org/abs/2410.03160) paper. With this paradigm, Pusa smoothly supports many video generation tasks (e.g., Text/Image/Video-to-Video)  while maintaining high-fidelity motion and strong prompt adherence because of our slight modification to the base model. This Pusa-V0.5 is an early preview version based on [Mochi1-Preview](https://huggingface.co/genmo/mochi-1-preview). We are open-sourcing this to invite community colloboration to improve the method and extend its capabilities.
+
+✨ **Key Features**
+- **Multi-task support**: Text-to-Video, Image-to-Video, Interpolation, Transition, Loop, Long Video, and more
+- **Shockingly-efficient**: Trained with only 0.1k H800 GPU hours, cost $0.1k, with 16 H800 GPUs, batchsize 32, 500 training iterations. It can be more efficient if train the model on a single node with more parallelism techniques. Welcome to colloboration :-)
+- **Full Open-Source**: Code, architecture, and training details included
+
+🔍 **Unique Architecture**
+- A novel diffusion model supporting frame-level noise with vectorized timesteps originally introduced in the [FVDM paper](https://arxiv.org/abs/2410.03160) for flexibility and scalability. 
+- Our modification to the base model does not influence its original Text-to-Video generation ability if no fintuing.
+- The method can be similarlly applied to mainstream video diffusion models like Hunyuan Video, wan2.1, etc. Welcome to colloboration again :-)
+
+## Download Weights
+
+You can use the Hugging Face CLI to download the model:
+```
+pip install huggingface_hub
+huggingface-cli download RaphaelLiu/Pusa-V0.5 --local-dir <path_to_downloaded_directory>
+
+```
+Or, directly download the weights from [Hugging Face](https://huggingface.co/RaphaelLiu/Pusa-V0.5) to a folder on your computer.
+
+
+## Limitations
+Pusa has a few known limitations. The base model Mochi generates videos at low resolution (480p). We expect to get better results when use our proposed method to more powerful models like Wan2.1. We also welcom collobartion from the community to improve the model and extend its capabilities.
+
+## Related Work
+- [mochi](https://huggingface.co/genmo/mochi-1-preview) is our base model, a top-tier open-source video generation models in Artifical Analysis Leaderboard for video generation.
+- [FVDM](https://arxiv.org/abs/2410.03160) introduces frame-level noise control with the vectorized timestep approach that inspired Pusa.
+
+## BibTeX
+```
+@misc{Liu2025pusa,
+      title={Pusa: A Next-Level All-in-One Video Diffusion Model},
+      author={Yaofang Liu and Rui Liu},
+      year={2025},
+      publisher = {GitHub},
+      journal = {GitHub repository},
+      howpublished={\url{https://github.com/Yaofang-Liu/Pusa-VidGen}}
+}
+```
+
+```
+@article{liu2024redefining,
+  title={Redefining Temporal Modeling in Video Diffusion: The Vectorized Timestep Approach},
+  author={Liu, Yaofang and Ren, Yumeng and Cun, Xiaodong and Artola, Aitor and Liu, Yang and Zeng, Tieyong and Chan, Raymond H and Morel, Jean-michel},
+  journal={arXiv preprint arXiv:2410.03160},
+  year={2024}
+}
+```
+

README.md

@@ -1,62 +1,80 @@
-
 # Pusa VidGen
 
-[Codes](https://github.com/Yaofang-Liu/Pusa-VidGen) | [Hugging Face](https://huggingface.co/RaphaelLiu/Pusa-V0.5)
+<div align="center">
+
+[![GitHub](https://img.shields.io/badge/GitHub-Code-blue?logo=github)](https://github.com/Yaofang-Liu/Pusa-VidGen)
+[![Paper](https://img.shields.io/badge/Paper-Coming%20Soon-red)](https://huggingface.co/RaphaelLiu/Pusa-V0.5)
+
+</div>
+
+<p align="center">
+  <img src="./Pusa-V0.5/assets/methods_overview.gif" width="80%">
+</p>
 
 ## Overview
 
-Pusa is an advanced open-source video generation model that builds upon Mochi 1 with significant enhancements. It supports multiple video generation tasks while maintaining high-fidelity motion and strong prompt adherence. The model is released under a permissive Apache 2.0 license.
+Pusa introduces a paradigm shift in video diffusion modeling through frame-level noise control, departing from conventional approaches. This innovation was first presented in our [FVDM](https://arxiv.org/abs/2410.03160) paper. Leveraging this architecture, Pusa seamlessly supports diverse video generation tasks (Text-to-Video, Image-to-Video, etc.) while maintaining exceptional motion fidelity and prompt adherence through our refined base model adaptations. Pusa-V0.5 represents an early preview based on [Mochi1-Preview](https://huggingface.co/genmo/mochi-1-preview). We are open-sourcing this work to foster community collaboration, enhance methodologies, and expand capabilities.
 
-✨ **Key Features**
-- **Multi-task support**: Text-to-Video, Image-to-Video, Interpolation, Transition, Loop, Long Video, and more
-- **Cost-efficient**: Trained with just 100 H100 GPU hours
-- **Full Open-Source**: Code, architecture, and training details included
+## ✨ Key Features
 
-🔍 **Unique Architecture**
-- A novel diffusion model supporting frame-level noise with vectorized timesteps originally introduced in the [FVDM paper](https://arxiv.org/abs/2410.03160) for flexibility and scalability
+- **Comprehensive Multi-task Support**:
+  - Text-to-Video generation
+  - Image-to-Video transformation
+  - Frame interpolation
+  - Video transitions
+  - Seamless looping
+  - Extended video generation
+  - And more...
 
-## Installation
+- **Unprecedented Efficiency**:
+  - Trained with only 0.1k H800 GPU hours
+  - Total training cost: $0.1k
+  - Hardware: 16 H800 GPUs
+  - Configuration: Batch size 32, 500 training iterations
+  - *Note: Efficiency can be further improved with single-node training and advanced parallelism techniques. Collaborations welcome!*
 
-Install using [uv](https://github.com/astral-sh/uv):
+- **Complete Open-Source Release**:
+  - Full codebase
+  - Detailed architecture specifications
+  - Comprehensive training methodology
 
-```bash
-git clone https://github.com/Yaofang-Liu/Pusa-VidGen
-cd models 
-pip install uv
-uv venv .venv
-source .venv/bin/activate
-uv pip install setuptools
-uv pip install -e . --no-build-isolation
-```
+## 🔍 Unique Architecture
 
+- **Novel Diffusion Paradigm**: Implements frame-level noise control with vectorized timesteps, originally introduced in the [FVDM paper](https://arxiv.org/abs/2410.03160), enabling unprecedented flexibility and scalability.
 
-If you want to install flash attention, you can use:
-```
-uv pip install -e .[flash] --no-build-isolation
-```
+- **Non-destructive Modification**: Our adaptations to the base model preserve its original Text-to-Video generation capabilities without requiring fine-tuning.
 
-You will also need to install [FFMPEG](https://www.ffmpeg.org/) to turn your outputs into videos.
+- **Universal Applicability**: The methodology can be readily applied to leading video diffusion models including Hunyuan Video, Wan2.1, and others. *Collaborations enthusiastically welcomed!*
 
-## Download Weights
+## Installation and Usage
 
-You can use the Hugging Face CLI to download the model:
-```
+### Download Weights
+
+**Option 1**: Use the Hugging Face CLI:
+```bash
 pip install huggingface_hub
 huggingface-cli download RaphaelLiu/Pusa-V0.5 --local-dir <path_to_downloaded_directory>
-
 ```
-Or, directly download the weights from [Hugging Face](https://huggingface.co/RaphaelLiu/Pusa-V0.5) to a folder on your computer.
 
+**Option 2**: Download directly from [Hugging Face](https://huggingface.co/RaphaelLiu/Pusa-V0.5) to your local machine.
 
 ## Limitations
-Pusa has a few known limitations. The base model Mochi generates videos at 480p. We expect to get better results when use our proposed method to more powerful models like Wan2.1. We also welcom collobartion from the community to improve the model and extend its capabilities.
+
+Pusa currently has several known limitations:
+- The base Mochi model generates videos at relatively low resolution (480p)
+- We anticipate significant quality improvements when applying our methodology to more advanced models like Wan2.1
+- We welcome community contributions to enhance model performance and extend its capabilities
 
 ## Related Work
-- [mochi](https://huggingface.co/genmo/mochi-1-preview) is our base model, top 3 open-source video generation models in Artifical Analysis Leaderboard for video generation.
-- [FVDM](https://arxiv.org/abs/2410.03160) introduces the vectorized timestep approach that inspired Pusa's frame-level noise control.
 
-## BibTeX
-```
+- [Mochi](https://huggingface.co/genmo/mochi-1-preview): Our foundation model, recognized as a leading open-source video generation system on the Artificial Analysis Leaderboard.
+- [FVDM](https://arxiv.org/abs/2410.03160): Introduces the groundbreaking frame-level noise control with vectorized timestep approach that inspired Pusa.
+
+## Citation
+
+If you find our work useful in your research, please consider citing:
+
+```bibtex
 @misc{Liu2025pusa,
       title={Pusa: A Next-Level All-in-One Video Diffusion Model},
       author={Yaofang Liu and Rui Liu},
@@ -67,7 +85,7 @@ Pusa has a few known limitations. The base model Mochi generates videos at 480p.
 }
 ```
 
-```
+```bibtex
 @article{liu2024redefining,
   title={Redefining Temporal Modeling in Video Diffusion: The Vectorized Timestep Approach},
   author={Liu, Yaofang and Ren, Yumeng and Cun, Xiaodong and Artola, Aitor and Liu, Yang and Zeng, Tieyong and Chan, Raymond H and Morel, Jean-michel},