svjack/Petio_Ball_wan_2_1_1_3_B_text2video_lora

Petio_Ball Text-to-Video Generation

This repository contains the necessary steps and scripts to generate videos using the Petio_Ball text-to-video model. The model leverages LoRA (Low-Rank Adaptation) weights and pre-trained components to create high-quality anime-style videos based on textual prompts.

Prerequisites

Before proceeding, ensure that you have the following installed on your system:

• Ubuntu (or a compatible Linux distribution) • Python 3.x • pip (Python package manager) • Git • Git LFS (Git Large File Storage) • FFmpeg

Installation

1. Update and Install Dependencies

   sudo apt-get update && sudo apt-get install cbm git-lfs ffmpeg
   

2. Clone the Repository

   git clone https://huggingface.co/svjack/Petio_Ball_wan_2_1_1_3_B_text2video_lora
   cd Petio_Ball_wan_2_1_1_3_B_text2video_lora
   

3. Install Python Dependencies

   pip install torch torchvision
   pip install -r requirements.txt
   pip install ascii-magic matplotlib tensorboard huggingface_hub datasets
   pip install moviepy==1.0.3
   pip install sageattention==1.0.6
   

4. Download Model Weights

   wget https://huggingface.co/Wan-AI/Wan2.1-T2V-14B/resolve/main/models_t5_umt5-xxl-enc-bf16.pth
   wget https://huggingface.co/DeepBeepMeep/Wan2.1/resolve/main/models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth
   wget https://huggingface.co/Wan-AI/Wan2.1-T2V-14B/resolve/main/Wan2.1_VAE.pth
   wget https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/resolve/main/split_files/diffusion_models/wan2.1_t2v_1.3B_bf16.safetensors
   wget https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/resolve/main/split_files/diffusion_models/wan2.1_t2v_14B_bf16.safetensors
   

Usage

To generate a video, use the wan_generate_video.py script with the appropriate parameters. Below are examples of how to generate videos using the Petio_Ball model.

Woods

python wan_generate_video.py --fp8 --task t2v-1.3B --video_size 480 832 --video_length 81 --infer_steps 20 \
--save_path save --output_type both \
--dit wan2.1_t2v_1.3B_bf16.safetensors --vae Wan2.1_VAE.pth \
--t5 models_t5_umt5-xxl-enc-bf16.pth \
--attn_mode torch \
--lora_weight petio_outputs/petio_w1_3_lora.safetensors \
--lora_multiplier 1.0 \
--prompt "In the style of Petio Animation , The video features a light blue, round character with large, expressive yellow eyes and small, rounded ears. The character wears a simple baseball cap, slightly tilted, adding a touch of playfulness. The background is a field of purple lavender, with a soft focus effect that gives the scene a dreamy quality. The character gently bounces in place amidst the lavender field, moving with a light and rhythmic motion, as if harmonizing with the surrounding sea of flowers. With each bounce, the character's facial expressions subtly shift, conveying emotions such as joy, curiosity, or contemplation, as though experiencing the serene and beautiful world in their own unique way."

Road Car

python wan_generate_video.py --fp8 --task t2v-1.3B --video_size 480 832 --video_length 81 --infer_steps 20 \
--save_path save --output_type both \
--dit wan2.1_t2v_1.3B_bf16.safetensors --vae Wan2.1_VAE.pth \
--t5 models_t5_umt5-xxl-enc-bf16.pth \
--attn_mode torch \
--lora_weight petio_outputs/petio_w1_3_lora.safetensors \
--lora_multiplier 1.0 \
--prompt "In the style of Petio Animation , The scene shifts to a sleek, high-speed highway at night—a fiery red fox in a fitted black racing jacket grips the wheel of a modified silver sports car, its amber eyes sharp under the neon glow of passing streetlights. The wind rushes through its fur as it leans into a curve, tail flicking with effortless precision, while the engine’s growl harmonizes with the pulsing beat of distant city lights. A half-finished energy drink rattles in the cupholder, and the reflection of streaking headlights dances across the windshield—every detail humming with adrenaline and untamed momentum."

Birthday

python wan_generate_video.py --fp8 --task t2v-1.3B --video_size 480 832 --video_length 81 --infer_steps 20 \
--save_path save --output_type both \
--dit wan2.1_t2v_1.3B_bf16.safetensors --vae Wan2.1_VAE.pth \
--t5 models_t5_umt5-xxl-enc-bf16.pth \
--attn_mode torch \
--lora_weight petio_outputs/petio_w1_3_lora.safetensors \
--lora_multiplier 1.0 \
--prompt 'In the style of Petio Animation , The warm glow of birthday candles flickers across Pooh’s honey-smeared muzzle as he beams at his little pink friend. Piglet, barely taller than the towering cake, clutches a party hat that keeps slipping over his eyes, his tiny hooves waving excitedly. Streamers tangle in Pooh’s ears as he bounces on his padded feet—"Oh bother!"—nearly toppling the wobbling stack of presents, while Piglet’s giggles chime like windchimes. A smear of frosting decorates both their noses, and the air smells of sugar, friendship, and the kind of joy that makes even bees pause mid-flight to smile.'

Parameters

• --fp8: Enable FP8 precision (optional).
• --task: Specify the task (e.g., t2v-1.3B).
• --video_size: Set the resolution of the generated video (e.g., 1024 1024).
• --video_length: Define the length of the video in frames.
• --infer_steps: Number of inference steps.
• --save_path: Directory to save the generated video.
• --output_type: Output type (e.g., both for video and frames).
• --dit: Path to the diffusion model weights.
• --vae: Path to the VAE model weights.
• --t5: Path to the T5 model weights.
• --attn_mode: Attention mode (e.g., torch).
• --lora_weight: Path to the LoRA weights.
• --lora_multiplier: Multiplier for LoRA weights.
• --prompt: Textual prompt for video generation.

Output

The generated video and frames will be saved in the specified save_path directory.

Troubleshooting

• Ensure all dependencies are correctly installed. • Verify that the model weights are downloaded and placed in the correct locations. • Check for any missing Python packages and install them using pip.

License

This project is licensed under the MIT License. See the LICENSE file for details.

Acknowledgments

• Hugging Face for hosting the model weights. • Wan-AI for providing the pre-trained models. • DeepBeepMeep for contributing to the model weights.

Contact

For any questions or issues, please open an issue on the repository or contact the maintainer.

---