SDXL LoRA - MohamedAhmedAE/stable-diffusion-xl-base_lora_finetuning

Prompt

a cute pokemon character

Model description

These are MohamedAhmedAE/stable-diffusion-xl-base_lora_finetuning LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0.

The weights were trained using LoRA fine-tuning on SDXL.

Trigger words

You should use the following prompts to generate images:

• a cute pokemon character
• a fire type pokemon
• a water type pokemon with big eyes

Download model

Weights for this model are available in Safetensors format.

Download pytorch_lora_weights.safetensors in the Files & versions tab.

Training Details

• Base Model: stabilityai/stable-diffusion-xl-base-1.0
• Training Epochs: 30
• Batch Size: 1
• Learning Rate: 5e-05
• LoRA Rank: 4
• LoRA Alpha: 32
• Dataset: reach-vb/pokemon-blip-captions
• Resolution: 1024x1024
• Checkpoint Step: 500

Usage

from diffusers import StableDiffusionXLPipeline
import torch

pipeline = StableDiffusionXLPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16,
    variant="fp16",
    use_safetensors=True
).to("cuda")

pipeline.load_lora_weights("MohamedAhmedAE/stable-diffusion-xl-base_lora_finetuning")

prompt = "a cute pokemon character"
image = pipeline(prompt, num_inference_steps=20, guidance_scale=7.5).images[0]
image.save("generated_image.png")

Training Configuration

{
  "base_model": "stabilityai/stable-diffusion-xl-base-1.0",
  "epochs": 30,
  "batch_size": 1,
  "learning_rate": 5e-05,
  "lora_rank": 4,
  "lora_alpha": 32,
  "resolution": 1024
}