ColorfulXL-Lightning

Model Details

v1.6. Based on ColorfulXL, with lightning ability.

• Fine-tuned on aesthetics
• Unet trained from 512 to 1280 with 64 steps
• Good prompt follow, trained text encoder
• Custom VAE
• Merged with 2,4,8, steps lightning unets from bytedance (supermario)
• Merged back with base model (not SGMUniform Euler works)
• Use with zero/low cfg
• Custom VAE
• Ability to generate pure white/black - colorful model with true colors

High range of resolutions supported (576 - 1280), 576*832 example:

Colorful:

There are problems with hands and faces (But who cares? Just say it's art!):

Detailed review (version 1.0), thx to Stevie2k8!

Usage

from diffusers import DiffusionPipeline
from diffusers import EulerDiscreteScheduler
import torch


pipeline = DiffusionPipeline.from_pretrained("recoilme/ColorfulXL-Lightning", torch_dtype=torch.float16,variant="fp16", use_safetensors=True).to("cuda")
pipeline.scheduler = EulerDiscreteScheduler.from_config(pipeline.scheduler.config, timestep_spacing="trailing")

prompt = "girl sitting on a small hill looking at night sky, fflix_dmatter, back view, distant exploding moon, nights darkness, intricate circuits and sensors, photographic realism style, detailed textures, peacefulness, mysterious."
height = 1024
width = 1024
steps = 3
scale = 0
seed = 2139965163
generator = torch.Generator(device="cpu").manual_seed(seed)

image = pipeline(
            prompt = prompt,
            height=height,
            width=width,
            guidance_scale=scale,
            num_inference_steps=steps,
            generator=generator,
        ).images[0]
image.show()
image.save("girl.png")

Local inference

https://github.com/recoilme/100lineSDXL

Space (upgrade on GPU)

https://huggingface.co/spaces/recoilme/ColorfulXL-Lightning

Model Details

• Developed by: AiArtLab
• Model type: Diffusion-based text-to-image generative model
• Model Description: This model is a fine-tuned model based on colorfulxl.
• License: This model is not permitted to be used behind API services. Please contact [email protected] for business inquires, commercial licensing, custom models, and consultation.

Uses

Direct Use

Research: possible research areas/tasks include:

• Generation of artworks and use in design and other artistic processes.
• Applications in educational or creative tools.
• Research on generative models.
• Safe deployment of models which have the potential to generate harmful content.
• Probing and understanding the limitations and biases of generative models.

Excluded uses are described below.

Out-of-Scope Use

The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.

Limitations and Bias

Limitations

• The model does not achieve perfect photorealism
• The model cannot render legible text
• The model struggles with more difficult tasks which involve compositionality, such as rendering an image corresponding to “A red cube on top of a blue sphere”
• Faces and people in general may not be generated properly.
• The autoencoding part of the model is lossy.

Bias

While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.

Contact

• For questions and comments about the model, please join https://aiartlab.org/.
• For future announcements / information about AiArtLab AI models, research, and events, please follow Discord.
• For business and partnership inquiries, please contact https://t.me/recoilme