pilot-a-flux-lokr-lion-1e-5-bs2-ga3-v02

This is a LyCORIS adapter derived from black-forest-labs/FLUX.1-dev.

No validation prompt was used during training.

None

Validation settings

CFG: 3.0
CFG Rescale: 0.0
Steps: 20
Sampler: FlowMatchEulerDiscreteScheduler
Seed: 42
Resolution: 1024x1024
Skip-layer guidance:

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt
unconditional (blank prompt)

Negative Prompt
blurry, cropped, ugly

Prompt
formula race car with full helmet, fiery background with black streaks, capturing high-speed racing action and adrenaline

Negative Prompt
blurry, cropped, ugly

Prompt
an intense close-up of a focused racer boy with black eyes wearing a white helmet with an M, holding the red steering wheel, next to a worried female character with long red-orange hair and green eyes, vibrant yellow and orange background with dynamic motion effects, expressive faces, emphasizing the tension and adrenaline of a high-speed scenario, creating an engaging and dramatic visual

Negative Prompt
blurry, cropped, ugly

Prompt
an intense race on a bright track, highlighting a deep teal evo car with white stripes and number 5 in the lead, surrounded by several colorful competitors, dynamic speed lines, detailed track curvature, lush green forest below, emphasizing the high-speed excitement and competitive energy of the scene with striking visual impact

Negative Prompt
blurry, cropped, ugly

Prompt
in a panel, a determined little racer in a white helmet, closely followed by various colorful race cars, dramatic motion lines and GAROOOMMMM sound effect

Negative Prompt
blurry, cropped, ugly

Prompt
A young woman with straight dark hair is seated in a race car, her expression determined, in a racing environment, driving on a textured tarmac, wearing a racing jumpsuit featuring a prominent logo, her hair flowing naturally, exuding a focused and intense energy through her unwavering gaze and firm grip on the steering wheel, capturing the essence of motorsport bravado with elegance and poise

Negative Prompt
blurry, cropped, ugly

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

Training epochs: 83
Training steps: 9180
Learning rate: 1e-05
- Learning rate schedule: polynomial
- Warmup steps: 100
Max grad norm: 0.01
Effective batch size: 6
- Micro-batch size: 2
- Gradient accumulation steps: 3
- Number of GPUs: 1
Gradient checkpointing: True
Prediction type: flow-matching (extra parameters=['flux_schedule_auto_shift', 'shift=0.0', 'flux_guidance_mode=constant', 'flux_guidance_value=1.0', 'flow_matching_loss=compatible'])
Optimizer: optimi-lionweight_decay=1e-3
Trainable parameter precision: Pure BF16
Caption dropout probability: 10.0%

LyCORIS Config:

{
    "bypass_mode": true,
    "algo": "lokr",
    "multiplier": 1.0,
    "full_matrix": true,
    "linear_dim": 10000,
    "linear_alpha": 1,
    "factor": 12,
    "apply_preset": {
        "target_module": [
            "Attention",
            "FeedForward"
        ],
        "module_algo_map": {
            "FeedForward": {
                "factor": 12
            },
            "Attention": {
                "factor": 6
            }
        }
    }
}

Datasets

PILOT-A-FLUX-V02-512

Repeats: 1
Total number of images: 102
Total number of aspect buckets: 8
Resolution: 0.262144 megapixels
Cropped: True
Crop style: random
Crop aspect: closest
Used for regularisation data: No

PILOT-A-FLUX-V02-768

Repeats: 1
Total number of images: 102
Total number of aspect buckets: 1
Resolution: 0.589824 megapixels
Cropped: True
Crop style: random
Crop aspect: closest
Used for regularisation data: No

PILOT-A-FLUX-V02-1024

Repeats: 1
Total number of images: 102
Total number of aspect buckets: 5
Resolution: 1.048576 megapixels
Cropped: True
Crop style: random
Crop aspect: closest
Used for regularisation data: No

Inference

import torch
from diffusers import DiffusionPipeline
from lycoris import create_lycoris_from_weights


def download_adapter(repo_id: str):
    import os
    from huggingface_hub import hf_hub_download
    adapter_filename = "pytorch_lora_weights.safetensors"
    cache_dir = os.environ.get('HF_PATH', os.path.expanduser('~/.cache/huggingface/hub/models'))
    cleaned_adapter_path = repo_id.replace("/", "_").replace("\\", "_").replace(":", "_")
    path_to_adapter = os.path.join(cache_dir, cleaned_adapter_path)
    path_to_adapter_file = os.path.join(path_to_adapter, adapter_filename)
    os.makedirs(path_to_adapter, exist_ok=True)
    hf_hub_download(
        repo_id=repo_id, filename=adapter_filename, local_dir=path_to_adapter
    )

    return path_to_adapter_file
    
model_id = 'black-forest-labs/FLUX.1-dev'
adapter_repo_id = 'gattaplayer/pilot-a-flux-lokr-lion-1e-5-bs2-ga3-v02'
adapter_filename = 'pytorch_lora_weights.safetensors'
adapter_file_path = download_adapter(repo_id=adapter_repo_id)
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
lora_scale = 1.0
wrapper, _ = create_lycoris_from_weights(lora_scale, adapter_file_path, pipeline.transformer)
wrapper.merge_to()

prompt = "An astronaut is riding a horse through the jungles of Thailand."


## Optional: quantise the model to save on vram.
## Note: The model was quantised during training, and so it is recommended to do the same during inference time.
from optimum.quanto import quantize, freeze, qint8
quantize(pipeline.transformer, weights=qint8)
freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
image = pipeline(
    prompt=prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=1024,
    height=1024,
    guidance_scale=3.0,
).images[0]
image.save("output.png", format="PNG")

gattaplayer
/

pilot-a-flux-lokr-lion-1e-5-bs2-ga3-v02