xcheng20/stable-diffusion-painting-style-v1

This model is a fine-tuned version of CompVis/stable-diffusion-v1-4, trained on a small but rich dataset of 198 unique paintings by a single painter. It is optimized for generating text-to-image outputs with a distinctive hand-painted aesthetic.

This model card aims to document model details, usage recommendations, risks, and fine-tuning specifics in a transparent and reproducible manner.

Model Description

This model adapts Stable Diffusion v1.4 to replicate a specific human-created painting style. The training dataset includes 198 paintings in various themes and formats, designed to give the model a sense of color, brushwork, and composition typical to traditional art. It is suitable for generating stylized images with expressive, painterly textures. This model is for research purpose and discover how small dataset fine-tune can impact stable diffusion model behavior.

• Developed by: xcheng20
• Funded by: Self-funded
• Shared by: xcheng20
• Model type: Text-to-image generation
• Language(s): en
• License: Apache License 2.0
• Finetuned from model: CompVis/stable-diffusion-v1-4

Model Sources

• Repository: https://huggingface.co/xcheng20/stable-diffusion-painting-style-v1

Performance Comparison

Below is a visual comparison between images generated by this fine-tuned model (xcheng20/stable-diffusion-painting-style-v1) and the base model (CompVis/stable-diffusion-v1-4) using the same prompts.

Prompt	Base Model Output	Fine-tuned Model Output
"Two very detailed owls with yellow eyes"
"A phenix painted with white watercolor in the black background"
"A modern city landscpae skyline in watercolor"

Direct Use

This model is intended for artistic text-to-image generation. Prompt examples include:

• "a peaceful cabin in the woods, painterly style"
• "a surreal dreamscape in soft brushstrokes"

It is especially useful for artists, illustrators, and designers seeking an aesthetic similar to traditional hand-painted works.

Downstream Use

• Artistic draft generation
• Custom stylized prompt-to-image tools
• Inspiration for illustration and concept art workflows

Out-of-Scope Use

• Not suited for realistic portrait generation
• Should not be used for any NSFW, violent, or biased content
• Not recommended for medical, legal, or factual content generation

Bias, Risks, and Limitations

This model may not generalize well outside the stylistic patterns present in the dataset. It could reflect unintentional biases of the source style or create unrealistic outputs under complex prompts.

Recommendations

• Avoid prompts involving sensitive content
• Use with human review in artistic workflows
• Not intended for factual accuracy or realism

How to Get Started with the Model

Option A: Download stable_diffusion_loader.py from the "Files and versions" tab, and run the code below:

from stable_diffusion_loader import load_custom_pipeline, generate_image

pipe = load_custom_pipeline("./fine-tuned-model")
image = generate_image(pipe, "Two very detailed owls with yellow eyes")
image.show()

Option B: Clone the Github project