metadata
library_name: diffusers
datasets:
- NNNan/UniEM-3M
base_model:
- stabilityai/stable-diffusion-xl-base-1.0
pipeline_tag: text-to-image
tags:
- materials
- microstructure
- electron_micrograph
- characterization
- scientific_figure_understanding
UniEM-Gen
📘 Model Summary
This is the text-to-image diffusion model trained on the complete UniEM-3M dataset.
It is designed for electron microscopy (EM)-style image generation, enabling:
- Scientific data augmentation
- Proxy generation for microstructural distributions
- Multimodal research in materials science
🚀 Usage Example
Using diffusers
from diffusers import StableDiffusionPipeline
import torch
# Load model from Hugging Face
model_id = "NNNan/UniEM-Gen"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")
# Example prompt, sampled from UniEM-3M.
prompt = "SEM of Ceramic Powder: A mix of plate-like and rod-shaped particles. nanostructured. high density. densely packed and agglomerated. Multilayer. Wide range of particle sizes. Grayscale particles on a dark background."
# Generate image
image = pipe(prompt).images[0]
# Save or display
image.save("generated_em.png")
image.show()
🔄 Flexible Prompt Composition
You can generate structured scientific descriptions by randomly sampling one term from each of the nine attribute categories defined in the UniEM-3M. This repository includes attribute_values.json, which contains all observed values for these attributes, enabling reproducible and diverse prompt generation.
Use the following template to assemble the sampled terms into a coherent description:
<microscopy_type> of <subject>: <morphology>. <surface_texture>. <particle_density>. <distribution>. <layering>. <pixel_size_profile>. <color_profile>.
"💬 Replace each <...> placeholder with a real value from the corresponding category."
# Sampled attributes from UniEM-3M
from huggingface_hub import hf_hub_download
import json
import random
# Download the attribute_values.json file from the repo
json_path = hf_hub_download(
repo_id="NNNan/UniEM-Gen",
filename="attribute_values.json"
)
# Load the attribute values
with open(json_path, "r", encoding="utf-8") as f:
attribute_values = json.load(f)
# Generate a random description
template = "{microscopy_type} of {subject}: {morphology}. {surface_texture}. {particle_density}. {distribution}. {layering}. {pixel_size_profile}. {color_profile}."
sampled = {k: random.choice(list(v)) for k, v in attribute_values.items()}
prompt = template.format(**sampled)
print(prompt)
🖼️ Example Outputs
