README.md

---
library_name: diffusers
datasets:
- NNNan/UniEM-3M
base_model:
- stabilityai/stable-diffusion-xl-base-1.0
pipeline_tag: text-to-image
tags:
- materials
- microstructure
- electron_micrograph
- characterization
- scientific_figure_understanding
license: mit
---

# UniEM-Gen

## 🖼️ Example Outputs  

![Generated Example](examples/figure1.png)

## 📘 Model Summary  
This is the text-to-image diffusion model trained on the complete **[UniEM-3M](https://huggingface.co/datasets/NNNan/UniEM-3M)** dataset.  
It is designed for **electron microscopy (EM)-style image generation**, enabling:  
- Scientific data augmentation  
- Proxy generation for microstructural distributions  
- Multimodal research in materials science

---

## 🚀 Usage Example  

### Using `diffusers`
```python
from diffusers import StableDiffusionPipeline
import torch

# Load model from Hugging Face
model_id = "NNNan/UniEM-Gen"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")

# Example prompt, sampled from UniEM-3M.
prompt = "SEM of Ceramic Powder: A mix of plate-like and rod-shaped particles. nanostructured. high density. densely packed and agglomerated. Multilayer. Wide range of particle sizes. Grayscale particles on a dark background."

# Generate image
image = pipe(prompt).images[0]

# Save or display
image.save("generated_em.png")
image.show()
```
---

### 🔄 Flexible Prompt Composition

You can generate structured scientific descriptions by randomly sampling one term from each of the nine attribute categories defined in the **[UniEM-3M](https://huggingface.co/datasets/NNNan/UniEM-3M)**. This repository includes attribute_values.json, which contains all observed values for these attributes, enabling reproducible and diverse prompt generation.

Use the following template to assemble the sampled terms into a coherent description:

```text
<microscopy_type> of <subject>: <morphology>. <surface_texture>. <particle_density>. <distribution>. <layering>. <pixel_size_profile>. <color_profile>.
"💬 Replace each <...> placeholder with a real value from the corresponding category."
```

```python
# Sampled attributes from UniEM-3M
from huggingface_hub import hf_hub_download
import json
import random

# Download the attribute_values.json file from the repo
json_path = hf_hub_download(
    repo_id="NNNan/UniEM-Gen",
    filename="attribute_values.json"
)

# Load the attribute values
with open(json_path, "r", encoding="utf-8") as f:
    attribute_values = json.load(f)

# Generate a random description
template = "{microscopy_type} of {subject}: {morphology}. {surface_texture}. {particle_density}. {distribution}. {layering}. {pixel_size_profile}. {color_profile}."
sampled = {k: random.choice(list(v)) for k, v in attribute_values.items()}
prompt = template.format(**sampled)
print(prompt)
```

---

## 📖 Citation  
If you use this dataset, please cite:  

```bibtex
@misc{wang2025uniem3muniversalelectronmicrograph,
      title={UniEM-3M: A Universal Electron Micrograph Dataset for Microstructural Segmentation and Generation}, 
      author={Nan wang and Zhiyi Xia and Yiming Li and Shi Tang and Zuxin Fan and Xi Fang and Haoyi Tao and Xiaochen Cai and Guolin Ke and Linfeng Zhang and Yanhui Hong},
      year={2025},
      eprint={2508.16239},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2508.16239}, 
}