Commit
Β·
72e28da
1
Parent(s):
e8f489c
docs(README): update image paths and add resources section
Browse files
README.md
CHANGED
@@ -29,7 +29,7 @@ sdk_version: 5.44.1
|
|
29 |
# π¬ Interactive Demo: Hybrid Transformer for Multi-Focus Image Fusion
|
30 |
|
31 |
<div align="center">
|
32 |
-
<img src="
|
33 |
|
34 |
[](https://huggingface.co/divitmittal/HybridTransformer-MFIF)
|
35 |
[](https://github.com/DivitMittal/HybridTransformer-MFIF)
|
@@ -44,6 +44,16 @@ sdk_version: 5.44.1
|
|
44 |
|
45 |
> π‘ **New to multi-focus fusion?** It's like having a camera that can focus on everything at once! Perfect for photography, microscopy, and document scanning.
|
46 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
47 |
## π How to Use This Demo
|
48 |
|
49 |
### Quick Start (30 seconds)
|
@@ -84,7 +94,7 @@ Our **FocalCrossViTHybrid** model represents a breakthrough in AI-powered image
|
|
84 |
### ποΈ Architecture Deep Dive
|
85 |
|
86 |
<div align="center">
|
87 |
-
<img src="
|
88 |
<p><em>Complete architecture diagram showing the hybrid transformer pipeline</em></p>
|
89 |
</div>
|
90 |
|
@@ -142,16 +152,6 @@ Our model was meticulously trained on the **Lytro Multi-Focus Dataset** using st
|
|
142 |
- **Historical Preservation**: Create high-quality digital archives
|
143 |
- **Technical Documentation**: Clear images of complex 3D objects
|
144 |
|
145 |
-
## π Complete Project Ecosystem
|
146 |
-
|
147 |
-
| Resource | Purpose | Best For | Link |
|
148 |
-
|----------|---------|----------|------|
|
149 |
-
| π **This Demo** | Interactive testing | Quick experimentation | *You're here!* |
|
150 |
-
| π€ **Model Hub** | Pre-trained weights | Integration & deployment | [Download Model](https://huggingface.co/divitmittal/HybridTransformer-MFIF) |
|
151 |
-
| π **GitHub Repository** | Source code & docs | Development & research | [View Code](https://github.com/DivitMittal/HybridTransformer-MFIF) |
|
152 |
-
| π **Kaggle Notebook** | Training pipeline | Learning & custom training | [Launch Notebook](https://www.kaggle.com/code/divitmittal/hybrid-transformer-mfif) |
|
153 |
-
| π¦ **Training Dataset** | Lytro Multi-Focus data | Research & benchmarking | [Download Dataset](https://www.kaggle.com/datasets/divitmittal/lytro-multi-focal-images) |
|
154 |
-
|
155 |
|
156 |
## π οΈ Run This Demo Locally
|
157 |
|
@@ -236,89 +236,27 @@ docker run -p 7860:7860 hybrid-transformer-demo
|
|
236 |
2. **Product Photography**: Ensure all product features are in sharp focus
|
237 |
3. **Art Documentation**: Capture textured surfaces with varying depths
|
238 |
|
239 |
-
##
|
240 |
-
|
241 |
-
|
242 |
-
- **
|
243 |
-
- **
|
244 |
-
- **
|
245 |
-
|
246 |
-
|
247 |
-
|
248 |
-
|
249 |
-
|
250 |
-
- **Error Handling**: Graceful handling of invalid inputs
|
251 |
-
- **Format Support**: JPEG, PNG, WebP, and most common formats
|
252 |
-
- **Size Limits**: Automatic resizing for optimal processing
|
253 |
-
- **Quality Preservation**: Maintains maximum possible image quality
|
254 |
-
|
255 |
-
### π Real-time Metrics (Displayed in Demo)
|
256 |
-
- **Fusion Quality Score**: Overall fusion effectiveness (0-100)
|
257 |
-
- **Focus Transfer Rate**: How well focus regions are preserved (%)
|
258 |
-
- **Edge Preservation**: Sharpness retention metric
|
259 |
-
- **Processing Time**: Actual computation time for your images
|
260 |
-
|
261 |
-
## π¬ Research & Development
|
262 |
-
|
263 |
-
### π Academic Value
|
264 |
-
- **Novel Architecture**: First implementation combining Focal Transformer + CrossViT for MFIF
|
265 |
-
- **Reproducible Research**: Complete codebase with deterministic training
|
266 |
-
- **Benchmark Dataset**: Standard evaluation on Lytro Multi-Focus Dataset
|
267 |
-
- **Comprehensive Metrics**: 6+ evaluation metrics for thorough assessment
|
268 |
-
|
269 |
-
### π§ͺ Experimental Framework
|
270 |
-
- **Modular Design**: Easy to modify components for ablation studies
|
271 |
-
- **Hyperparameter Tuning**: Configurable architecture and training parameters
|
272 |
-
- **Extension Support**: Framework for adding new transformer components
|
273 |
-
- **Comparative Analysis**: Built-in tools for method comparison
|
274 |
-
|
275 |
-
### π Educational Resource
|
276 |
-
- **Step-by-step Tutorials**: From basic concepts to advanced implementation
|
277 |
-
- **Interactive Learning**: Hands-on experience with transformer architectures
|
278 |
-
- **Code Documentation**: Extensively commented for educational use
|
279 |
-
- **Research Integration**: Easy to incorporate into academic projects
|
280 |
-
|
281 |
-
## π€ Community & Support
|
282 |
-
|
283 |
-
### π¬ Get Help
|
284 |
-
- **GitHub Issues**: Report bugs or request features
|
285 |
-
- **HuggingFace Discussions**: Community Q&A and tips
|
286 |
-
- **Kaggle Comments**: Dataset and training discussions
|
287 |
-
- **Email Support**: Direct contact for collaboration inquiries
|
288 |
-
|
289 |
-
### π Contributing
|
290 |
-
- **Code Contributions**: Submit PRs for improvements
|
291 |
-
- **Dataset Expansion**: Help grow the training data
|
292 |
-
- **Documentation**: Improve guides and tutorials
|
293 |
-
- **Testing**: Report issues and edge cases
|
294 |
-
|
295 |
-
### π·οΈ Citation
|
296 |
If you use this work in your research:
|
297 |
```bibtex
|
298 |
@software{mittal2024hybridtransformer,
|
299 |
title={HybridTransformer-MFIF: Focal Transformer and CrossViT Hybrid for Multi-Focus Image Fusion},
|
300 |
author={Mittal, Divit},
|
301 |
year={2024},
|
302 |
-
url={https://github.com/DivitMittal/HybridTransformer-MFIF}
|
303 |
-
note={Interactive demo available at HuggingFace Spaces}
|
304 |
}
|
305 |
```
|
306 |
|
307 |
-
## π License & Terms
|
308 |
-
|
309 |
-
### π Open Source License
|
310 |
-
**MIT License** - Free for commercial and non-commercial use
|
311 |
-
- β
**Commercial Use**: Integrate into products and services
|
312 |
-
- β
**Modification**: Adapt and customize for your needs
|
313 |
-
- β
**Distribution**: Share with proper attribution
|
314 |
-
- β
**Private Use**: Use in proprietary projects
|
315 |
-
|
316 |
-
### βοΈ Usage Terms
|
317 |
-
- **Attribution Required**: Credit the original work when using
|
318 |
-
- **No Warranty**: Provided "as-is" without guarantees
|
319 |
-
- **Ethical Use**: Please use responsibly and ethically
|
320 |
-
- **Research Friendly**: Encouraged for academic and research purposes
|
321 |
-
|
322 |
---
|
323 |
|
324 |
<div align="center">
|
|
|
29 |
# π¬ Interactive Demo: Hybrid Transformer for Multi-Focus Image Fusion
|
30 |
|
31 |
<div align="center">
|
32 |
+
<img src="./assets/logo.png" alt="HybridTransformer MFIF Logo" width="400"/>
|
33 |
|
34 |
[](https://huggingface.co/divitmittal/HybridTransformer-MFIF)
|
35 |
[](https://github.com/DivitMittal/HybridTransformer-MFIF)
|
|
|
44 |
|
45 |
> π‘ **New to multi-focus fusion?** It's like having a camera that can focus on everything at once! Perfect for photography, microscopy, and document scanning.
|
46 |
|
47 |
+
## π Project Resources
|
48 |
+
|
49 |
+
| Resource | Purpose | Best For | Link |
|
50 |
+
|----------|---------|----------|------|
|
51 |
+
| π **This Demo** | Interactive testing | Quick experimentation | *You're here!* |
|
52 |
+
| π€ **Model Hub** | Pre-trained weights | Integration & deployment | [Download Model](https://huggingface.co/divitmittal/HybridTransformer-MFIF) |
|
53 |
+
| π **GitHub Repository** | Source code & docs | Development & research | [View Code](https://github.com/DivitMittal/HybridTransformer-MFIF) |
|
54 |
+
| π **Kaggle Notebook** | Training pipeline | Learning & custom training | [Launch Notebook](https://www.kaggle.com/code/divitmittal/hybrid-transformer-mfif) |
|
55 |
+
| π¦ **Training Dataset** | Lytro Multi-Focus data | Research & benchmarking | [Download Dataset](https://www.kaggle.com/datasets/divitmittal/lytro-multi-focal-images) |
|
56 |
+
|
57 |
## π How to Use This Demo
|
58 |
|
59 |
### Quick Start (30 seconds)
|
|
|
94 |
### ποΈ Architecture Deep Dive
|
95 |
|
96 |
<div align="center">
|
97 |
+
<img src="./assets/model_architecture.png" alt="FocalCrossViTHybrid Architecture" width="700"/>
|
98 |
<p><em>Complete architecture diagram showing the hybrid transformer pipeline</em></p>
|
99 |
</div>
|
100 |
|
|
|
152 |
- **Historical Preservation**: Create high-quality digital archives
|
153 |
- **Technical Documentation**: Clear images of complex 3D objects
|
154 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
155 |
|
156 |
## π οΈ Run This Demo Locally
|
157 |
|
|
|
236 |
2. **Product Photography**: Ensure all product features are in sharp focus
|
237 |
3. **Art Documentation**: Capture textured surfaces with varying depths
|
238 |
|
239 |
+
## π οΈ Technical Details
|
240 |
+
|
241 |
+
- **Processing Time**: ~2-3 seconds per image pair (GPU) / ~8-12 seconds (CPU)
|
242 |
+
- **Model Size**: 73M+ parameters with hybrid transformer architecture
|
243 |
+
- **Input**: 224Γ224 pixel images (automatically resized)
|
244 |
+
- **Performance**: State-of-the-art results on Lytro Multi-Focus Dataset
|
245 |
+
|
246 |
+
## π Citation & License
|
247 |
+
|
248 |
+
**MIT License** - Free for commercial and non-commercial use.
|
249 |
+
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
250 |
If you use this work in your research:
|
251 |
```bibtex
|
252 |
@software{mittal2024hybridtransformer,
|
253 |
title={HybridTransformer-MFIF: Focal Transformer and CrossViT Hybrid for Multi-Focus Image Fusion},
|
254 |
author={Mittal, Divit},
|
255 |
year={2024},
|
256 |
+
url={https://github.com/DivitMittal/HybridTransformer-MFIF}
|
|
|
257 |
}
|
258 |
```
|
259 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
260 |
---
|
261 |
|
262 |
<div align="center">
|