Add metadata
#1
by
nielsr
HF Staff
- opened
README.md
CHANGED
|
@@ -1,135 +1,140 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
<
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
<a href="https://
|
| 12 |
-
<a href="https://
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
<
|
| 16 |
-
<
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
<a href="https://
|
| 23 |
-
<a href="https://
|
| 24 |
-
<a href="https://
|
| 25 |
-
<a href="https://
|
| 26 |
-
<a href="https://
|
| 27 |
-
<a href="https://
|
| 28 |
-
|
| 29 |
-
</
|
| 30 |
-
|
| 31 |
-
<
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
<p>JarvisArt
|
| 87 |
-
</div>
|
| 88 |
-
|
| 89 |
-
<div align="center">
|
| 90 |
-
<img src="assets/
|
| 91 |
-
<p>JarvisArt
|
| 92 |
-
</div>
|
| 93 |
-
|
| 94 |
-
<div align="center">
|
| 95 |
-
<img src="assets/
|
| 96 |
-
<p
|
| 97 |
-
</div>
|
| 98 |
-
|
| 99 |
-
|
| 100 |
-
<
|
| 101 |
-
<
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
| 129 |
-
|
| 130 |
-
|
| 131 |
-
|
| 132 |
-
|
| 133 |
-
|
| 134 |
-
|
| 135 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: transformers
|
| 3 |
+
pipeline_tag: image-text-to-text
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
<div align="center">
|
| 7 |
+
<img src="assets/logo.png" alt="JarvisArt Icon" width="100"/>
|
| 8 |
+
|
| 9 |
+
# JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent
|
| 10 |
+
<!-- **JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent** -->
|
| 11 |
+
<a href="https://arxiv.org/pdf/2506.17612"><img src="https://img.shields.io/badge/arXiv-2506.17612-b31b1b.svg" alt="Paper"></a>
|
| 12 |
+
<a href="https://huggingface.co/papers/2506.17612"><img src="https://img.shields.io/badge/๐ค-Daily%20Papers-ffbd00.svg" alt="Huggingface Daily Papers"></a>
|
| 13 |
+
<a href="https://jarvisart.vercel.app/"><img src="https://img.shields.io/badge/Project%20Page-Visit-blue" alt="Project Page"></a>
|
| 14 |
+
<a href="https://www.youtube.com/watch?v=Ol28DQj8wV8"><img src="https://img.shields.io/badge/YouTube-Watch-red" alt="YouTube"></a>
|
| 15 |
+
<a href="https://www.bilibili.com/video/BV1Sd3nzREvP/?spm_id_from=333.1007.top_right_bar_window_history.content.click&vd_source=3939804dc1d27869e194605ae46329ec"><img src="https://img.shields.io/badge/BiliBili-ๅๅฉๅๅฉ-FF69B4" alt="BiliBili"></a>
|
| 16 |
+
<a href="https://x.com/ling_yunlong/status/1940010865627103419"><img src="https://img.shields.io/twitter/follow/LYL1015?style=social" alt="Twitter Follow"></a>
|
| 17 |
+
<a href="https://github.com/LYL1015/JarvisArt"><img src="https://img.shields.io/github/stars/LYL1015/JarvisArt?style=social" alt="GitHub Stars"></a>
|
| 18 |
+
</div>
|
| 19 |
+
|
| 20 |
+
<div align="center">
|
| 21 |
+
<p>
|
| 22 |
+
<a href="https://lyl1015.github.io/">Yunlong Lin</a><sup>1*</sup>,
|
| 23 |
+
<a href="https://github.com/iendi">Zixu Lin</a><sup>1*</sup>,
|
| 24 |
+
<a href="https://github.com/kunjie-lin">Kunjie Lin</a><sup>1*</sup>,
|
| 25 |
+
<a href="https://noyii.github.io/">Jinbin Bai</a><sup>5</sup>,
|
| 26 |
+
<a href="https://paulpanwang.github.io/">Panwang Pan</a><sup>4</sup>,
|
| 27 |
+
<a href="https://chenxinli001.github.io/">Chenxin Li</a><sup>3</sup>,
|
| 28 |
+
<a href="https://haoyuchen.com/">Haoyu Chen</a><sup>2</sup>,
|
| 29 |
+
<a href="https://zhongdao.github.io/">Zhongdao Wang</a><sup>6</sup>,
|
| 30 |
+
<a href="https://scholar.google.com/citations?user=k5hVBfMAAAAJ&hl=zh-CN">Xinghao Ding</a><sup>1โ </sup>,
|
| 31 |
+
<a href="https://fenglinglwb.github.io/">Wenbo Li</a><sup>3โฃ</sup>,
|
| 32 |
+
<a href="https://yanshuicheng.info/">Shuicheng Yan</a><sup>5โ </sup>
|
| 33 |
+
</p>
|
| 34 |
+
</div>
|
| 35 |
+
|
| 36 |
+
<div align="center">
|
| 37 |
+
<p>
|
| 38 |
+
<sup>1</sup>Xiamen University, <sup>2</sup>The Hong Kong University of Science and Technology (Guangzhou), <sup>3</sup> The Chinese University of Hong Kong, <sup>4</sup>Bytedance, <sup>5</sup>National University of Singapore, <sup>6</sup>Tsinghua University
|
| 39 |
+
</p>
|
| 40 |
+
<!-- <sup>*</sup>Equal Contributions <sup>โฃ</sup>Project Leader <sup>โ </sup>Corresponding Author -->
|
| 41 |
+
<!-- <p>Accepted by CVPR 2025</p> -->
|
| 42 |
+
</div>
|
| 43 |
+
|
| 44 |
+
---
|
| 45 |
+
|
| 46 |
+
## โ ๏ธ Security Warning
|
| 47 |
+
|
| 48 |
+
**IMPORTANT: This is the ONLY official JarvisArt repository!**
|
| 49 |
+
|
| 50 |
+
We have identified **fake repositories** claiming to be JarvisArt that may contain **malware, viruses, or malicious code**. Please be extremely cautious and only use this official repository.
|
| 51 |
+
|
| 52 |
+
**Known fake/malicious repositories:**
|
| 53 |
+
- โ `https://github.com/joelp0/JarvisArt` - **FAKE & POTENTIALLY DANGEROUS**
|
| 54 |
+
- โ Any other repositories not from our official organization
|
| 55 |
+
|
| 56 |
+
<!-- --- -->
|
| 57 |
+
|
| 58 |
+
<!-- ## ๐ฎ Updates
|
| 59 |
+
|
| 60 |
+
- **[Coming Soon]** ๐ Gradio demo and Hugging Face demo will be released first.
|
| 61 |
+
- **[Coming Soon]** ๐ฏ Training and inference code will be released.
|
| 62 |
+
- **[2025.06]** ๐ Paper is now available on arXiv.
|
| 63 |
+
- **[2025.06]** ๐ Project page is live. -->
|
| 64 |
+
|
| 65 |
+
---
|
| 66 |
+
|
| 67 |
+
## ๐ Overview
|
| 68 |
+
|
| 69 |
+
<div align="center">
|
| 70 |
+
<img src="assets/teaser.jpg" alt="JarvisArt Teaser" width="800"/>
|
| 71 |
+
<br>
|
| 72 |
+
<em>JarvisArt workflow and results showcase</em>
|
| 73 |
+
</div>
|
| 74 |
+
|
| 75 |
+
JarvisArt is a multi-modal large language model (MLLM)-driven agent for intelligent photo retouching. It is designed to liberate human creativity by understanding user intent, mimicking the reasoning of professional artists, and coordinating over 200 tools in Adobe Lightroom. JarvisArt utilizes a novel two-stage training framework, starting with Chain-of-Thought supervised fine-tuning for foundational reasoning, followed by Group Relative Policy Optimization for Retouching (GRPO-R) to enhance its decision-making and tool proficiency. Supported by the newly created MMArt dataset (55K samples) and MMArt-Bench, JarvisArt demonstrates superior performance, outperforming GPT-4o with a 60% improvement in pixel-level metrics for content fidelity while maintaining comparable instruction-following capabilities.
|
| 76 |
+
|
| 77 |
+
---
|
| 78 |
+
|
| 79 |
+
## ๐ฌ Demo Videos
|
| 80 |
+
|
| 81 |
+
<!-- <div align="center">
|
| 82 |
+
<video width="800" controls>
|
| 83 |
+
<source src="assets/demo.mp4" type="video/mp4">
|
| 84 |
+
Your browser does not support the video tag.
|
| 85 |
+
</video>
|
| 86 |
+
<p>JarvisArt Demo Video: Showcasing intelligent photo retouching capabilities</p>
|
| 87 |
+
</div> -->
|
| 88 |
+
|
| 89 |
+
<!-- <div align="center">
|
| 90 |
+
<img src="assets/demo1.gif" alt="JarvisArt Demo" width="800px">
|
| 91 |
+
<p>JarvisArt Interactive Retouching Demonstration</p>
|
| 92 |
+
</div>
|
| 93 |
+
|
| 94 |
+
<div align="center">
|
| 95 |
+
<img src="assets/demo2.gif" alt="JarvisArt Demo" width="800px">
|
| 96 |
+
<p>JarvisArt Multimodal Instruction Understanding and Execution</p>
|
| 97 |
+
</div> -->
|
| 98 |
+
Global Retouching Case
|
| 99 |
+
<div align="center">
|
| 100 |
+
<img src="assets/global_demo1.gif" alt="JarvisArt Demo" width="800px">
|
| 101 |
+
<p></p>
|
| 102 |
+
</div>
|
| 103 |
+
|
| 104 |
+
Local Retouching Case
|
| 105 |
+
<div align="center">
|
| 106 |
+
<img src="assets/local_demo1.gif" alt="JarvisArt Demo" width="800px">
|
| 107 |
+
<p>JarvisArt supports multi-granularity retouching goals, ranging from scene-level adjustments to region-specific refinements. Users can perform intuitive, free-form edits through natural inputs such as text prompts and bounding boxes</p>
|
| 108 |
+
</div>
|
| 109 |
+
|
| 110 |
+
## ๐ Citation
|
| 111 |
+
|
| 112 |
+
If you find JarvisArt useful in your research, please consider citing:
|
| 113 |
+
|
| 114 |
+
```bibtex
|
| 115 |
+
@article{jarvisart2025,
|
| 116 |
+
title={JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent},
|
| 117 |
+
author={Yunlong Lin and Zixu Lin and Kunjie Lin and Jinbin Bai and Panwang Pan and Chenxin Li and Haoyu Chen and Zhongdao Wang and Xinghao Ding and Wenbo Li and Shuicheng Yan},
|
| 118 |
+
year={2025},
|
| 119 |
+
journal={arXiv preprint arXiv:2506.17612}
|
| 120 |
+
}
|
| 121 |
+
```
|
| 122 |
+
|
| 123 |
+
---
|
| 124 |
+
|
| 125 |
+
|
| 126 |
+
## ๐ง Contact
|
| 127 |
+
|
| 128 |
+
For any questions or inquiries, please reach out to us:
|
| 129 |
+
|
| 130 |
+
- **Yunlong Lin**: [email protected]
|
| 131 |
+
- **Zixu Lin**: [email protected]
|
| 132 |
+
- **Kunjie Lin**: [email protected]
|
| 133 |
+
|
| 134 |
+
---
|
| 135 |
+
|
| 136 |
+
## ๐ Acknowledgements
|
| 137 |
+
|
| 138 |
+
We would like to express our gratitude to [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory.git) and [gradio_image_annotator](https://github.com/edgarGracia/gradio_image_annotator.git) for their valuable open-source contributions which have provided important technical references for our work.
|
| 139 |
+
|
| 140 |
+
|