Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -1,5 +1,5 @@
|
|
1 |
|
2 |
-
# 🧬 ViDRiP-LLaVA:
|
3 |
|
4 |
**ViDRiP-LLaVA** is a vision-language framework designed for instruction-based diagnostic reasoning using both image patches and video clips from pathology slides. It builds on LLaVA and extends it to the medical domain with domain-specific datasets and fine-tuned models.
|
5 |
|
@@ -19,10 +19,12 @@ Our method leverages chain-of-thought (CoT) prompting to distill the reasoning c
|
|
19 |
</p>
|
20 |
|
21 |
|
22 |
-
## 📚 Datasets
|
23 |
|
24 |
### 🔹 [ViDRiP_Instruct_Train](https://huggingface.co/datasets/trinhvg/ViDRiP_Instruct_Train)
|
25 |
-
|
|
|
|
|
26 |
### 🔹 [ViDRiP_Instruct_Train_Video_Hugging Face](https://huggingface.co/datasets/trinhvg/ViDRiP_Instruct_Train) (There is 10 zip files)
|
27 |
|
28 |
- 4,000+ instruction-style samples
|
@@ -39,6 +41,11 @@ Our method leverages chain-of-thought (CoT) prompting to distill the reasoning c
|
|
39 |
- Held-out test set of diagnostic Q&A pairs
|
40 |
- Used for benchmarking reasoning performance
|
41 |
|
|
|
|
|
|
|
|
|
|
|
42 |
|
43 |
|
44 |
---
|
@@ -101,3 +108,31 @@ license: cc-by-nc-nd-3.0
|
|
101 |
|
102 |
### Citation:
|
103 |
Coming soon
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
|
2 |
+
# 🧬 ViDRiP-LLaVA: A Dataset and Benchmark for Diagnostic Reasoning from Pathology Videos
|
3 |
|
4 |
**ViDRiP-LLaVA** is a vision-language framework designed for instruction-based diagnostic reasoning using both image patches and video clips from pathology slides. It builds on LLaVA and extends it to the medical domain with domain-specific datasets and fine-tuned models.
|
5 |
|
|
|
19 |
</p>
|
20 |
|
21 |
|
22 |
+
## 📚 Video Datasets
|
23 |
|
24 |
### 🔹 [ViDRiP_Instruct_Train](https://huggingface.co/datasets/trinhvg/ViDRiP_Instruct_Train)
|
25 |
+
The videos data is ~ 100 GB:
|
26 |
+
|
27 |
+
[//]: # (### 🔹 [ViDRiP_Instruct_Train_Video_GoogleDrive](https://drive.google.com/drive/folders/1oxZlaJpE7PGDYt32LeoGgIzwEvWdnupY?usp=sharing))
|
28 |
### 🔹 [ViDRiP_Instruct_Train_Video_Hugging Face](https://huggingface.co/datasets/trinhvg/ViDRiP_Instruct_Train) (There is 10 zip files)
|
29 |
|
30 |
- 4,000+ instruction-style samples
|
|
|
41 |
- Held-out test set of diagnostic Q&A pairs
|
42 |
- Used for benchmarking reasoning performance
|
43 |
|
44 |
+
## 📚 Image Datasets
|
45 |
+
We use publicly available datasets: Quilt-LLaVA and PathAsst.
|
46 |
+
Please refer to their respective repositories for download instructions.
|
47 |
+
- [**Quilt-LLaVA**](https://github.com/aldraus/quilt-llava): A vision-language dataset for pathology adapted from LLaVA.
|
48 |
+
- [**PathAsst**](https://github.com/superjamessyx/Generative-Foundation-AI-Assistant-for-Pathology): A generative assistant for pathology with curated image-text pairs.
|
49 |
|
50 |
|
51 |
---
|
|
|
108 |
|
109 |
### Citation:
|
110 |
Coming soon
|
111 |
+
|
112 |
+
|
113 |
+
|
114 |
+
## 📄 Usage and License Notices
|
115 |
+
|
116 |
+
**ViDRiP-LLaVA** (Vision-language Diagnostic Reasoning in Pathology), including its dataset, code, and model checkpoints, is released strictly for **non-commercial research purposes only**.
|
117 |
+
|
118 |
+
### 📁 Licenses
|
119 |
+
|
120 |
+
* **Dataset:**
|
121 |
+
Licensed under [**CC BY-NC-ND 3.0**](https://creativecommons.org/licenses/by-nc-nd/3.0/) (Attribution–NonCommercial–NoDerivatives)
|
122 |
+
* **Code and pretrained models:**
|
123 |
+
Licensed under [**CC BY-NC 3.0**](https://creativecommons.org/licenses/by-nc/3.0/) (Attribution–NonCommercial)
|
124 |
+
|
125 |
+
### ⚙️ Dependencies and Components
|
126 |
+
|
127 |
+
This project may incorporate or build upon resources such as **LLaVA-Next**, **QUILT-1M**, **LLaMA**, **PathAsst**, and **GPT-4**, each subject to their own licenses and **Terms of Use**.
|
128 |
+
|
129 |
+
### 🎥 Source Acknowledgment
|
130 |
+
|
131 |
+
ViDRiP-LLaVA includes data derived from **public educational pathology videos hosted on YouTube**.
|
132 |
+
All content usage complies with [**YouTube’s Terms of Service**](https://www.youtube.com/t/terms), and the **intellectual property rights of the original pathologist creators are fully acknowledged and respected**.
|
133 |
+
|
134 |
+
### 🚫 Restrictions
|
135 |
+
|
136 |
+
* Not for **commercial use**
|
137 |
+
* Not to be used in **clinical care** or **medical decision-making**
|
138 |
+
* For **academic research, development, and evaluation only**
|