trinhvg commited on
Commit
b29a222
·
verified ·
1 Parent(s): 4436a91

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +38 -3
README.md CHANGED
@@ -1,5 +1,5 @@
1
 
2
- # 🧬 ViDRiP-LLaVA: Multimodal Diagnostic Reasoning in Pathology
3
 
4
  **ViDRiP-LLaVA** is a vision-language framework designed for instruction-based diagnostic reasoning using both image patches and video clips from pathology slides. It builds on LLaVA and extends it to the medical domain with domain-specific datasets and fine-tuned models.
5
 
@@ -19,10 +19,12 @@ Our method leverages chain-of-thought (CoT) prompting to distill the reasoning c
19
  </p>
20
 
21
 
22
- ## 📚 Datasets
23
 
24
  ### 🔹 [ViDRiP_Instruct_Train](https://huggingface.co/datasets/trinhvg/ViDRiP_Instruct_Train)
25
- ### 🔹 [ViDRiP_Instruct_Train_Video_GoogleDrive](https://drive.google.com/drive/folders/1oxZlaJpE7PGDYt32LeoGgIzwEvWdnupY?usp=sharing)
 
 
26
  ### 🔹 [ViDRiP_Instruct_Train_Video_Hugging Face](https://huggingface.co/datasets/trinhvg/ViDRiP_Instruct_Train) (There is 10 zip files)
27
 
28
  - 4,000+ instruction-style samples
@@ -39,6 +41,11 @@ Our method leverages chain-of-thought (CoT) prompting to distill the reasoning c
39
  - Held-out test set of diagnostic Q&A pairs
40
  - Used for benchmarking reasoning performance
41
 
 
 
 
 
 
42
 
43
 
44
  ---
@@ -101,3 +108,31 @@ license: cc-by-nc-nd-3.0
101
 
102
  ### Citation:
103
  Coming soon
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
 
2
+ # 🧬 ViDRiP-LLaVA: A Dataset and Benchmark for Diagnostic Reasoning from Pathology Videos
3
 
4
  **ViDRiP-LLaVA** is a vision-language framework designed for instruction-based diagnostic reasoning using both image patches and video clips from pathology slides. It builds on LLaVA and extends it to the medical domain with domain-specific datasets and fine-tuned models.
5
 
 
19
  </p>
20
 
21
 
22
+ ## 📚 Video Datasets
23
 
24
  ### 🔹 [ViDRiP_Instruct_Train](https://huggingface.co/datasets/trinhvg/ViDRiP_Instruct_Train)
25
+ The videos data is ~ 100 GB:
26
+
27
+ [//]: # (### 🔹 [ViDRiP_Instruct_Train_Video_GoogleDrive]&#40;https://drive.google.com/drive/folders/1oxZlaJpE7PGDYt32LeoGgIzwEvWdnupY?usp=sharing&#41;)
28
  ### 🔹 [ViDRiP_Instruct_Train_Video_Hugging Face](https://huggingface.co/datasets/trinhvg/ViDRiP_Instruct_Train) (There is 10 zip files)
29
 
30
  - 4,000+ instruction-style samples
 
41
  - Held-out test set of diagnostic Q&A pairs
42
  - Used for benchmarking reasoning performance
43
 
44
+ ## 📚 Image Datasets
45
+ We use publicly available datasets: Quilt-LLaVA and PathAsst.
46
+ Please refer to their respective repositories for download instructions.
47
+ - [**Quilt-LLaVA**](https://github.com/aldraus/quilt-llava): A vision-language dataset for pathology adapted from LLaVA.
48
+ - [**PathAsst**](https://github.com/superjamessyx/Generative-Foundation-AI-Assistant-for-Pathology): A generative assistant for pathology with curated image-text pairs.
49
 
50
 
51
  ---
 
108
 
109
  ### Citation:
110
  Coming soon
111
+
112
+
113
+
114
+ ## 📄 Usage and License Notices
115
+
116
+ **ViDRiP-LLaVA** (Vision-language Diagnostic Reasoning in Pathology), including its dataset, code, and model checkpoints, is released strictly for **non-commercial research purposes only**.
117
+
118
+ ### 📁 Licenses
119
+
120
+ * **Dataset:**
121
+ Licensed under [**CC BY-NC-ND 3.0**](https://creativecommons.org/licenses/by-nc-nd/3.0/) (Attribution–NonCommercial–NoDerivatives)
122
+ * **Code and pretrained models:**
123
+ Licensed under [**CC BY-NC 3.0**](https://creativecommons.org/licenses/by-nc/3.0/) (Attribution–NonCommercial)
124
+
125
+ ### ⚙️ Dependencies and Components
126
+
127
+ This project may incorporate or build upon resources such as **LLaVA-Next**, **QUILT-1M**, **LLaMA**, **PathAsst**, and **GPT-4**, each subject to their own licenses and **Terms of Use**.
128
+
129
+ ### 🎥 Source Acknowledgment
130
+
131
+ ViDRiP-LLaVA includes data derived from **public educational pathology videos hosted on YouTube**.
132
+ All content usage complies with [**YouTube’s Terms of Service**](https://www.youtube.com/t/terms), and the **intellectual property rights of the original pathologist creators are fully acknowledged and respected**.
133
+
134
+ ### 🚫 Restrictions
135
+
136
+ * Not for **commercial use**
137
+ * Not to be used in **clinical care** or **medical decision-making**
138
+ * For **academic research, development, and evaluation only**