Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,88 @@
|
|
1 |
-
|
2 |
-
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
|
2 |
+
---
|
3 |
+
license: apache-2.0
|
4 |
+
tags:
|
5 |
+
- medical
|
6 |
+
- chest-xray
|
7 |
+
- reasoning
|
8 |
+
- vision-language
|
9 |
+
- medgemma
|
10 |
+
---
|
11 |
+
|
12 |
+
# 🧠 MedGEMMA Reasoning Model — Fine-tuned on CXR-10K
|
13 |
+
|
14 |
+
This is a fine-tuned version of `google/medgemma-4b-it`, trained on the [CXR-10K Reasoning Dataset](https://huggingface.co/datasets/Manusinhh/cxr-10k-reasoning-dataset) consisting of chest X-ray images paired with step-by-step clinical reasoning.
|
15 |
+
|
16 |
+
---
|
17 |
+
|
18 |
+
## 🩻 Task
|
19 |
+
|
20 |
+
**Multimodal Clinical Reasoning**:
|
21 |
+
Given a chest X-ray image, the model generates a step-by-step diagnostic reasoning path covering:
|
22 |
+
|
23 |
+
- Lung fields
|
24 |
+
- Cardiac size
|
25 |
+
- Mediastinal structures
|
26 |
+
- Surgical history
|
27 |
+
- Skeletal findings
|
28 |
+
|
29 |
+
---
|
30 |
+
|
31 |
+
## 🧪 Example Usage (Inference)
|
32 |
+
|
33 |
+
```python
|
34 |
+
from transformers import AutoProcessor, AutoModelForImageTextToText
|
35 |
+
from PIL import Image
|
36 |
+
import torch
|
37 |
+
|
38 |
+
# Load model and processor
|
39 |
+
model = AutoModelForImageTextToText.from_pretrained("Manusinhh/medgemma-finetuned-cxr-reasoning")
|
40 |
+
processor = AutoProcessor.from_pretrained("google/medgemma-4b-it")
|
41 |
+
|
42 |
+
# Load image
|
43 |
+
image = Image.open("example.png").convert("RGB")
|
44 |
+
|
45 |
+
# Create prompt
|
46 |
+
messages = [
|
47 |
+
{
|
48 |
+
"role": "user",
|
49 |
+
"content": [
|
50 |
+
{"type": "image", "image": image},
|
51 |
+
{"type": "text", "text": "Analyze this medical image and provide step-by-step findings."}
|
52 |
+
]
|
53 |
+
}
|
54 |
+
]
|
55 |
+
|
56 |
+
# Tokenize and generate
|
57 |
+
inputs = processor.apply_chat_template(messages, tokenize=True, return_tensors="pt").to(model.device)
|
58 |
+
with torch.no_grad():
|
59 |
+
output = model.generate(**inputs, max_new_tokens=300)
|
60 |
+
|
61 |
+
print(processor.decode(output[0], skip_special_tokens=True))
|
62 |
+
````
|
63 |
+
|
64 |
+
---
|
65 |
+
|
66 |
+
## 📊 Training Details
|
67 |
+
|
68 |
+
* **Base model**: `google/medgemma-4b-it`
|
69 |
+
* **LoRA Fine-tuning**: Used `peft` with low-rank adapters
|
70 |
+
* **Training set**: 10k chest X-ray samples with reasoning steps
|
71 |
+
* **Frameworks**: HuggingFace Transformers, TRL, PEFT, DeepSpeed
|
72 |
+
|
73 |
+
---
|
74 |
+
|
75 |
+
## 📚 Dataset Attribution
|
76 |
+
|
77 |
+
Training data derived from:
|
78 |
+
[CXR-10K Reasoning Dataset](https://huggingface.co/datasets/Manusinhh/cxr-10k-reasoning-dataset)
|
79 |
+
Built upon:
|
80 |
+
[itsanmolgupta/mimic-cxr-dataset-10k](https://huggingface.co/datasets/itsanmolgupta/mimic-cxr-dataset-10k)
|
81 |
+
|
82 |
+
Base dataset:
|
83 |
+
[MIMIC-CXR](https://physionet.org/content/mimic-cxr/2.0.0/) by MIT LCP
|
84 |
+
|
85 |
+
> Johnson AE, Pollard TJ, Berkowitz SJ, et al. *Scientific Data*. 2019;6:317. [https://doi.org/10.1038/s41597-019-0322-0](https://doi.org/10.1038/s41597-019-0322-0)
|
86 |
+
|
87 |
+
```
|
88 |
+
|