Update README.md
Browse files
README.md
CHANGED
@@ -3,13 +3,16 @@ license: apache-2.0
|
|
3 |
library_name: transformers
|
4 |
---
|
5 |
# RadVLM Model Card
|
6 |
-
A Multitask Conversational Vision-Language Model for Radiology (paper: https://arxiv.org/abs/2502.03333)
|
|
|
|
|
|
|
|
|
7 |
|
8 |
# Github repo
|
9 |
The code for data curation, finetuning and evaluation is shared in the following github repo: https://github.com/uzh-dqbm-cmi/RadVLM.git
|
10 |
|
11 |
|
12 |
-
|
13 |
## Model Development
|
14 |
|
15 |
- **Developed by**: KrauthammerLab, University of Zurich, ETH Zurich, Kyoto University of Applied Science, Kobe University, Swiss AI Initiative
|
@@ -163,6 +166,7 @@ def inference_radvlm(model, processor, image, prompt, chat_history=None, max_new
|
|
163 |
|
164 |
## Quick-Start: Multi-turn Demo
|
165 |
Below is a demonstration of how to utilize the inference_radvlm function in a multi-turn conversation.
|
|
|
166 |
|
167 |
```python
|
168 |
import torch
|
@@ -172,8 +176,8 @@ import requests
|
|
172 |
from io import BytesIO
|
173 |
import numpy as np
|
174 |
|
175 |
-
Initialize the model and processor
|
176 |
-
model_id = "
|
177 |
model = LlavaOnevisionForConditionalGeneration.from_pretrained(
|
178 |
model_id,
|
179 |
torch_dtype=torch.float16,
|
|
|
3 |
library_name: transformers
|
4 |
---
|
5 |
# RadVLM Model Card
|
6 |
+
A Multitask Conversational Vision-Language Model for Radiology (paper: https://arxiv.org/abs/2502.03333).
|
7 |
+
|
8 |
+
Here, we provide the link to access RadVLM github repository and the inference code to use RadVLM once trained following the repo's instructions.
|
9 |
+
Instruction dataset will be shared in the future on the PhysioNet platform.
|
10 |
+
|
11 |
|
12 |
# Github repo
|
13 |
The code for data curation, finetuning and evaluation is shared in the following github repo: https://github.com/uzh-dqbm-cmi/RadVLM.git
|
14 |
|
15 |
|
|
|
16 |
## Model Development
|
17 |
|
18 |
- **Developed by**: KrauthammerLab, University of Zurich, ETH Zurich, Kyoto University of Applied Science, Kobe University, Swiss AI Initiative
|
|
|
166 |
|
167 |
## Quick-Start: Multi-turn Demo
|
168 |
Below is a demonstration of how to utilize the inference_radvlm function in a multi-turn conversation.
|
169 |
+
For this you need to set the variable `model_id` with the path containing the model weights.
|
170 |
|
171 |
```python
|
172 |
import torch
|
|
|
176 |
from io import BytesIO
|
177 |
import numpy as np
|
178 |
|
179 |
+
# Initialize the model and processor
|
180 |
+
model_id = "your/local/folder/with/RadVLM/weights"
|
181 |
model = LlavaOnevisionForConditionalGeneration.from_pretrained(
|
182 |
model_id,
|
183 |
torch_dtype=torch.float16,
|