NicoZenith commited on
Commit
c8d5503
·
verified ·
1 Parent(s): 4e93b2b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -4
README.md CHANGED
@@ -3,13 +3,16 @@ license: apache-2.0
3
  library_name: transformers
4
  ---
5
  # RadVLM Model Card
6
- A Multitask Conversational Vision-Language Model for Radiology (paper: https://arxiv.org/abs/2502.03333)
 
 
 
 
7
 
8
  # Github repo
9
  The code for data curation, finetuning and evaluation is shared in the following github repo: https://github.com/uzh-dqbm-cmi/RadVLM.git
10
 
11
 
12
-
13
  ## Model Development
14
 
15
  - **Developed by**: KrauthammerLab, University of Zurich, ETH Zurich, Kyoto University of Applied Science, Kobe University, Swiss AI Initiative
@@ -163,6 +166,7 @@ def inference_radvlm(model, processor, image, prompt, chat_history=None, max_new
163
 
164
  ## Quick-Start: Multi-turn Demo
165
  Below is a demonstration of how to utilize the inference_radvlm function in a multi-turn conversation.
 
166
 
167
  ```python
168
  import torch
@@ -172,8 +176,8 @@ import requests
172
  from io import BytesIO
173
  import numpy as np
174
 
175
- Initialize the model and processor
176
- model_id = "KrauthammerLab/RadVLM"
177
  model = LlavaOnevisionForConditionalGeneration.from_pretrained(
178
  model_id,
179
  torch_dtype=torch.float16,
 
3
  library_name: transformers
4
  ---
5
  # RadVLM Model Card
6
+ A Multitask Conversational Vision-Language Model for Radiology (paper: https://arxiv.org/abs/2502.03333).
7
+
8
+ Here, we provide the link to access RadVLM github repository and the inference code to use RadVLM once trained following the repo's instructions.
9
+ Instruction dataset will be shared in the future on the PhysioNet platform.
10
+
11
 
12
  # Github repo
13
  The code for data curation, finetuning and evaluation is shared in the following github repo: https://github.com/uzh-dqbm-cmi/RadVLM.git
14
 
15
 
 
16
  ## Model Development
17
 
18
  - **Developed by**: KrauthammerLab, University of Zurich, ETH Zurich, Kyoto University of Applied Science, Kobe University, Swiss AI Initiative
 
166
 
167
  ## Quick-Start: Multi-turn Demo
168
  Below is a demonstration of how to utilize the inference_radvlm function in a multi-turn conversation.
169
+ For this you need to set the variable `model_id` with the path containing the model weights.
170
 
171
  ```python
172
  import torch
 
176
  from io import BytesIO
177
  import numpy as np
178
 
179
+ # Initialize the model and processor
180
+ model_id = "your/local/folder/with/RadVLM/weights"
181
  model = LlavaOnevisionForConditionalGeneration.from_pretrained(
182
  model_id,
183
  torch_dtype=torch.float16,