ShahzebKhoso commited on
Commit
8dca0cc
·
1 Parent(s): d7636f8

Update README.md with new details

Browse files
Files changed (1) hide show
  1. README.md +9 -10
README.md CHANGED
@@ -19,11 +19,11 @@ pipeline_tag: text-generation
19
 
20
  # 🧪 Qwen2.5-0.5B-Instruct + LoRA Fine-Tuned on PubMedQA (pqa_labeled)
21
 
22
- This model is a LoRA-adapted version of [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct), fine-tuned using [Unsloth](https://github.com/unslothai/unsloth) on the `pqa_labeled` subset of the [PubMedQA](https://huggingface.co/datasets/pubmed_qa) dataset.
23
 
24
  ## ✅ Summary
25
 
26
- This work demonstrates that even a compact instruction-tuned model like Qwen2.5 0.5B can achieve near state-of-the-art performance on biomedical QA tasks. With LoRA fine-tuning on just 1,000 examples, this model achieves **98.99% accuracy** on the PubMedQA test set.
27
 
28
  It reframes the classification task as a text generation problem — prompting the model to generate "yes", "no", or "maybe" responses. This results in highly interpretable and efficient predictions with excellent generalization.
29
 
@@ -59,10 +59,9 @@ It reframes the classification task as a text generation problem — prompting t
59
  - `r`: 16
60
  - `alpha`: 16
61
  - `target_modules`: ["q_proj", "v_proj"]
62
- - **Epochs:** [Insert if known]
63
- - **Batch Size:** [Insert if known]
64
- - **Learning Rate:** [Insert if known]
65
- - **NEFTune:** [Insert if used]
66
 
67
  ---
68
 
@@ -73,8 +72,8 @@ from transformers import AutoTokenizer, AutoModelForCausalLM
73
  from peft import PeftModel
74
 
75
  model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
76
- model = PeftModel.from_pretrained(model, "ShahzebKhoso/qwen2.5-pubmedqa-lora")
77
- tokenizer = AutoTokenizer.from_pretrained("ShahzebKhoso/qwen2.5-pubmedqa-lora")
78
  ```
79
 
80
  ---
@@ -94,7 +93,7 @@ tokenizer = AutoTokenizer.from_pretrained("ShahzebKhoso/qwen2.5-pubmedqa-lora")
94
  title={Fine-tuning Qwen2.5-0.5B on PubMedQA with LoRA},
95
  author={Shahzeb Khoso},
96
  year={2025},
97
- howpublished={\\url{https://huggingface.co/ShahzebKhoso/qwen2.5-pubmedqa-lora}},
98
  }
99
  ```
100
 
@@ -103,6 +102,6 @@ tokenizer = AutoTokenizer.from_pretrained("ShahzebKhoso/qwen2.5-pubmedqa-lora")
103
  ## ✨ Acknowledgements
104
 
105
  - [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct)
106
- - [PubMedQA Dataset](https://huggingface.co/datasets/pubmed_qa)
107
  - [Unsloth](https://github.com/unslothai/unsloth)
108
  - [Hugging Face PEFT](https://github.com/huggingface/peft)
 
19
 
20
  # 🧪 Qwen2.5-0.5B-Instruct + LoRA Fine-Tuned on PubMedQA (pqa_labeled)
21
 
22
+ This model is a LoRA-adapted version of [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct), fine-tuned using [Unsloth](https://github.com/unslothai/unsloth) on the `pqa_labeled` subset of the [PubMedQA](https://huggingface.co/datasets/qiaojin/PubMedQA) dataset.
23
 
24
  ## ✅ Summary
25
 
26
+ This work demonstrates that even a compact instruction-tuned model like Qwen2.5 0.5B Instruct can achieve near state-of-the-art performance on biomedical QA tasks. With LoRA fine-tuning using just 1,000 examples, this model achieves **98.99% accuracy** on the PubMedQA test set.
27
 
28
  It reframes the classification task as a text generation problem — prompting the model to generate "yes", "no", or "maybe" responses. This results in highly interpretable and efficient predictions with excellent generalization.
29
 
 
59
  - `r`: 16
60
  - `alpha`: 16
61
  - `target_modules`: ["q_proj", "v_proj"]
62
+ - **Epochs:** 100
63
+ - **Batch Size:** 16
64
+ - **Learning Rate:** 2e-4
 
65
 
66
  ---
67
 
 
72
  from peft import PeftModel
73
 
74
  model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
75
+ model = PeftModel.from_pretrained(model, "ShahzebKhoso/qwen2.5-instruct-0.5B-pubmedqa-lora")
76
+ tokenizer = AutoTokenizer.from_pretrained("ShahzebKhoso/qwen2.5-instruct-0.5B-pubmedqa-lora")
77
  ```
78
 
79
  ---
 
93
  title={Fine-tuning Qwen2.5-0.5B on PubMedQA with LoRA},
94
  author={Shahzeb Khoso},
95
  year={2025},
96
+ howpublished={\\url{https://huggingface.co/ShahzebKhoso/qwen2.5-instruct-0.5B-pubmedqa-lora}},
97
  }
98
  ```
99
 
 
102
  ## ✨ Acknowledgements
103
 
104
  - [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct)
105
+ - [PubMedQA Dataset](https://huggingface.co/datasets/qiaojin/PubMedQA)
106
  - [Unsloth](https://github.com/unslothai/unsloth)
107
  - [Hugging Face PEFT](https://github.com/huggingface/peft)