Model Card for DeepSeek-R1-Distill-Qwen-1.5B Fine-tuned on PubMedQA

🧠 deepseek-medical-lora

A LoRA-finetuned variant of DeepSeek-Medical for medical QA and summarization tasks.

Model Details

Model Description

This model is a LoRA fine-tuned version of deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B on the PubMedQA - pqa_labeled dataset. It was trained for 1 epoch on 1,000 biomedical QA pairs to generate Yes/No/Maybe style answers from PubMed article abstracts.

Developed by: Abdul Moid
Shared by: Abdul Moid
Model type: Causal Language Model with LoRA adapters
Language(s): English (biomedical domain)
License: [More Information Needed]
Base model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B

Model Sources [optional]

Repository: [More Information Needed]
Paper [optional]: [More Information Needed]
Demo [optional]: [More Information Needed]

Uses

Biomedical question answering
Literature-based evidence QA

Direct Use

Clinical research assistance
PubMed-style abstract summarization for QA [More Information Needed]

Intended Tasks

Clinical research assistance
PubMed-style abstract summarization for QA

Downstream Use [optional]

Clinical decision-support systems (non-diagnostic)
Biomedical search/chatbots

[More Information Needed]

Out-of-Scope Use

Not for real-time diagnosis or patient care
Not tested on full-text clinical records [More Information Needed]

Bias, Risks, and Limitations

Reflects bias from PubMed abstracts and annotations
Limited to biomedical QA domain
Use with medical oversight

Recommendations

Validate outputs with domain experts
Avoid usage for diagnostic or treatment decisions

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("10mabdulmoid/deepseek-medical-lora")
model = AutoModelForCausalLM.from_pretrained("10mabdulmoid/deepseek-medical-lora")
model.to("cuda" if torch.cuda.is_available() else "cpu")

question = "Is aspirin effective for migraine prevention?"
context = "...abstract from PubMed article..."
prompt = f"Question: {question}\nContext: {context}\nAnswer:"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Procedure

Fine-tuned for 1 epoch using LoRA adapters.

Preprocessing [optional]

Concatenated question and context for input.
Used long_answer and final_decision fields.

Training Hyperparameters

Batch Size: 8
Epochs: 1
Optimizer: AdamW
Learning Rate: 2e-5

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

Evaluation Set: 20% of PubMedQA labeled split
Metric: Accuracy on "yes", "no", "maybe" classification
Results: [More Information Needed] [More Information Needed]

Factors

Biomedical domain only [More Information Needed]

Metrics

Accuracy
F1 Score (optional) [More Information Needed]

Results

[More Information Needed]

Summary

Biomedical QA model using DeepSeek + LoRA + PubMedQA. Quick to adapt and fine-tune further.

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: NVIDIA A100 40GB
Hours used: ~1 hour
Cloud Provider: Google Cloud
Compute Region: us-central1
Carbon Emitted: Use calculator

Technical Specifications [optional]

Model Architecture and Objective

Transformer + LoRA adapters

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

@misc{deepseekmedical-lora,
  title={DeepSeek Medical - LoRA Fine-Tuned Model},
  author={Abdul Moid},
  howpublished={\url{https://huggingface.co/10mabdulmoid/deepseek-medical-lora}},
  year={2025}
}

[More Information Needed]

APA:

[More Information Needed]

📞 Contact

For issues, contact huggingface.co/10mabdulmoid

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

Abdul Moid

Model Card Contact

[More Information Needed]