Medical DeepSeek Reasoning Model

This model is a fine-tuned version of DeepSeek-R1-Distill-Llama-8B on medical chain-of-thought reasoning data.

Model Description

Base Model: DeepSeek-R1-Distill-Llama-8B
Fine-tuning Method: LoRA (Low-Rank Adaptation)
Dataset: Medical O1 Reasoning SFT Dataset
Training: 500 samples, 1 epoch
Specialization: Medical reasoning with step-by-step chain-of-thought

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "Soumyajit-7/medical-deepseek-reasoning-8b",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Soumyajit-7/medical-deepseek-reasoning-8b")

# Example usage
prompt = '''Below is an instruction that describes a task, paired with an input that provides further context. 
Write a response that appropriately completes the request. 
Before answering, think carefully about the question and create a step-by-step chain of thoughts to ensure a logical and accurate response.

### Instruction:
You are a medical expert with advanced knowledge in clinical reasoning, diagnostics, and treatment planning. 
Please answer the following medical question. 

### Question:
A 61-year-old woman with a long history of involuntary urine loss during activities like coughing or sneezing but no leakage at night undergoes a gynecological exam and Q-tip test. Based on these findings, what would cystometry most likely reveal about her residual volume and detrusor contractions?

### Response:
<think>'''

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=1200,
    temperature=0.7,
    do_sample=True,
    pad_token_id=tokenizer.eos_token_id
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Training Details

Training Framework: Unsloth
Optimization: LoRA with rank 16
Batch Size: 2 per device, gradient accumulation steps: 4
Learning Rate: 2e-4
Max Sequence Length: 2048
Quantization: 4-bit

Intended Use

This model is designed for educational and research purposes in medical reasoning. It should not be used for actual medical diagnosis or treatment decisions.

Limitations

Limited to 500 training samples
Should not replace professional medical advice
May generate incorrect medical information
Requires careful validation for any medical applications