π§ Qwen3-50M Insomnia Therapist
Fine-tuned version of Qwen3-50M specialized for insomnia therapy conversations with Chain of Thought reasoning.
π― Model Details
- Base Model: Mostafa8Mehrabi/qwen3-50m-fp16
- Fine-tuned on: Mostafa8Mehrabi/insomnia-dataset-with-cot
- Precision: BF16
- Model Size: ~50M parameters
- Specialization: Insomnia therapy with Chain of Thought reasoning
π Training Results
- Final Training Loss: 1.1862
- Final Validation Loss: 1.2026013135910034
- Training Epochs: 3
- Batch Size: 4
- Learning Rate: 2e-05
- Max Length: 1024
- Precision Used: BF16
π Usage
Quick Start
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("Mostafa8Mehrabi/qwen3-50m-insomnia-therapist")
model = AutoModelForCausalLM.from_pretrained(
"Mostafa8Mehrabi/qwen3-50m-insomnia-therapist",
device_map="auto"
)
# Generate therapy response
prompt = '''<|system|>Generate insomnia therapy conversation. Gather sleep information, establish rapport, introduce sleep diary. Include CoT reasoning. Validate-Educate-Recommend-Check format.<|endoftext|>
<|patient|>I can't sleep at night. I keep thinking about work and worrying about tomorrow.<|endoftext|>
<|cot|>'''
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
outputs = model.generate(
**inputs,
max_length=400,
temperature=0.7,
top_p=0.9,
do_sample=True,
pad_token_id=tokenizer.pad_token_id,
eos_token_id=tokenizer.eos_token_id
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
π Format Structure
The model follows a specific conversation format:
- Input:
<|system|>
+<|patient|>
- Output:
<|cot|>
(Chain of Thought reasoning) +<|therapist|>
(Therapeutic response)
Format Example:
<|system|>Generate insomnia therapy conversation...<|endoftext|>
<|patient|>I can't sleep at night.<|endoftext|>
<|cot|>Patient reports sleep difficulty. Need to validate concern, gather more information about sleep patterns, potential causes, and establish therapeutic rapport. Should introduce sleep diary concept and provide initial guidance.<|endoftext|>
<|therapist|>I understand how frustrating it must be to struggle with sleep. This is a common issue that many people face, and I'm here to help you work through it. Can you tell me a bit more about your sleep patterns? For instance, how long have you been experiencing this difficulty, and what typically happens when you try to fall asleep?<|endoftext|>
π― Key Features
- Chain of Thought Reasoning: The model provides transparent reasoning before generating responses
- Therapeutic Approach: Follows evidence-based therapy principles
- Validation-Education-Recommendation-Check: Structured therapeutic format
- Optimized Training: Trained with BF16 precision for efficiency
- Specialized Training: Fine-tuned specifically on insomnia therapy conversations
βοΈ Technical Specifications
- Architecture: Qwen3 (Transformer-based)
- Parameters: ~50M
- Training Precision: BF16
- Context Length: 1024 tokens
- Training Framework: PyTorch + Transformers
- Optimization: AdamW with warmup
π§ Hardware Requirements
- Minimum: 2GB GPU VRAM
- Recommended: 4GB+ GPU VRAM
- CPU: Compatible with CPU inference (slower)
π Training Data
Trained on curated insomnia therapy conversations with Chain of Thought annotations from the Mostafa8Mehrabi/insomnia-dataset-with-cot dataset.
π¨ Important Notes
- This model is for educational and research purposes
- Not a replacement for professional medical advice
- Always consult healthcare professionals for serious sleep disorders
- Model outputs should be reviewed by qualified therapists
π License
Apache 2.0 License
π€ Citation
If you use this model in your research, please cite:
@misc{qwen3-50m-insomnia-therapist,
title={Qwen3-50M Insomnia Therapist},
author={Mostafa8Mehrabi},
year={2024},
publisher={Hugging Face},
url={https://huggingface.co/Mostafa8Mehrabi/qwen3-50m-insomnia-therapist}
}
Fine-tuned with β€οΈ for better sleep health
- Downloads last month
- 18
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
π
Ask for provider support