MedWhisper: Fine-Tuned Whisper-Small for Medical Transcription

MedWhisper is a fine-tuned version of OpenAI's Whisper-small model, optimized specifically for automatic speech recognition (ASR) in the medical domain.

Model Overview

MedWhisper builds upon the strong baseline of Whisper-small and adapts it for medical speech, achieving significant reductions in error rates on evaluation data. This makes it well-suited for applications such as medical dictation, clinical note transcription, and other healthcare communication tasks.

Performance Metrics on Evaluation Set

Metric	Value
Loss	0.0303
CER (Character Error Rate)	1.90%
WER (Word Error Rate)	2.65%
SER (Sentence Error Rate)	10.68%

These metrics demonstrate the model's high transcription accuracy in the medical domain.

Training Details

Base model: openai/whisper-small
Dataset: Custom medical transcription corpus (proprietary)
Epochs: 50
Batch size: 8 (train and eval), with gradient accumulation to total batch size 64
Optimizer: AdamW (betas=(0.9, 0.999), epsilon=1e-8)
Learning rate: 1e-5 with linear scheduler and 500 warmup steps
Mixed precision: Native AMP (Automatic Mixed Precision)
Seed: 42

Training Progress Highlights

Step	Epoch	Train Loss	Val Loss	CER (%)	WER (%)	SER (%)
1000	3.6	0.0105	0.0343	3.23	4.56	17.58
2000	7.3	0.0022	0.0307	1.77	2.59	12.73
8000	29.1	0.0002	0.0289	1.75	2.47	10.91
13000	47.3	0.0001	0.0303	1.90	2.65	10.68

Intended Uses & Limitations

Intended uses:

Accurate transcription of medical audio recordings
Integration in clinical documentation workflows
Supporting healthcare providers with efficient speech-to-text tools

Limitations:

Performance depends on audio quality and domain-specific vocabulary coverage
May require further adaptation for non-English or highly specialized medical subdomains

Framework & Dependencies

Transformers 4.54.0.dev0
PyTorch 2.7.1+cu126
Datasets 3.6.0
Tokenizers 0.21.2

Acknowledgments

medwhisper is provided under the MIT license. If you use this model in your work, please acknowledge the creators and consider referencing this model as follows:

@model{medwhisper,
title = {medwhisper Model},
author = {Mahwiz Khalil},
year = {2025},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/mahwizzzz/medwhishper}},
}

mahwizzzz
/

medwhishper