---
library_name: transformers
datasets:
- FBK-MT/Speech-MASSIVE
language:
- pl
metrics:
- wer
- bleu
base_model:
- openai/whisper-tiny
pipeline_tag: automatic-speech-recognition
---

# Model Card

## Model Details

### Model Description

This model is a fine-tuned version of OpenAI's Whisper-Tiny ASR model,
optimized for transcribing Polish voice commands. The fine-tuning process 
utilized the MASSIVE Speech dataset to enhance the model's performance
on Polish utterances. The Whisper-Tiny model is a transformer-based
encoder-decoder architecture, pre-trained on 680,000 hours of labeled speech data.

- **Developed by:** gs224
- **Language(s) (NLP):** Polish
- **Finetuned from model:** Whisper-tiny

Link to the training code: https://github.com/gs224/Fine-tuning-Whisper-for-Polish-voice-commands

## Uses

The model can be used for automatic transcription of Polish speech-to-text tasks, including voice command recognition.

### Out-of-Scope Use

The model may not perform well on languages or domains it was not fine-tuned for, and it is not suitable for sensitive applications requiring very high accuracy.

## Bias, Risks, and Limitations

The fine-tuning was performed on a relatively small subset of Polish voice data
with limited epochs, leading to potential underperformance in certain dialects or accents.
The presence of capital letters and punctuation in the ground-truth transcription
may affect the Word Error Rate (WER) score.

### Recommendations

Future improvements could include training on larger datasets, more diverse utterances,
and addressing case sensitivity and punctuation in ground-truth labels.

## Training Details

### Training Data

https://huggingface.co/datasets/FBK-MT/Speech-MASSIVE-test

## Evaluation

Word Error Rate (WER)

### Testing Data, Factors & Metrics


#### Metrics

WER, a typical metrics for ASR.

### Results

Word Error Rate on the test set:

| Base model | Fine-tuned model |
|------------|------------------|
| 0.8435 | 0.3176 |

Example sentences:

| Reference | Base model | Fine-tuned model |
|-----------|------------|------------------|
| wyślij maila do mojego brata i przypomnij o rocznicy ślubu | wysli myę latą mojego biata i przypamni o nici ślubu | wyślij maila do mojego bryata i przypomnij mi o lepszy ślubu |
| przypomnij mi o jutrzejszym spotkaniu godzinę wcześniej | przypomnij mi o jutrzejszym spotkaniu godzinę wcześniej |  przypomnij mi o jutrzejszym spotkaniu godzina wcześniej |
| graj plejlistę boba dylana | gra i play listę boba dylana | graj playlistę boba delana |
| graj ale jazz autorki sanah | grei, al het rust autoorkisana | graj ale jazz autorki sanah |
| olly posłuchajmy sto jeden i trzy f. m. | oli posłuchajmy sto jeden i trzefam | olly posłuchaj we z to jeden i trzy f. m. |