voxreality/nllb-asr-synthetic-robust

NLLB-600M Finetuned for Robustness

We finetuned NLLB-200-distilled-600M using adapters for robustness to ASR and synthetic non-native speakers noise. This model can only be used for translating from en to de, it, es, nl and el.

How to use

Start by installing transformers with NLLB-200 model with added adapters

git clone https://gitlab.com/horizon-europe-voxreality/multilingual-translation/speech-translation-demo.git
cd speech-translation-demo
# You might need to switch to dev branch
pip install -e transformers

And now we can use the model:

model_name = 'voxreality/nllb-asr-synthetic-robust'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = M2M100ForConditionalGenerationWithAdapters.from_pretrained(model_name)

src_lang = 'eng_Latn'
tgt_lang = 'deu_Latn'

input_text = "This is a good day"

tokenizer.src_lang = src_lang
inputs = tokenizer(input_text, return_tensors='pt').to(model.device)
model_output = model.generate(**inputs,
                              forced_bos_token_id=tokenizer.lang_code_to_id[tgt_lang])
output_text = tokenizer.batch_decode(model_output, skip_special_tokens=True)[0]

print(output_text)