LYRICAL Russian to English Machine Translation Model
By Silver Age Poets
Variant 0.1: ORPO-tuned Qwen2.5-3B Model Weights
Fine-tuned leveraging Odds Ratio Preference Optimization (ORPO) on our Russian-to-English song lyrics translation/localization dataset.
Our aim with this work is to foster a translation model capable of adaptively localizing idiomatic, formal/poetic/rhythmic, and performance-catered features of lyrical input texts, whilst retaining adequate accuracy at the level of direct semantic translation.
The training was held for 6 Epochs at Rank 128.
This is an experimental/WIP fine-tune, and probably first of many.
We find that its zero-shot translations (without extensive specialized instruction templating/priming) are already generally more reminiscent of input songs' phrasal rhythm/meter and expressive tone, as compared to base model outputs.
However, it makes far more semantic errors/mis-attributions and/or arbitrary substitutions (straying from translation into extrapolative composition).
We would appreciate feedback/reports from anyone else who happens to try out this model, or its other variants (to be released in the near future).
Next, we will attempt to fine-tune a larger or/and an already translation-specialized model, and continue experimenting with hyperparameters/techniques.
- Downloads last month
- 5