---
library_name: transformers
tags:
- machine translation
- english-german
- english
- german
- bilingual
license: apache-2.0
datasets:
- rewicks/english-german-data
language:
- en
- de
pipeline_tag: translation
---

# Model Card for Model ID

This model is a simple bilingual English-German machine translation trained with [MarianNMT](https://marian-nmt.github.io/).
They were converted to huggingface using [scripts](https://huggingface.co/Helsinki-NLP/opus-mt-en-zh/discussions/1) derived from the Helsinki-NLP group.
We collected most datasets listed via [mtdata](https://github.com/thammegowda/mtdata) and filtered.
The [processed data](https://huggingface.co/datasets/rewicks/english-german-data) is also available on huggingface.

We trained these models in order to develop a new ensembling algorithm.
**Agreement-Based Ensembling** is an inference-time-only algorithm that allows for ensembling models with different vocabularies, without the ned to learn additional parameters or alter the underlying models.
Instead, the algorithm ensures that tokens generated by the ensembled models _agree_ in their surface form.
For more information, please check out [our code available on GitHub](https://github.com/mjpost/ensemble24), or read our paper on Arxiv.


## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

- **Shared, Developed by:** Rachel Wicks
- **Funded By:** Johns Hopkins University
- **Model type:** Encoder-Decoder (Transformer, Transformer)
- **Language(s) (NLP):** English, German
- **License:** Apache 2.0

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Paper [optional]:** Coming Soon!



## How to Get Started with the Model

The code below can be used to translate lines read from standard input (our baseline in our paper).

```
import sys
import torch

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_id = sys.argv[1]

device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(model_id, torch_dtype=torch.bfloat16)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id).to(device)
model = model.eval()

for line in sys.stdin:
    line = line.strip()
    inputs = tokenizer(line, return_tensors="pt").to(device)
    translated_tokens = model.generate(
        **inputs, max_length=256,
        num_beams = 5,
    )
    print(tokenizer.batch_decode(translated_tokens, skip_special_tokens=True)[0])
```

## Training Details

Data is available [here](https://huggingface.co/datasets/rewicks/english-german-data).
We use [sotastream](https://pypi.org/project/sotastream/) to stream data over stdin.
We use [MarianNMT](https://marian-nmt.github.io/) to train. 
The config is available in the repo as `config.yml`.


## Evaluation

BLEU on WMT24 is XX.

#### Hardware

RTX Titan (24GB)

## Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

[More Information Needed]

**APA:**