COMET-poly-cand2-wmt25

This model is based on COMET-poly, which is a fork but not compatible with original Unbabel's COMET. To run the model, you need to first install this version of COMET either with:

pip install "git+https://github.com/zouharvi/COMET-poly#egg=comet-poly&subdirectory=comet_poly"

or in editable mode:

git clone https://github.com/zouharvi/COMET-poly.git
cd COMET-poly
pip3 install -e comet_poly

This model scores the translation mt but takes additional translations mt2 and mt3 of the same source src for more context, which makes it a better quality estimator:

import comet_poly
model = comet_poly.load_from_checkpoint(comet_poly.download_model("zouharvi/COMET-poly-cand2-wmt25"))
data = [
    {
        "src": "Iceberg lettuce got its name in the 1920s when it was shipped packed in ice to stay fresh.",
        "mt": "Eisbergsalat erhielt seinen Namen in den 1920er-Jahren, als er in Eis verpackt verschickt wurde, um frisch zu bleiben.",
        "mt2": "Eisbergsalat bekam seinen Namen, weil man ihn mit Eis schickte, damit er frisch bleibt.",
        "mt3": "Seinen Namen erhielt der Eisbergsalat in den 1920er-Jahren, weil er damals in Eis verpackt transportiert wurde, um frisch zu bleiben.",
    },
    {
        "src": "Goats have rectangular pupils, which give them a wide field of vision—up to 320 degrees!",
        "mt": "Kozy mají obdélníkové zornice, což jim umožňuje vidět skoro všude kolem sebe, aniž by musely otáčet hlavou.",
        "mt2": "Kozy mají obdélníkové zornice, které jim umožňují mít zorné pole až 320 stupňů.",
        "mt3": "Kozy mají obdélníkové zorničky, které jim umožňují velmi široké zorné pole – až 320 stupňů!",
    },
    {
        "src": "This helps them spot predators from almost all directions without moving their heads.",
        "mt": "Điều này giúp chúng phát hiện kẻ săn mồi từ gần như mọi hướng mà không cần quay đầu.",
        "mt2": "Nhờ vậy, chúng có thể thấy kẻ săn mồi từ hầu hết mọi phía mà không phải xoay đầu.",
        "mt3": "Điều này giúp họ nhìn thấy kẻ ăn thịt từ hầu như mọi chỗ mà không cần lắc đầu.",
    }
]
print("scores", model.predict(data, batch_size=8, gpus=1).scores)

Outputs:

scores [98.503173828125, 83.22859954833984, 88.41258239746094]

The training data is WMT up to 2024 (inclusive) with DA/ESA/MQM merged on a single scale. This model is based on the work TODO which can be cited as:

@misc{zuefle2025comet,
    title={COMET-poly: Machine Translation Metric Grounded in Other Candidates},
    author={Maike Züfle, Vilém Zouhar, Tu Anh Dinh, Felipe Polo, Jan Niehues, Mrinmaya Sachan},
    year={2025},
}