🧠 ICLviaQE

This repository contains the data, code, and models required to replicate the experiments from our paper:

Guiding In-Context Learning of LLMs through Quality Estimation for Machine Translation
Javad Pourmostafa R. Sh., Dimitar Shterionov, Pieter Spronck
Accepted at AMTA 2024
📄 Read the paper on ACL Anthology

---

📘 Summary

The quality of large language model (LLM) outputs in machine translation (MT) strongly depends on the in-context examples (ICEs) provided along with the input text.
Selecting the most effective examples typically requires reference translations or human judgment — both costly and impractical at scale.

Our method, ICLviaQE, introduces a quality-estimation-guided in-context learning framework that selects and orders examples based on predicted translation quality, without using reference translations.
By leveraging XGLM as a QE estimator, the system automatically identifies the most beneficial examples, leading to consistent improvements across domains and outperforming fine-tuned mBART-50 baselines.

---

🧩 Methodology Overview

Below is a high-level summary of our approach:

🎥 Watch our 5-minute video presentation

---

⚙️ Methodology Breakdown

For full implementation details and code for all stages and baselines, please refer to our GitHub repository:

👉 ICLviaQE on GitHub

---

🧪 Baselines

Baseline	Description	Code
Random	Randomly selects examples for ICL.	random_file.py → run_generation.py
Task-level	Uses task-level contextual examples.	create_task_file.py → run_generation.py
BM25	Retrieves similar pairs using BM25.	create_BM25_file.py
R-BM25	Enhanced BM25 version (external).	R-BM25 Repository
mBART-50	Fine-tuned reference model.	mBART-50 Repository

---

🧾 Citation

If you use this work, please cite:

@inproceedings{pourmostafa-roshan-sharami-etal-2024-guiding,
  title     = "Guiding In-Context Learning of {LLM}s through Quality Estimation for Machine Translation",
  author    = "Pourmostafa Roshan Sharami, Javad and Shterionov, Dimitar and Spronck, Pieter",
  booktitle = "Proceedings of the 16th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)",
  month     = sep,
  year      = "2024",
  address   = "Chicago, USA",
  publisher = "Association for Machine Translation in the Americas",
  url       = "https://aclanthology.org/2024.amta-research.9",
  pages     = "88--101"
}