HPLT

https://hplt-project.org/

Our project name, HPLT, is an acronym for High Performance Language Technologies. We are aiming high at combining large quantities of data, a number of languages and high-performance computing to build powerful and efficient language and translation models. Another goal of HPLT is to publish the results of this project in a shared space with open licenses.

• HPLT datasets paper
• Version 2 of the HPLT datasets (193 languages):
  • https://hplt-project.org/datasets/v2.0
  • https://hf.co/datasets/HPLT/HPLT2.0_cleaned
• Version 1.2 of the HPLT datasets (75 languages):
  • https://hplt-project.org/datasets/v1.2
  • https://huggingface.co/datasets/HPLT/hplt_monolingual_v1_2

This project has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No 101070350 and from UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee [grant number 10052546]