LumiOpen/Poro-34B-chat-OpenAssistant

This is an SFT-tuned model of Poro-34B with English and Finnish data. We trained this model as part of our experiments on the impact of multilingual instruction-tuning on Poro-34B. For a better chat experience, we recommend using Poro-34B-chat instead.

Datasets

SFT

We use a curated subset of Open Assistant 2 and translated the dataset into Finnish using Poro-34B.

• English: oasst2_curated

• Finnish: instruction-collection-fin (oasst2 subset)

DPO

We use the HelpSteer2 preference binarized into chosen-rejected pairs using the helpfulness score as recommended in the HelpSteer2 paper. We translated the dataset into Finnish using Poro.

• English: HelpSteer2

• Finnish: TBA

Recipes

For finetuning, we used 4 nodes (8 x AMD MI250X) to obtain a global batch size of 128 for SFT and 64 for DPO. We used the Alignment Handbook codebase.

SFT

bf16: true
do_eval: true
evaluation_strategy: epoch
gradient_accumulation_steps: 2
gradient_checkpointing: true
gradient_checkpointing_kwargs:
  use_reentrant: False
learning_rate: 2.0e-05
log_level: info
logging_steps: 50  
logging_strategy: steps
lr_scheduler_type: cosine
max_seq_length: 2048
max_steps: -1
num_train_epochs: 3
output_dir: data/poro-sft-oasst2
overwrite_output_dir: true
per_device_eval_batch_size: 4
per_device_train_batch_size: 2
remove_unused_columns: true
save_strategy: "epoch"
save_total_limit: 1
seed: 42
warmup_ratio: 0.1

DPO

bf16: true
beta: 0.05
do_eval: true
evaluation_strategy: epoch
gradient_accumulation_steps: 1
gradient_checkpointing: true
gradient_checkpointing_kwargs:
  use_reentrant: False
learning_rate: 5.0e-7
log_level: info
logging_steps: 20
lr_scheduler_type: cosine
max_length: 1024
max_prompt_length: 512
num_train_epochs: 5
optim: adamw_torch
output_dir: data/poro-dpo-helpsteer2
per_device_train_batch_size: 2
per_device_eval_batch_size: 4
save_strategy: "epoch"
save_total_limit: 1
seed: 42
warmup_ratio: 0.1

Evaluation

We use IFEval to evaluate the performance of the model in English. For Finnish, we translated the IFEval prompts to Finnish with DeepL. We report the instruction-level strict accuracy:

• English: 0.3997
• Finnish: 0.3448

Citation

We discuss our experimental setup and results in our NoDaLiDa 2025 paper.

@inproceedings{
zosa2024got,
title={Got Compute, but No Data: Lessons From Post-training a Finnish {LLM}},
author={Elaine Zosa and Ville Komulainen and Sampo Pyysalo},
booktitle={The Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies},
year={2024},
url={https://openreview.net/forum?id=8wWlu1stNK}
}