compas-prm-correctness-full-meta8B

This model is a fine-tuned version of zarahall/bias-prm-v3 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7259

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• gradient_accumulation_steps: 4
• total_train_batch_size: 8
• total_eval_batch_size: 2
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.15
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
0.9307	0.1352	100	0.8397
0.8305	0.2704	200	0.8036
0.7258	0.4055	300	0.7699
0.7102	0.5407	400	0.7480
0.7924	0.6759	500	0.7370
0.7386	0.8111	600	0.7322
0.7283	0.9463	700	0.7294
0.706	1.0814	800	0.7278
0.7382	1.2166	900	0.7272
0.7837	1.3518	1000	0.7263
0.8122	1.4870	1100	0.7263
0.7265	1.6222	1200	0.7260
0.7012	1.7574	1300	0.7258
0.7828	1.8925	1400	0.7259

Framework versions

• Transformers 4.43.3
• Pytorch 2.1.2+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1