Add evaluation results on the kmfoda--booksum config and test split of kmfoda/booksum
#1
by
autoevaluator
HF staff
- opened
Beep boop, I am a bot from Hugging Face's automatic model evaluator 👋!
Your model has been evaluated on the kmfoda--booksum config and test split of the kmfoda/booksum dataset by
@pszemraj
, using the predictions stored here.
Accept this pull request to see the results displayed on the Hub leaderboard.
Evaluate your model on more datasets here.
pszemraj
changed pull request status to
merged
Based on running more detailed predictions myself with the trainer, I think something is wrong with these and will remove them for now.
The command for this and details are here (to see metrics on predict you will have to scroll through the logs/terminal page of wandb)
***** predict metrics *****
predict_gen_len = 506.4368
predict_loss = 2.028
predict_rouge1 = 36.8815
predict_rouge2 = 8.0625
predict_rougeL = 17.6161
predict_rougeLsum = 34.9068
predict_runtime = 2:04:14.37
predict_samples = 1431
predict_samples_per_second = 0.192
predict_steps_per_second = 0.048