bioner_medmentions_st21pv / performance_report.md
jakelever's picture
Upload folder using huggingface_hub
c2e7c4a verified

Performance on Training Set

Span Level

Label Precision Recall F1-score Support
Anatomy 0.870 0.872 0.871 11509
Chemicals & Drugs 0.902 0.889 0.895 22432
Concepts & Ideas 0.790 0.630 0.701 10273
Devices 0.805 0.791 0.798 1170
Disorders 0.847 0.817 0.832 24634
Genes & Molecular Sequences 0.809 0.851 0.829 3382
Geographic Areas 0.876 0.907 0.892 1712
Living Beings 0.888 0.888 0.888 11388
Objects 0.806 0.886 0.844 851
Occupations 0.576 0.709 0.636 522
Organizations 0.797 0.885 0.839 1308
Phenomena 0.448 0.469 0.458 1131
Physiology 0.784 0.789 0.787 11726
Procedures 0.773 0.801 0.787 20145
macro avg 0.784 0.799 0.790 122183
weighted avg 0.833 0.820 0.826 122183

Token Level

Label Precision Recall F1-score Support
O 0.972 0.976 0.974 557052
B-Anatomy 0.919 0.913 0.916 11456
I-Anatomy 0.917 0.945 0.931 9946
B-Chemicals & Drugs 0.943 0.934 0.938 22294
I-Chemicals & Drugs 0.963 0.955 0.959 31036
B-Concepts & Ideas 0.877 0.652 0.748 10226
I-Concepts & Ideas 0.884 0.856 0.869 8750
B-Devices 0.868 0.801 0.833 1163
I-Devices 0.850 0.886 0.868 1602
B-Disorders 0.900 0.850 0.875 24455
I-Disorders 0.897 0.907 0.902 22954
B-Genes & Molecular Sequences 0.868 0.905 0.886 3373
I-Genes & Molecular Sequences 0.891 0.935 0.913 5266
B-Geographic Areas 0.917 0.950 0.933 1702
I-Geographic Areas 0.937 0.934 0.936 1370
B-Living Beings 0.925 0.935 0.930 11285
I-Living Beings 0.958 0.963 0.960 12389
B-Objects 0.857 0.956 0.903 832
I-Objects 0.904 0.936 0.920 705
B-Occupations 0.649 0.759 0.699 518
I-Occupations 0.600 0.756 0.669 393
B-Organizations 0.837 0.916 0.874 1296
I-Organizations 0.857 0.958 0.904 1766
B-Phenomena 0.541 0.513 0.527 1124
I-Phenomena 0.582 0.486 0.530 989
B-Physiology 0.849 0.819 0.834 11666
I-Physiology 0.810 0.881 0.844 8875
B-Procedures 0.829 0.844 0.837 20022
I-Procedures 0.891 0.896 0.893 20889
macro avg 0.851 0.863 0.855 805394
weighted avg 0.949 0.949 0.949 805394

Performance on Validation Set

Span Level

Label Precision Recall F1-score Support
Anatomy 0.682 0.697 0.690 3881
Chemicals & Drugs 0.751 0.757 0.754 7490
Concepts & Ideas 0.487 0.339 0.400 3365
Devices 0.606 0.436 0.507 493
Disorders 0.700 0.646 0.672 8320
Genes & Molecular Sequences 0.546 0.580 0.563 965
Geographic Areas 0.747 0.760 0.753 678
Living Beings 0.716 0.724 0.720 4228
Objects 0.552 0.656 0.600 259
Occupations 0.466 0.480 0.473 198
Organizations 0.502 0.578 0.537 453
Phenomena 0.277 0.321 0.297 274
Physiology 0.559 0.566 0.562 3751
Procedures 0.588 0.600 0.594 6509
macro avg 0.584 0.582 0.580 40864
weighted avg 0.650 0.633 0.640 40864

Token Level

Label Precision Recall F1-score Support
O 0.931 0.947 0.939 187057
B-Anatomy 0.761 0.754 0.757 3862
I-Anatomy 0.720 0.775 0.747 3271
B-Chemicals & Drugs 0.821 0.823 0.822 7409
I-Chemicals & Drugs 0.856 0.847 0.851 10003
B-Concepts & Ideas 0.590 0.360 0.447 3342
I-Concepts & Ideas 0.556 0.459 0.503 2857
B-Devices 0.696 0.443 0.542 492
I-Devices 0.628 0.554 0.588 688
B-Disorders 0.770 0.688 0.727 8238
I-Disorders 0.729 0.711 0.720 7334
B-Genes & Molecular Sequences 0.643 0.656 0.649 957
I-Genes & Molecular Sequences 0.677 0.710 0.693 1529
B-Geographic Areas 0.793 0.809 0.801 674
I-Geographic Areas 0.746 0.768 0.756 542
B-Living Beings 0.795 0.801 0.798 4185
I-Living Beings 0.844 0.820 0.832 4850
B-Objects 0.610 0.715 0.658 256
I-Objects 0.675 0.672 0.674 244
B-Occupations 0.521 0.508 0.514 197
I-Occupations 0.474 0.364 0.412 173
B-Organizations 0.579 0.650 0.613 443
I-Organizations 0.640 0.711 0.673 622
B-Phenomena 0.341 0.343 0.342 274
I-Phenomena 0.285 0.321 0.302 212
B-Physiology 0.640 0.604 0.621 3726
I-Physiology 0.549 0.574 0.561 2688
B-Procedures 0.661 0.651 0.656 6454
I-Procedures 0.679 0.662 0.671 6567
macro avg 0.662 0.645 0.651 269146
weighted avg 0.869 0.873 0.870 269146

Performance on Testing Set

Span Level

Label Precision Recall F1-score Support
Anatomy 0.656 0.672 0.664 3277
Chemicals & Drugs 0.748 0.745 0.747 7398
Concepts & Ideas 0.515 0.370 0.430 3683
Devices 0.447 0.372 0.406 355
Disorders 0.691 0.641 0.665 8109
Genes & Molecular Sequences 0.506 0.567 0.535 1115
Geographic Areas 0.671 0.737 0.703 598
Living Beings 0.718 0.739 0.728 3994
Objects 0.518 0.598 0.555 336
Occupations 0.367 0.480 0.416 196
Organizations 0.504 0.634 0.561 382
Phenomena 0.206 0.271 0.234 269
Physiology 0.560 0.582 0.571 3833
Procedures 0.597 0.607 0.602 6599
macro avg 0.550 0.573 0.558 40144
weighted avg 0.641 0.630 0.634 40144

Token Level

Label Precision Recall F1-score Support
O 0.933 0.945 0.939 188640
B-Anatomy 0.733 0.725 0.729 3258
I-Anatomy 0.715 0.766 0.740 3035
B-Chemicals & Drugs 0.814 0.803 0.808 7363
I-Chemicals & Drugs 0.825 0.843 0.834 9659
B-Concepts & Ideas 0.625 0.382 0.474 3661
I-Concepts & Ideas 0.562 0.511 0.535 3083
B-Devices 0.583 0.367 0.451 354
I-Devices 0.465 0.451 0.458 566
B-Disorders 0.765 0.681 0.721 8047
I-Disorders 0.738 0.703 0.720 7601
B-Genes & Molecular Sequences 0.608 0.665 0.635 1111
I-Genes & Molecular Sequences 0.607 0.656 0.631 1694
B-Geographic Areas 0.729 0.810 0.767 594
I-Geographic Areas 0.720 0.684 0.702 557
B-Living Beings 0.787 0.801 0.794 3984
I-Living Beings 0.837 0.815 0.826 4514
B-Objects 0.589 0.645 0.615 335
I-Objects 0.698 0.642 0.669 299
B-Occupations 0.444 0.549 0.491 195
I-Occupations 0.379 0.508 0.434 130
B-Organizations 0.551 0.675 0.607 382
I-Organizations 0.595 0.802 0.683 511
B-Phenomena 0.292 0.321 0.306 265
I-Phenomena 0.272 0.340 0.302 209
B-Physiology 0.642 0.614 0.628 3814
I-Physiology 0.552 0.625 0.586 2805
B-Procedures 0.675 0.657 0.666 6558
I-Procedures 0.707 0.669 0.687 6928
macro avg 0.636 0.643 0.636 270152
weighted avg 0.869 0.871 0.869 270152