Upload folder using huggingface_hub

README.md

@@ -0,0 +1,82 @@
+---
+task: token-classification
+tags:
+- biomedical
+- bionlp
+license: mit
+base_model: microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext
+---
+
+# bioner_medmentions_st21pv
+
+This is a named entity recognition model fine-tuned from the [microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext](https://huggingface.co/microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext) model. It predicts spans with 14 possible labels. The labels are **Anatomy, Chemicals & Drugs, Concepts & Ideas, Devices, Disorders, Genes & Molecular Sequences, Geographic Areas, Living Beings, Objects, Occupations, Organizations, Phenomena, Physiology and Procedures**.
+
+The code used for training this model can be found at https://github.com/Glasgow-AI4BioMed/bioner along with links to other biomedical NER models trained on well-known biomedical corpora. The source dataset information is below.
+
+## Example Usage
+
+The code below will load up the model and apply it to the provided text. It uses a simple aggregation strategy to post-process the individual tokens into larger multi-token entities where needed.
+
+```python
+from transformers import pipeline
+
+# Load the model as part of an NER pipeline
+ner_pipeline = pipeline("token-classification", 
+                        model="Glasgow-AI4BioMed/bioner_medmentions_st21pv",
+                        aggregation_strategy="max")
+
+# Apply it to some text
+ner_pipeline("EGFR T790M mutations have been known to affect treatment outcomes for NSCLC patients receiving erlotinib.")
+
+# Output:
+# [ {"entity_group": "Disorders", "score": 0.62466, "word": "egfr t790m mutations", "start": 0, "end": 20},
+#   {"entity_group": "Disorders", "score": 0.98835, "word": "nsclc", "start": 51, "end": 56},
+#   {"entity_group": "Chemicals & Drugs", "score": 0.97885, "word": "erlotinib", "start": 76, "end": 85} ]
+```
+
+## Dataset Info
+
+**Source:** The ST21pv version of MedMentions was downloaded from: https://github.com/chanzuckerberg/MedMentions/tree/master/st21pv
+
+The dataset should be cited with: Mohan, Sunil, and Donghui Li. "MedMentions: A Large Biomedical Corpus Annotated with UMLS Concepts." Automated Knowledge Base Construction (AKBC), 2019, https://openreview.net/forum?id=SylxCx5pTQ. DOI: [10.24432/C5G59C](https://doi.org/10.24432/C5G59C)
+
+An overview of semantic types can be found at: https://www.nlm.nih.gov/research/umls/META3_current_semantic_types.html
+
+**Preprocessing:** The training, validation and test splits were maintained from the original dataset. Concept identifiers (CUIs) were used to map each annotation to its associated UMLS entry to recover semantic types (from the MRSTY.RRF UMLS file). Semantic types provided in MedMentions were not used. Annotations were mapped to specific *semantic groups* names using the Semantic Groups file available at: https://www.nlm.nih.gov/research/umls/knowledge_sources/semantic_network/index.html. This contrasts with the finegrained version that mapped annotations to *semantic types*. The preprocessing script for this dataset is [prepare_medmentions.py](https://github.com/Glasgow-AI4BioMed/bioner/blob/main/prepare_medmentions.py.py) without the --finegrain flag.
+
+## Performance
+
+The span-level performance on the test split for the different labels are shown in the tables below. The full performance results are available in the model repo in Markdown format for viewing and JSON format for easier loading. These include the performance at token level (with individual B- and I- labels as the token classifier uses IOB2 token labelling).
+
+| Label | Precision | Recall | F1-score | Support |
+| --- | --- | --- | --- | --- |
+| Anatomy | 0.656 | 0.672 | 0.664 | 3277 |
+| Chemicals & Drugs | 0.748 | 0.745 | 0.747 | 7398 |
+| Concepts & Ideas | 0.515 | 0.370 | 0.430 | 3683 |
+| Devices | 0.447 | 0.372 | 0.406 | 355 |
+| Disorders | 0.691 | 0.641 | 0.665 | 8109 |
+| Genes & Molecular Sequences | 0.506 | 0.567 | 0.535 | 1115 |
+| Geographic Areas | 0.671 | 0.737 | 0.703 | 598 |
+| Living Beings | 0.718 | 0.739 | 0.728 | 3994 |
+| Objects | 0.518 | 0.598 | 0.555 | 336 |
+| Occupations | 0.367 | 0.480 | 0.416 | 196 |
+| Organizations | 0.504 | 0.634 | 0.561 | 382 |
+| Phenomena | 0.206 | 0.271 | 0.234 | 269 |
+| Physiology | 0.560 | 0.582 | 0.571 | 3833 |
+| Procedures | 0.597 | 0.607 | 0.602 | 6599 |
+| macro avg | 0.550 | 0.573 | 0.558 | 40144 |
+| weighted avg | 0.641 | 0.630 | 0.634 | 40144 |
+
+
+## Hyperparameters
+
+Hyperparameter tuning was done with [optuna](https://optuna.org/) and the [hyperparameter_search](https://huggingface.co/docs/transformers/en/hpo_train) functionality. 100 trials were run. Early stopping was applied during training. The best performing model was selected using the macro F1 performance on the validation set. The selected hyperparameters are in the table below.
+
+| Hyperparameter | Value |
+|----------------|-------|
+| epochs | 4.0 |
+| learning_rate | 9.767344966191627e-05 |
+| per_device_train_batch_size | 16 |
+| weight_decay | 0.025286446963170207 |
+| warmup_ratio | 0.021367464793327073 |
+

best_hyperparameters.json

@@ -0,0 +1,7 @@
+{
+  "epochs": 4.0,
+  "learning_rate": 9.767344966191627e-05,
+  "per_device_train_batch_size": 16,
+  "weight_decay": 0.025286446963170207,
+  "warmup_ratio": 0.021367464793327073
+}

config.json

@@ -0,0 +1,56 @@
+{
+  "_name_or_path": "microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext",
+  "architectures": [
+    "BertForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "O",
+    "1": "B-Anatomy",
+    "2": "I-Anatomy",
+    "3": "B-Chemicals & Drugs",
+    "4": "I-Chemicals & Drugs",
+    "5": "B-Concepts & Ideas",
+    "6": "I-Concepts & Ideas",
+    "7": "B-Devices",
+    "8": "I-Devices",
+    "9": "B-Disorders",
+    "10": "I-Disorders",
+    "11": "B-Genes & Molecular Sequences",
+    "12": "I-Genes & Molecular Sequences",
+    "13": "B-Geographic Areas",
+    "14": "I-Geographic Areas",
+    "15": "B-Living Beings",
+    "16": "I-Living Beings",
+    "17": "B-Objects",
+    "18": "I-Objects",
+    "19": "B-Occupations",
+    "20": "I-Occupations",
+    "21": "B-Organizations",
+    "22": "I-Organizations",
+    "23": "B-Phenomena",
+    "24": "I-Phenomena",
+    "25": "B-Physiology",
+    "26": "I-Physiology",
+    "27": "B-Procedures",
+    "28": "I-Procedures"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.1",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f089f7dbe5fce6283f669a8ca9cb4ed37d330acc16a1cc129b4086edbc54404e
+size 435679140

performance_report.json

@@ -0,0 +1,869 @@
+{
+  "train": {
+    "token_level": {
+      "O": {
+        "precision": 0.9720447712640268,
+        "recall": 0.976257153730711,
+        "f1-score": 0.9741464087457614,
+        "support": 557052.0
+      },
+      "B-Anatomy": {
+        "precision": 0.919032967032967,
+        "recall": 0.9125349162011173,
+        "f1-score": 0.9157724146993124,
+        "support": 11456.0
+      },
+      "I-Anatomy": {
+        "precision": 0.9173255246461689,
+        "recall": 0.9449024733561231,
+        "f1-score": 0.9309098113020653,
+        "support": 9946.0
+      },
+      "B-Chemicals & Drugs": {
+        "precision": 0.9425027164070989,
+        "recall": 0.9337938458778147,
+        "f1-score": 0.9381280699382633,
+        "support": 22294.0
+      },
+      "I-Chemicals & Drugs": {
+        "precision": 0.9631471190406552,
+        "recall": 0.9549233148601624,
+        "f1-score": 0.9590175870047082,
+        "support": 31036.0
+      },
+      "B-Concepts & Ideas": {
+        "precision": 0.8773684210526316,
+        "recall": 0.6520633678857813,
+        "f1-score": 0.74812072254011,
+        "support": 10226.0
+      },
+      "I-Concepts & Ideas": {
+        "precision": 0.8836440877979702,
+        "recall": 0.8557714285714285,
+        "f1-score": 0.8694844403158384,
+        "support": 8750.0
+      },
+      "B-Devices": {
+        "precision": 0.8684701492537313,
+        "recall": 0.8005159071367154,
+        "f1-score": 0.8331096196868009,
+        "support": 1163.0
+      },
+      "I-Devices": {
+        "precision": 0.8502097064110246,
+        "recall": 0.8857677902621723,
+        "f1-score": 0.8676245796392541,
+        "support": 1602.0
+      },
+      "B-Disorders": {
+        "precision": 0.9004460226042523,
+        "recall": 0.8502964628910243,
+        "f1-score": 0.8746529822495163,
+        "support": 24455.0
+      },
+      "I-Disorders": {
+        "precision": 0.8971253717191743,
+        "recall": 0.9068571926461619,
+        "f1-score": 0.9019650323894534,
+        "support": 22954.0
+      },
+      "B-Genes & Molecular Sequences": {
+        "precision": 0.8677474402730375,
+        "recall": 0.9045360213459828,
+        "f1-score": 0.8857599070982726,
+        "support": 3373.0
+      },
+      "I-Genes & Molecular Sequences": {
+        "precision": 0.8909386869234943,
+        "recall": 0.9354348651728067,
+        "f1-score": 0.9126447429365447,
+        "support": 5266.0
+      },
+      "B-Geographic Areas": {
+        "precision": 0.9166666666666666,
+        "recall": 0.9500587544065805,
+        "f1-score": 0.933064050778996,
+        "support": 1702.0
+      },
+      "I-Geographic Areas": {
+        "precision": 0.9370424597364568,
+        "recall": 0.9343065693430657,
+        "f1-score": 0.935672514619883,
+        "support": 1370.0
+      },
+      "B-Living Beings": {
+        "precision": 0.9251754385964912,
+        "recall": 0.9346034559149313,
+        "f1-score": 0.9298655499228565,
+        "support": 11285.0
+      },
+      "I-Living Beings": {
+        "precision": 0.9578313253012049,
+        "recall": 0.9625474210993623,
+        "f1-score": 0.9601835822698176,
+        "support": 12389.0
+      },
+      "B-Objects": {
+        "precision": 0.8566810344827587,
+        "recall": 0.9555288461538461,
+        "f1-score": 0.9034090909090909,
+        "support": 832.0
+      },
+      "I-Objects": {
+        "precision": 0.9041095890410958,
+        "recall": 0.9361702127659575,
+        "f1-score": 0.9198606271777003,
+        "support": 705.0
+      },
+      "B-Occupations": {
+        "precision": 0.6485148514851485,
+        "recall": 0.7586872586872587,
+        "f1-score": 0.699288256227758,
+        "support": 518.0
+      },
+      "I-Occupations": {
+        "precision": 0.6,
+        "recall": 0.7557251908396947,
+        "f1-score": 0.668918918918919,
+        "support": 393.0
+      },
+      "B-Organizations": {
+        "precision": 0.8365045806906272,
+        "recall": 0.9158950617283951,
+        "f1-score": 0.874401473296501,
+        "support": 1296.0
+      },
+      "I-Organizations": {
+        "precision": 0.8566362715298885,
+        "recall": 0.9575311438278595,
+        "f1-score": 0.9042780748663102,
+        "support": 1766.0
+      },
+      "B-Phenomena": {
+        "precision": 0.5407685098406748,
+        "recall": 0.5133451957295374,
+        "f1-score": 0.5267001369237791,
+        "support": 1124.0
+      },
+      "I-Phenomena": {
+        "precision": 0.5816203143893591,
+        "recall": 0.4863498483316481,
+        "f1-score": 0.5297356828193832,
+        "support": 989.0
+      },
+      "B-Physiology": {
+        "precision": 0.8493248045486852,
+        "recall": 0.8194753985942054,
+        "f1-score": 0.8341331471948347,
+        "support": 11666.0
+      },
+      "I-Physiology": {
+        "precision": 0.8103037213641546,
+        "recall": 0.8807887323943662,
+        "f1-score": 0.8440773134650685,
+        "support": 8875.0
+      },
+      "B-Procedures": {
+        "precision": 0.8294550810014728,
+        "recall": 0.8438717410848067,
+        "f1-score": 0.8366013071895425,
+        "support": 20022.0
+      },
+      "I-Procedures": {
+        "precision": 0.8911117462132038,
+        "recall": 0.8955909808990378,
+        "f1-score": 0.8933457488718574,
+        "support": 20889.0
+      },
+      "accuracy": 0.9494197870855755,
+      "macro avg": {
+        "precision": 0.8514396337694524,
+        "recall": 0.8625562259220192,
+        "f1-score": 0.8553404066895931,
+        "support": 805394.0
+      },
+      "weighted avg": {
+        "precision": 0.9492471775259178,
+        "recall": 0.9494197870855755,
+        "f1-score": 0.9490351220280436,
+        "support": 805394.0
+      }
+    },
+    "span_level": {
+      "Anatomy": {
+        "precision": 0.8700173310225303,
+        "recall": 0.872360761143453,
+        "f1-score": 0.8711874701722417,
+        "support": 11509
+      },
+      "Chemicals & Drugs": {
+        "precision": 0.9019501380028053,
+        "recall": 0.8886412268188303,
+        "f1-score": 0.8952462219028585,
+        "support": 22432
+      },
+      "Concepts & Ideas": {
+        "precision": 0.7898267870212247,
+        "recall": 0.630293001070768,
+        "f1-score": 0.7010990200855395,
+        "support": 10273
+      },
+      "Devices": {
+        "precision": 0.8045178105994787,
+        "recall": 0.7914529914529914,
+        "f1-score": 0.7979319258940112,
+        "support": 1170
+      },
+      "Disorders": {
+        "precision": 0.8474890461745871,
+        "recall": 0.8165949500690103,
+        "f1-score": 0.8317552201777961,
+        "support": 24634
+      },
+      "Genes & Molecular Sequences": {
+        "precision": 0.8086003372681282,
+        "recall": 0.8506800709639266,
+        "f1-score": 0.829106628242075,
+        "support": 3382
+      },
+      "Geographic Areas": {
+        "precision": 0.876410835214447,
+        "recall": 0.907126168224299,
+        "f1-score": 0.89150401836969,
+        "support": 1712
+      },
+      "Living Beings": {
+        "precision": 0.8884203127745564,
+        "recall": 0.8879522304179839,
+        "f1-score": 0.8881862099253405,
+        "support": 11388
+      },
+      "Objects": {
+        "precision": 0.8064171122994652,
+        "recall": 0.8860164512338425,
+        "f1-score": 0.8443449048152295,
+        "support": 851
+      },
+      "Occupations": {
+        "precision": 0.5763239875389408,
+        "recall": 0.7088122605363985,
+        "f1-score": 0.6357388316151202,
+        "support": 522
+      },
+      "Organizations": {
+        "precision": 0.7973811164713991,
+        "recall": 0.8845565749235474,
+        "f1-score": 0.8387096774193549,
+        "support": 1308
+      },
+      "Phenomena": {
+        "precision": 0.4477234401349072,
+        "recall": 0.46949602122015915,
+        "f1-score": 0.45835131635735865,
+        "support": 1131
+      },
+      "Physiology": {
+        "precision": 0.7840562521179262,
+        "recall": 0.7892717039058502,
+        "f1-score": 0.7866553336166596,
+        "support": 11726
+      },
+      "Procedures": {
+        "precision": 0.7731168893358233,
+        "recall": 0.801439563167039,
+        "f1-score": 0.7870234961489714,
+        "support": 20145
+      },
+      "macro avg": {
+        "precision": 0.78373224256973,
+        "recall": 0.7989067125105784,
+        "f1-score": 0.7897743053387318,
+        "support": 122183
+      },
+      "weighted avg": {
+        "precision": 0.8334626249065854,
+        "recall": 0.8204496533887693,
+        "f1-score": 0.8260050590679613,
+        "support": 122183
+      }
+    }
+  },
+  "val": {
+    "token_level": {
+      "O": {
+        "precision": 0.930847233100809,
+        "recall": 0.9466419326729286,
+        "f1-score": 0.9386781450775005,
+        "support": 187057.0
+      },
+      "B-Anatomy": {
+        "precision": 0.7610457516339869,
+        "recall": 0.7537545313309166,
+        "f1-score": 0.757382593989853,
+        "support": 3862.0
+      },
+      "I-Anatomy": {
+        "precision": 0.7202499289974439,
+        "recall": 0.7752980739834913,
+        "f1-score": 0.7467608951707891,
+        "support": 3271.0
+      },
+      "B-Chemicals & Drugs": {
+        "precision": 0.8210823909531503,
+        "recall": 0.823188014576866,
+        "f1-score": 0.8221338545528072,
+        "support": 7409.0
+      },
+      "I-Chemicals & Drugs": {
+        "precision": 0.8563915857605178,
+        "recall": 0.8465460361891433,
+        "f1-score": 0.8514403499069931,
+        "support": 10003.0
+      },
+      "B-Concepts & Ideas": {
+        "precision": 0.5897058823529412,
+        "recall": 0.3599640933572711,
+        "f1-score": 0.44704570791527315,
+        "support": 3342.0
+      },
+      "I-Concepts & Ideas": {
+        "precision": 0.5564516129032258,
+        "recall": 0.45887294364718234,
+        "f1-score": 0.5029733358910417,
+        "support": 2857.0
+      },
+      "B-Devices": {
+        "precision": 0.6964856230031949,
+        "recall": 0.44308943089430897,
+        "f1-score": 0.5416149068322982,
+        "support": 492.0
+      },
+      "I-Devices": {
+        "precision": 0.627677100494234,
+        "recall": 0.5537790697674418,
+        "f1-score": 0.5884169884169884,
+        "support": 688.0
+      },
+      "B-Disorders": {
+        "precision": 0.7699307347548554,
+        "recall": 0.6881524641903375,
+        "f1-score": 0.7267482853663226,
+        "support": 8238.0
+      },
+      "I-Disorders": {
+        "precision": 0.7285814116002796,
+        "recall": 0.7107990182710663,
+        "f1-score": 0.7195803713161709,
+        "support": 7334.0
+      },
+      "B-Genes & Molecular Sequences": {
+        "precision": 0.6427840327533265,
+        "recall": 0.6562173458725182,
+        "f1-score": 0.6494312306101344,
+        "support": 957.0
+      },
+      "I-Genes & Molecular Sequences": {
+        "precision": 0.6770573566084788,
+        "recall": 0.71026814911707,
+        "f1-score": 0.6932652409830833,
+        "support": 1529.0
+      },
+      "B-Geographic Areas": {
+        "precision": 0.7933042212518195,
+        "recall": 0.8086053412462908,
+        "f1-score": 0.8008817046289493,
+        "support": 674.0
+      },
+      "I-Geographic Areas": {
+        "precision": 0.7455197132616488,
+        "recall": 0.7675276752767528,
+        "f1-score": 0.7563636363636363,
+        "support": 542.0
+      },
+      "B-Living Beings": {
+        "precision": 0.79478672985782,
+        "recall": 0.8014336917562724,
+        "f1-score": 0.7980963712076146,
+        "support": 4185.0
+      },
+      "I-Living Beings": {
+        "precision": 0.8440483768300445,
+        "recall": 0.8202061855670103,
+        "f1-score": 0.8319564990065879,
+        "support": 4850.0
+      },
+      "B-Objects": {
+        "precision": 0.61,
+        "recall": 0.71484375,
+        "f1-score": 0.658273381294964,
+        "support": 256.0
+      },
+      "I-Objects": {
+        "precision": 0.6748971193415638,
+        "recall": 0.6721311475409836,
+        "f1-score": 0.6735112936344969,
+        "support": 244.0
+      },
+      "B-Occupations": {
+        "precision": 0.5208333333333334,
+        "recall": 0.5076142131979695,
+        "f1-score": 0.5141388174807198,
+        "support": 197.0
+      },
+      "I-Occupations": {
+        "precision": 0.47368421052631576,
+        "recall": 0.36416184971098264,
+        "f1-score": 0.4117647058823529,
+        "support": 173.0
+      },
+      "B-Organizations": {
+        "precision": 0.579476861167002,
+        "recall": 0.6501128668171557,
+        "f1-score": 0.6127659574468085,
+        "support": 443.0
+      },
+      "I-Organizations": {
+        "precision": 0.6396526772793053,
+        "recall": 0.7106109324758842,
+        "f1-score": 0.6732673267326733,
+        "support": 622.0
+      },
+      "B-Phenomena": {
+        "precision": 0.34057971014492755,
+        "recall": 0.34306569343065696,
+        "f1-score": 0.3418181818181818,
+        "support": 274.0
+      },
+      "I-Phenomena": {
+        "precision": 0.28451882845188287,
+        "recall": 0.32075471698113206,
+        "f1-score": 0.30155210643015523,
+        "support": 212.0
+      },
+      "B-Physiology": {
+        "precision": 0.6399317406143344,
+        "recall": 0.6038647342995169,
+        "f1-score": 0.6213753106876554,
+        "support": 3726.0
+      },
+      "I-Physiology": {
+        "precision": 0.5488801990757198,
+        "recall": 0.5744047619047619,
+        "f1-score": 0.5613524813670242,
+        "support": 2688.0
+      },
+      "B-Procedures": {
+        "precision": 0.6613690007867821,
+        "recall": 0.6512240471025721,
+        "f1-score": 0.6562573190725272,
+        "support": 6454.0
+      },
+      "I-Procedures": {
+        "precision": 0.6790007806401249,
+        "recall": 0.6622506471752703,
+        "f1-score": 0.6705211224175146,
+        "support": 6567.0
+      },
+      "accuracy": 0.8725375818329085,
+      "macro avg": {
+        "precision": 0.6623715223268644,
+        "recall": 0.6448063227018537,
+        "f1-score": 0.6506678662586591,
+        "support": 269146.0
+      },
+      "weighted avg": {
+        "precision": 0.8694071423336295,
+        "recall": 0.8725375818329085,
+        "f1-score": 0.8703500500944416,
+        "support": 269146.0
+      }
+    },
+    "span_level": {
+      "Anatomy": {
+        "precision": 0.6821275523065289,
+        "recall": 0.6972429786137594,
+        "f1-score": 0.6896024464831805,
+        "support": 3881
+      },
+      "Chemicals & Drugs": {
+        "precision": 0.7514910536779325,
+        "recall": 0.7570093457943925,
+        "f1-score": 0.7542401064183571,
+        "support": 7490
+      },
+      "Concepts & Ideas": {
+        "precision": 0.4873985476292183,
+        "recall": 0.33907875185735514,
+        "f1-score": 0.39992989835261133,
+        "support": 3365
+      },
+      "Devices": {
+        "precision": 0.6056338028169014,
+        "recall": 0.43610547667342797,
+        "f1-score": 0.5070754716981132,
+        "support": 493
+      },
+      "Disorders": {
+        "precision": 0.7004692387904067,
+        "recall": 0.6459134615384615,
+        "f1-score": 0.6720860430215108,
+        "support": 8320
+      },
+      "Genes & Molecular Sequences": {
+        "precision": 0.5463414634146342,
+        "recall": 0.5803108808290155,
+        "f1-score": 0.5628140703517589,
+        "support": 965
+      },
+      "Geographic Areas": {
+        "precision": 0.7474600870827286,
+        "recall": 0.7595870206489675,
+        "f1-score": 0.753474762253109,
+        "support": 678
+      },
+      "Living Beings": {
+        "precision": 0.7159887798036466,
+        "recall": 0.7244560075685903,
+        "f1-score": 0.7201975076416647,
+        "support": 4228
+      },
+      "Objects": {
+        "precision": 0.551948051948052,
+        "recall": 0.6563706563706564,
+        "f1-score": 0.599647266313933,
+        "support": 259
+      },
+      "Occupations": {
+        "precision": 0.46568627450980393,
+        "recall": 0.4797979797979798,
+        "f1-score": 0.472636815920398,
+        "support": 198
+      },
+      "Organizations": {
+        "precision": 0.5019157088122606,
+        "recall": 0.5783664459161147,
+        "f1-score": 0.5374358974358974,
+        "support": 453
+      },
+      "Phenomena": {
+        "precision": 0.27672955974842767,
+        "recall": 0.32116788321167883,
+        "f1-score": 0.29729729729729726,
+        "support": 274
+      },
+      "Physiology": {
+        "precision": 0.5590094836670179,
+        "recall": 0.5657158091175687,
+        "f1-score": 0.562342652709686,
+        "support": 3751
+      },
+      "Procedures": {
+        "precision": 0.5877219380078242,
+        "recall": 0.6000921800583807,
+        "f1-score": 0.5938426453819841,
+        "support": 6509
+      },
+      "macro avg": {
+        "precision": 0.5842801101582417,
+        "recall": 0.5815153484283105,
+        "f1-score": 0.5801873486628216,
+        "support": 40864
+      },
+      "weighted avg": {
+        "precision": 0.6500699757755491,
+        "recall": 0.6334915818324197,
+        "f1-score": 0.6401859322619969,
+        "support": 40864
+      }
+    }
+  },
+  "test": {
+    "token_level": {
+      "O": {
+        "precision": 0.9327288328920794,
+        "recall": 0.9450010602205259,
+        "f1-score": 0.9388248429279391,
+        "support": 188640.0
+      },
+      "B-Anatomy": {
+        "precision": 0.7327746741154563,
+        "recall": 0.7246777163904236,
+        "f1-score": 0.7287037037037037,
+        "support": 3258.0
+      },
+      "I-Anatomy": {
+        "precision": 0.7148125384142594,
+        "recall": 0.7663920922570017,
+        "f1-score": 0.7397042455080299,
+        "support": 3035.0
+      },
+      "B-Chemicals & Drugs": {
+        "precision": 0.8137902559867878,
+        "recall": 0.8030694010593508,
+        "f1-score": 0.8083942853236722,
+        "support": 7363.0
+      },
+      "I-Chemicals & Drugs": {
+        "precision": 0.8251571052098114,
+        "recall": 0.8428408737964592,
+        "f1-score": 0.8339052496798975,
+        "support": 9659.0
+      },
+      "B-Concepts & Ideas": {
+        "precision": 0.6245535714285714,
+        "recall": 0.3821360284075389,
+        "f1-score": 0.47415692255549907,
+        "support": 3661.0
+      },
+      "I-Concepts & Ideas": {
+        "precision": 0.5620985010706638,
+        "recall": 0.5108660395718456,
+        "f1-score": 0.5352591333899746,
+        "support": 3083.0
+      },
+      "B-Devices": {
+        "precision": 0.5829596412556054,
+        "recall": 0.3672316384180791,
+        "f1-score": 0.4506065857885615,
+        "support": 354.0
+      },
+      "I-Devices": {
+        "precision": 0.4653284671532847,
+        "recall": 0.450530035335689,
+        "f1-score": 0.4578096947935368,
+        "support": 566.0
+      },
+      "B-Disorders": {
+        "precision": 0.7646812665643744,
+        "recall": 0.6812476699391078,
+        "f1-score": 0.7205573080967402,
+        "support": 8047.0
+      },
+      "I-Disorders": {
+        "precision": 0.7384636639955788,
+        "recall": 0.7031969477700303,
+        "f1-score": 0.7203989487162208,
+        "support": 7601.0
+      },
+      "B-Genes & Molecular Sequences": {
+        "precision": 0.6082304526748972,
+        "recall": 0.6651665166516652,
+        "f1-score": 0.6354256233877902,
+        "support": 1111.0
+      },
+      "I-Genes & Molecular Sequences": {
+        "precision": 0.6074357572443958,
+        "recall": 0.6558441558441559,
+        "f1-score": 0.6307124609707635,
+        "support": 1694.0
+      },
+      "B-Geographic Areas": {
+        "precision": 0.7287878787878788,
+        "recall": 0.8097643097643098,
+        "f1-score": 0.7671451355661882,
+        "support": 594.0
+      },
+      "I-Geographic Areas": {
+        "precision": 0.720226843100189,
+        "recall": 0.6840215439856373,
+        "f1-score": 0.7016574585635359,
+        "support": 557.0
+      },
+      "B-Living Beings": {
+        "precision": 0.7869297163995068,
+        "recall": 0.8009538152610441,
+        "f1-score": 0.7938798358004727,
+        "support": 3984.0
+      },
+      "I-Living Beings": {
+        "precision": 0.8368229403732362,
+        "recall": 0.8145768719539211,
+        "f1-score": 0.8255500673551863,
+        "support": 4514.0
+      },
+      "B-Objects": {
+        "precision": 0.5885558583106267,
+        "recall": 0.6447761194029851,
+        "f1-score": 0.6153846153846154,
+        "support": 335.0
+      },
+      "I-Objects": {
+        "precision": 0.6981818181818182,
+        "recall": 0.6421404682274248,
+        "f1-score": 0.6689895470383276,
+        "support": 299.0
+      },
+      "B-Occupations": {
+        "precision": 0.44398340248962653,
+        "recall": 0.5487179487179488,
+        "f1-score": 0.4908256880733945,
+        "support": 195.0
+      },
+      "I-Occupations": {
+        "precision": 0.3793103448275862,
+        "recall": 0.5076923076923077,
+        "f1-score": 0.4342105263157895,
+        "support": 130.0
+      },
+      "B-Organizations": {
+        "precision": 0.5512820512820513,
+        "recall": 0.675392670157068,
+        "f1-score": 0.6070588235294118,
+        "support": 382.0
+      },
+      "I-Organizations": {
+        "precision": 0.5950653120464441,
+        "recall": 0.8023483365949119,
+        "f1-score": 0.6833333333333333,
+        "support": 511.0
+      },
+      "B-Phenomena": {
+        "precision": 0.2920962199312715,
+        "recall": 0.32075471698113206,
+        "f1-score": 0.3057553956834532,
+        "support": 265.0
+      },
+      "I-Phenomena": {
+        "precision": 0.2720306513409962,
+        "recall": 0.3397129186602871,
+        "f1-score": 0.3021276595744681,
+        "support": 209.0
+      },
+      "B-Physiology": {
+        "precision": 0.6418974499588703,
+        "recall": 0.6137912952281069,
+        "f1-score": 0.6275298217397132,
+        "support": 3814.0
+      },
+      "I-Physiology": {
+        "precision": 0.5521588402143083,
+        "recall": 0.6245989304812835,
+        "f1-score": 0.5861492137838742,
+        "support": 2805.0
+      },
+      "B-Procedures": {
+        "precision": 0.6753959542104437,
+        "recall": 0.6567551082647148,
+        "f1-score": 0.6659451101662157,
+        "support": 6558.0
+      },
+      "I-Procedures": {
+        "precision": 0.7067256367241116,
+        "recall": 0.6688799076212472,
+        "f1-score": 0.6872821653689284,
+        "support": 6928.0
+      },
+      "accuracy": 0.870661701560603,
+      "macro avg": {
+        "precision": 0.6359470912477494,
+        "recall": 0.6432095670571105,
+        "f1-score": 0.6357683931765253,
+        "support": 270152.0
+      },
+      "weighted avg": {
+        "precision": 0.8687298439825012,
+        "recall": 0.870661701560603,
+        "f1-score": 0.8690103283416164,
+        "support": 270152.0
+      }
+    },
+    "span_level": {
+      "Anatomy": {
+        "precision": 0.6564270802266627,
+        "recall": 0.67165090021361,
+        "f1-score": 0.6639517345399698,
+        "support": 3277
+      },
+      "Chemicals & Drugs": {
+        "precision": 0.748099891422367,
+        "recall": 0.745066234117329,
+        "f1-score": 0.7465799810375187,
+        "support": 7398
+      },
+      "Concepts & Ideas": {
+        "precision": 0.5149451381006432,
+        "recall": 0.36953570458865054,
+        "f1-score": 0.4302877015491622,
+        "support": 3683
+      },
+      "Devices": {
+        "precision": 0.44745762711864406,
+        "recall": 0.37183098591549296,
+        "f1-score": 0.40615384615384614,
+        "support": 355
+      },
+      "Disorders": {
+        "precision": 0.6907038512616201,
+        "recall": 0.6413861141941053,
+        "f1-score": 0.6651320416906452,
+        "support": 8109
+      },
+      "Genes & Molecular Sequences": {
+        "precision": 0.5060048038430744,
+        "recall": 0.5668161434977579,
+        "f1-score": 0.5346869712351946,
+        "support": 1115
+      },
+      "Geographic Areas": {
+        "precision": 0.6712328767123288,
+        "recall": 0.7374581939799331,
+        "f1-score": 0.7027888446215139,
+        "support": 598
+      },
+      "Living Beings": {
+        "precision": 0.7177242888402626,
+        "recall": 0.7391086629944917,
+        "f1-score": 0.7282595288022696,
+        "support": 3994
+      },
+      "Objects": {
+        "precision": 0.5180412371134021,
+        "recall": 0.5982142857142857,
+        "f1-score": 0.5552486187845304,
+        "support": 336
+      },
+      "Occupations": {
+        "precision": 0.3671875,
+        "recall": 0.47959183673469385,
+        "f1-score": 0.415929203539823,
+        "support": 196
+      },
+      "Organizations": {
+        "precision": 0.5041666666666667,
+        "recall": 0.6335078534031413,
+        "f1-score": 0.5614849187935034,
+        "support": 382
+      },
+      "Phenomena": {
+        "precision": 0.2056338028169014,
+        "recall": 0.27137546468401486,
+        "f1-score": 0.23397435897435898,
+        "support": 269
+      },
+      "Physiology": {
+        "precision": 0.5598194130925508,
+        "recall": 0.5823115053482911,
+        "f1-score": 0.570843989769821,
+        "support": 3833
+      },
+      "Procedures": {
+        "precision": 0.5965460771177609,
+        "recall": 0.607213214123352,
+        "f1-score": 0.6018323820967258,
+        "support": 6599
+      },
+      "macro avg": {
+        "precision": 0.5502850181666347,
+        "recall": 0.5725047928220821,
+        "f1-score": 0.558368151542063,
+        "support": 40144
+      },
+      "weighted avg": {
+        "precision": 0.6414502549457723,
+        "recall": 0.6297578716620167,
+        "f1-score": 0.6340080620691133,
+        "support": 40144
+      }
+    }
+  }
+}

performance_report.md

@@ -0,0 +1,180 @@
+# Performance on Training Set
+
+## Span Level
+
+| Label | Precision | Recall | F1-score | Support |
+| --- | --- | --- | --- | --- |
+| Anatomy | 0.870 | 0.872 | 0.871 | 11509 |
+| Chemicals & Drugs | 0.902 | 0.889 | 0.895 | 22432 |
+| Concepts & Ideas | 0.790 | 0.630 | 0.701 | 10273 |
+| Devices | 0.805 | 0.791 | 0.798 | 1170 |
+| Disorders | 0.847 | 0.817 | 0.832 | 24634 |
+| Genes & Molecular Sequences | 0.809 | 0.851 | 0.829 | 3382 |
+| Geographic Areas | 0.876 | 0.907 | 0.892 | 1712 |
+| Living Beings | 0.888 | 0.888 | 0.888 | 11388 |
+| Objects | 0.806 | 0.886 | 0.844 | 851 |
+| Occupations | 0.576 | 0.709 | 0.636 | 522 |
+| Organizations | 0.797 | 0.885 | 0.839 | 1308 |
+| Phenomena | 0.448 | 0.469 | 0.458 | 1131 |
+| Physiology | 0.784 | 0.789 | 0.787 | 11726 |
+| Procedures | 0.773 | 0.801 | 0.787 | 20145 |
+| macro avg | 0.784 | 0.799 | 0.790 | 122183 |
+| weighted avg | 0.833 | 0.820 | 0.826 | 122183 |
+
+## Token Level
+
+| Label | Precision | Recall | F1-score | Support |
+| --- | --- | --- | --- | --- |
+| O | 0.972 | 0.976 | 0.974 | 557052 |
+| B-Anatomy | 0.919 | 0.913 | 0.916 | 11456 |
+| I-Anatomy | 0.917 | 0.945 | 0.931 | 9946 |
+| B-Chemicals & Drugs | 0.943 | 0.934 | 0.938 | 22294 |
+| I-Chemicals & Drugs | 0.963 | 0.955 | 0.959 | 31036 |
+| B-Concepts & Ideas | 0.877 | 0.652 | 0.748 | 10226 |
+| I-Concepts & Ideas | 0.884 | 0.856 | 0.869 | 8750 |
+| B-Devices | 0.868 | 0.801 | 0.833 | 1163 |
+| I-Devices | 0.850 | 0.886 | 0.868 | 1602 |
+| B-Disorders | 0.900 | 0.850 | 0.875 | 24455 |
+| I-Disorders | 0.897 | 0.907 | 0.902 | 22954 |
+| B-Genes & Molecular Sequences | 0.868 | 0.905 | 0.886 | 3373 |
+| I-Genes & Molecular Sequences | 0.891 | 0.935 | 0.913 | 5266 |
+| B-Geographic Areas | 0.917 | 0.950 | 0.933 | 1702 |
+| I-Geographic Areas | 0.937 | 0.934 | 0.936 | 1370 |
+| B-Living Beings | 0.925 | 0.935 | 0.930 | 11285 |
+| I-Living Beings | 0.958 | 0.963 | 0.960 | 12389 |
+| B-Objects | 0.857 | 0.956 | 0.903 | 832 |
+| I-Objects | 0.904 | 0.936 | 0.920 | 705 |
+| B-Occupations | 0.649 | 0.759 | 0.699 | 518 |
+| I-Occupations | 0.600 | 0.756 | 0.669 | 393 |
+| B-Organizations | 0.837 | 0.916 | 0.874 | 1296 |
+| I-Organizations | 0.857 | 0.958 | 0.904 | 1766 |
+| B-Phenomena | 0.541 | 0.513 | 0.527 | 1124 |
+| I-Phenomena | 0.582 | 0.486 | 0.530 | 989 |
+| B-Physiology | 0.849 | 0.819 | 0.834 | 11666 |
+| I-Physiology | 0.810 | 0.881 | 0.844 | 8875 |
+| B-Procedures | 0.829 | 0.844 | 0.837 | 20022 |
+| I-Procedures | 0.891 | 0.896 | 0.893 | 20889 |
+| macro avg | 0.851 | 0.863 | 0.855 | 805394 |
+| weighted avg | 0.949 | 0.949 | 0.949 | 805394 |
+
+
+# Performance on Validation Set
+
+## Span Level
+
+| Label | Precision | Recall | F1-score | Support |
+| --- | --- | --- | --- | --- |
+| Anatomy | 0.682 | 0.697 | 0.690 | 3881 |
+| Chemicals & Drugs | 0.751 | 0.757 | 0.754 | 7490 |
+| Concepts & Ideas | 0.487 | 0.339 | 0.400 | 3365 |
+| Devices | 0.606 | 0.436 | 0.507 | 493 |
+| Disorders | 0.700 | 0.646 | 0.672 | 8320 |
+| Genes & Molecular Sequences | 0.546 | 0.580 | 0.563 | 965 |
+| Geographic Areas | 0.747 | 0.760 | 0.753 | 678 |
+| Living Beings | 0.716 | 0.724 | 0.720 | 4228 |
+| Objects | 0.552 | 0.656 | 0.600 | 259 |
+| Occupations | 0.466 | 0.480 | 0.473 | 198 |
+| Organizations | 0.502 | 0.578 | 0.537 | 453 |
+| Phenomena | 0.277 | 0.321 | 0.297 | 274 |
+| Physiology | 0.559 | 0.566 | 0.562 | 3751 |
+| Procedures | 0.588 | 0.600 | 0.594 | 6509 |
+| macro avg | 0.584 | 0.582 | 0.580 | 40864 |
+| weighted avg | 0.650 | 0.633 | 0.640 | 40864 |
+
+## Token Level
+
+| Label | Precision | Recall | F1-score | Support |
+| --- | --- | --- | --- | --- |
+| O | 0.931 | 0.947 | 0.939 | 187057 |
+| B-Anatomy | 0.761 | 0.754 | 0.757 | 3862 |
+| I-Anatomy | 0.720 | 0.775 | 0.747 | 3271 |
+| B-Chemicals & Drugs | 0.821 | 0.823 | 0.822 | 7409 |
+| I-Chemicals & Drugs | 0.856 | 0.847 | 0.851 | 10003 |
+| B-Concepts & Ideas | 0.590 | 0.360 | 0.447 | 3342 |
+| I-Concepts & Ideas | 0.556 | 0.459 | 0.503 | 2857 |
+| B-Devices | 0.696 | 0.443 | 0.542 | 492 |
+| I-Devices | 0.628 | 0.554 | 0.588 | 688 |
+| B-Disorders | 0.770 | 0.688 | 0.727 | 8238 |
+| I-Disorders | 0.729 | 0.711 | 0.720 | 7334 |
+| B-Genes & Molecular Sequences | 0.643 | 0.656 | 0.649 | 957 |
+| I-Genes & Molecular Sequences | 0.677 | 0.710 | 0.693 | 1529 |
+| B-Geographic Areas | 0.793 | 0.809 | 0.801 | 674 |
+| I-Geographic Areas | 0.746 | 0.768 | 0.756 | 542 |
+| B-Living Beings | 0.795 | 0.801 | 0.798 | 4185 |
+| I-Living Beings | 0.844 | 0.820 | 0.832 | 4850 |
+| B-Objects | 0.610 | 0.715 | 0.658 | 256 |
+| I-Objects | 0.675 | 0.672 | 0.674 | 244 |
+| B-Occupations | 0.521 | 0.508 | 0.514 | 197 |
+| I-Occupations | 0.474 | 0.364 | 0.412 | 173 |
+| B-Organizations | 0.579 | 0.650 | 0.613 | 443 |
+| I-Organizations | 0.640 | 0.711 | 0.673 | 622 |
+| B-Phenomena | 0.341 | 0.343 | 0.342 | 274 |
+| I-Phenomena | 0.285 | 0.321 | 0.302 | 212 |
+| B-Physiology | 0.640 | 0.604 | 0.621 | 3726 |
+| I-Physiology | 0.549 | 0.574 | 0.561 | 2688 |
+| B-Procedures | 0.661 | 0.651 | 0.656 | 6454 |
+| I-Procedures | 0.679 | 0.662 | 0.671 | 6567 |
+| macro avg | 0.662 | 0.645 | 0.651 | 269146 |
+| weighted avg | 0.869 | 0.873 | 0.870 | 269146 |
+
+
+# Performance on Testing Set
+
+## Span Level
+
+| Label | Precision | Recall | F1-score | Support |
+| --- | --- | --- | --- | --- |
+| Anatomy | 0.656 | 0.672 | 0.664 | 3277 |
+| Chemicals & Drugs | 0.748 | 0.745 | 0.747 | 7398 |
+| Concepts & Ideas | 0.515 | 0.370 | 0.430 | 3683 |
+| Devices | 0.447 | 0.372 | 0.406 | 355 |
+| Disorders | 0.691 | 0.641 | 0.665 | 8109 |
+| Genes & Molecular Sequences | 0.506 | 0.567 | 0.535 | 1115 |
+| Geographic Areas | 0.671 | 0.737 | 0.703 | 598 |
+| Living Beings | 0.718 | 0.739 | 0.728 | 3994 |
+| Objects | 0.518 | 0.598 | 0.555 | 336 |
+| Occupations | 0.367 | 0.480 | 0.416 | 196 |
+| Organizations | 0.504 | 0.634 | 0.561 | 382 |
+| Phenomena | 0.206 | 0.271 | 0.234 | 269 |
+| Physiology | 0.560 | 0.582 | 0.571 | 3833 |
+| Procedures | 0.597 | 0.607 | 0.602 | 6599 |
+| macro avg | 0.550 | 0.573 | 0.558 | 40144 |
+| weighted avg | 0.641 | 0.630 | 0.634 | 40144 |
+
+## Token Level
+
+| Label | Precision | Recall | F1-score | Support |
+| --- | --- | --- | --- | --- |
+| O | 0.933 | 0.945 | 0.939 | 188640 |
+| B-Anatomy | 0.733 | 0.725 | 0.729 | 3258 |
+| I-Anatomy | 0.715 | 0.766 | 0.740 | 3035 |
+| B-Chemicals & Drugs | 0.814 | 0.803 | 0.808 | 7363 |
+| I-Chemicals & Drugs | 0.825 | 0.843 | 0.834 | 9659 |
+| B-Concepts & Ideas | 0.625 | 0.382 | 0.474 | 3661 |
+| I-Concepts & Ideas | 0.562 | 0.511 | 0.535 | 3083 |
+| B-Devices | 0.583 | 0.367 | 0.451 | 354 |
+| I-Devices | 0.465 | 0.451 | 0.458 | 566 |
+| B-Disorders | 0.765 | 0.681 | 0.721 | 8047 |
+| I-Disorders | 0.738 | 0.703 | 0.720 | 7601 |
+| B-Genes & Molecular Sequences | 0.608 | 0.665 | 0.635 | 1111 |
+| I-Genes & Molecular Sequences | 0.607 | 0.656 | 0.631 | 1694 |
+| B-Geographic Areas | 0.729 | 0.810 | 0.767 | 594 |
+| I-Geographic Areas | 0.720 | 0.684 | 0.702 | 557 |
+| B-Living Beings | 0.787 | 0.801 | 0.794 | 3984 |
+| I-Living Beings | 0.837 | 0.815 | 0.826 | 4514 |
+| B-Objects | 0.589 | 0.645 | 0.615 | 335 |
+| I-Objects | 0.698 | 0.642 | 0.669 | 299 |
+| B-Occupations | 0.444 | 0.549 | 0.491 | 195 |
+| I-Occupations | 0.379 | 0.508 | 0.434 | 130 |
+| B-Organizations | 0.551 | 0.675 | 0.607 | 382 |
+| I-Organizations | 0.595 | 0.802 | 0.683 | 511 |
+| B-Phenomena | 0.292 | 0.321 | 0.306 | 265 |
+| I-Phenomena | 0.272 | 0.340 | 0.302 | 209 |
+| B-Physiology | 0.642 | 0.614 | 0.628 | 3814 |
+| I-Physiology | 0.552 | 0.625 | 0.586 | 2805 |
+| B-Procedures | 0.675 | 0.657 | 0.666 | 6558 |
+| I-Procedures | 0.707 | 0.669 | 0.687 | 6928 |
+| macro avg | 0.636 | 0.643 | 0.636 | 270152 |
+| weighted avg | 0.869 | 0.871 | 0.869 | 270152 |
+
+

rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9eb0be8c505b7f7bc80024c0457befd900a96d3f1eb3524224a115b11dd2ad72
+size 14244

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

trainer_state.json

@@ -0,0 +1,102 @@
+{
+  "best_metric": 0.6506678662586591,
+  "best_model_checkpoint": "tmp_ner_fantastic-bale-19_38/run-25/checkpoint-660",
+  "epoch": 4.0,
+  "eval_steps": 500,
+  "global_step": 660,
+  "is_hyper_param_search": true,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 1.0,
+      "eval_accuracy": 0.8553461689937804,
+      "eval_loss": 0.47332701086997986,
+      "eval_macro_f1": 0.5179058864521847,
+      "eval_macro_precision": 0.5969084028407209,
+      "eval_macro_recall": 0.5063081060851987,
+      "eval_runtime": 5.987,
+      "eval_samples_per_second": 146.651,
+      "eval_steps_per_second": 18.373,
+      "step": 165
+    },
+    {
+      "epoch": 2.0,
+      "eval_accuracy": 0.8667526175384364,
+      "eval_loss": 0.423623263835907,
+      "eval_macro_f1": 0.6126875404495669,
+      "eval_macro_precision": 0.6399554829251698,
+      "eval_macro_recall": 0.6102712004848894,
+      "eval_runtime": 5.9737,
+      "eval_samples_per_second": 146.978,
+      "eval_steps_per_second": 18.414,
+      "step": 330
+    },
+    {
+      "epoch": 3.0,
+      "eval_accuracy": 0.8703603248794335,
+      "eval_loss": 0.4243398904800415,
+      "eval_macro_f1": 0.6335065301362576,
+      "eval_macro_precision": 0.6801343369395938,
+      "eval_macro_recall": 0.6289027870007556,
+      "eval_runtime": 5.9773,
+      "eval_samples_per_second": 146.888,
+      "eval_steps_per_second": 18.403,
+      "step": 495
+    },
+    {
+      "epoch": 3.0303030303030303,
+      "grad_norm": 0.778854250907898,
+      "learning_rate": 9.035786517978707e-05,
+      "loss": 0.5906,
+      "step": 500
+    },
+    {
+      "epoch": 4.0,
+      "eval_accuracy": 0.8725375818329085,
+      "eval_loss": 0.43925729393959045,
+      "eval_macro_f1": 0.6506678662586591,
+      "eval_macro_precision": 0.6623715223268644,
+      "eval_macro_recall": 0.6448063227018537,
+      "eval_runtime": 5.9602,
+      "eval_samples_per_second": 147.311,
+      "eval_steps_per_second": 18.456,
+      "step": 660
+    }
+  ],
+  "logging_steps": 500,
+  "max_steps": 5280,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 32,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 3,
+        "early_stopping_threshold": 0.001
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 0
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3213359444608236.0,
+  "train_batch_size": 16,
+  "trial_name": null,
+  "trial_params": {
+    "learning_rate": 9.767344966191627e-05,
+    "per_device_train_batch_size": 16,
+    "warmup_ratio": 0.021367464793327073,
+    "weight_decay": 0.025286446963170207
+  }
+}