End of training

README.md

@@ -0,0 +1,91 @@
+---
+license: mit
+base_model: vinai/bertweet-base
+tags:
+- generated_from_trainer
+metrics:
+- f1
+- precision
+- recall
+- accuracy
+model-index:
+- name: bertweet-base_regression_7_seed13_EN
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# bertweet-base_regression_7_seed13_EN
+
+This model is a fine-tuned version of [vinai/bertweet-base](https://huggingface.co/vinai/bertweet-base) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.0892
+- Mse: 5.5584
+- Rmse: 2.3576
+- Mae: 1.3807
+- R2: 0.2207
+- F1: 0.7757
+- Precision: 0.7780
+- Recall: 0.7797
+- Accuracy: 0.7797
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-06
+- train_batch_size: 16
+- eval_batch_size: 16
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 200
+- num_epochs: 10
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss | Mse    | Rmse   | Mae    | R2      | F1     | Precision | Recall | Accuracy |
+|:-------------:|:------:|:----:|:---------------:|:------:|:------:|:------:|:-------:|:------:|:---------:|:------:|:--------:|
+| 1.7333        | 0.4630 | 100  | 1.8475          | 9.7883 | 3.1286 | 2.2877 | -0.4094 | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
+| 1.6952        | 0.9259 | 200  | 1.7889          | 8.8708 | 2.9784 | 2.2442 | -0.2773 | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
+| 1.6175        | 1.3889 | 300  | 1.6295          | 7.6123 | 2.7590 | 2.0223 | -0.0961 | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
+| 1.4401        | 1.8519 | 400  | 1.4962          | 6.6368 | 2.5762 | 1.8601 | 0.0444  | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
+| 1.2553        | 2.3148 | 500  | 1.3949          | 5.9003 | 2.4291 | 1.7518 | 0.1504  | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
+| 1.2296        | 2.7778 | 600  | 1.3520          | 5.9339 | 2.4360 | 1.6730 | 0.1456  | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
+| 1.0909        | 3.2407 | 700  | 1.2565          | 5.3251 | 2.3076 | 1.5831 | 0.2332  | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
+| 1.0031        | 3.7037 | 800  | 1.2159          | 4.7598 | 2.1817 | 1.5709 | 0.3146  | 0.4570 | 0.3669    | 0.6057 | 0.6057   |
+| 0.9833        | 4.1667 | 900  | 1.1544          | 4.6141 | 2.1480 | 1.5031 | 0.3356  | 0.7296 | 0.8018    | 0.7572 | 0.7572   |
+| 0.825         | 4.6296 | 1000 | 1.1512          | 5.0019 | 2.2365 | 1.4608 | 0.2798  | 0.7757 | 0.7943    | 0.7859 | 0.7859   |
+| 0.8187        | 5.0926 | 1100 | 1.1150          | 4.9111 | 2.2161 | 1.4352 | 0.2928  | 0.7815 | 0.7849    | 0.7859 | 0.7859   |
+| 0.7138        | 5.5556 | 1200 | 1.0724          | 4.8492 | 2.2021 | 1.3871 | 0.3018  | 0.7766 | 0.7791    | 0.7807 | 0.7807   |
+| 0.6706        | 6.0185 | 1300 | 1.0560          | 4.9024 | 2.2141 | 1.3650 | 0.2941  | 0.7786 | 0.7823    | 0.7833 | 0.7833   |
+| 0.6112        | 6.4815 | 1400 | 1.0594          | 5.0772 | 2.2533 | 1.3694 | 0.2689  | 0.7750 | 0.7759    | 0.7781 | 0.7781   |
+| 0.5906        | 6.9444 | 1500 | 1.0611          | 5.1421 | 2.2676 | 1.3794 | 0.2596  | 0.7736 | 0.7734    | 0.7755 | 0.7755   |
+| 0.5597        | 7.4074 | 1600 | 1.0286          | 5.0419 | 2.2454 | 1.3290 | 0.2740  | 0.7839 | 0.7879    | 0.7885 | 0.7885   |
+| 0.5422        | 7.8704 | 1700 | 1.0531          | 5.2061 | 2.2817 | 1.3596 | 0.2504  | 0.7672 | 0.7678    | 0.7702 | 0.7702   |
+| 0.5255        | 8.3333 | 1800 | 1.0478          | 5.2565 | 2.2927 | 1.3372 | 0.2431  | 0.7811 | 0.7853    | 0.7859 | 0.7859   |
+| 0.5116        | 8.7963 | 1900 | 1.0544          | 5.2090 | 2.2823 | 1.3546 | 0.2500  | 0.7721 | 0.7733    | 0.7755 | 0.7755   |
+| 0.5213        | 9.2593 | 2000 | 1.0423          | 5.1715 | 2.2741 | 1.3341 | 0.2554  | 0.7819 | 0.7846    | 0.7859 | 0.7859   |
+| 0.4999        | 9.7222 | 2100 | 1.0566          | 5.2819 | 2.2982 | 1.3481 | 0.2395  | 0.7721 | 0.7733    | 0.7755 | 0.7755   |
+
+
+### Framework versions
+
+- Transformers 4.40.2
+- Pytorch 2.1.2
+- Datasets 2.18.0
+- Tokenizers 0.19.1

config.json

@@ -0,0 +1,35 @@
+{
+  "_name_or_path": "vinai/bertweet-base",
+  "architectures": [
+    "RobertaForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 130,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "tokenizer_class": "BertweetTokenizer",
+  "torch_dtype": "float32",
+  "transformers_version": "4.40.2",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 64001
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09aa20c0862cf6f3039e81f034d2805ebb01436047337b254d54ec9d9235d936
+size 539627092

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d71df7e40c3e1776e124c29ce7ee09781e8eafcc42ace40bf961e037dc7bde6
+size 4984