Training in progress, step 300, checkpoint

last-checkpoint/1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

last-checkpoint/2_Dense/config.json

@@ -0,0 +1 @@
+{"in_features": 768, "out_features": 768, "bias": true, "activation_function": "torch.nn.modules.activation.Tanh"}

last-checkpoint/2_Dense/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ab714fe9e913f78f9144b04c96c998fedfaede69a11bc5197ea955a5b55bef8
+size 2362528

last-checkpoint/README.md

@@ -0,0 +1,480 @@
+---
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:5749
+- loss:CosineSimilarityLoss
+base_model: CocoRoF/ModernBERT-SimCSE_v02
+widget:
+- source_sentence: 우리는 움직이는 동행 우주 정지 좌표계에 비례하여 이동하고 있습니다 ... 약 371km / s에서 별자리 leo
+    쪽으로. "
+  sentences:
+  - 두 마리의 독수리가 가지에 앉는다.
+  - 다른 물체와는 관련이 없는 '정지'는 없다.
+  - 소녀는 버스의 열린 문 앞에 서 있다.
+- source_sentence: 숲에는 개들이 있다.
+  sentences:
+  - 양을 보는 아이들.
+  - 여왕의 배우자를 "왕"이라고 부르지 않는 것은 아주 좋은 이유가 있다. 왜냐하면 그들은 왕이 아니기 때문이다.
+  - 개들은 숲속에 혼자 있다.
+- source_sentence: '첫째, 두 가지 다른 종류의 대시가 있다는 것을 알아야 합니다 : en 대시와 em 대시.'
+  sentences:
+  - 그들은 그 물건들을 집 주변에 두고 가거나 집의 정리를 해칠 의도가 없다.
+  - 세미콜론은 혼자 있을 수 있는 문장에 참여하는데 사용되지만, 그들의 관계를 강조하기 위해 결합됩니다.
+  - 그의 남동생이 지켜보는 동안 집 앞에서 트럼펫을 연주하는 금발의 아이.
+- source_sentence: 한 여성이 생선 껍질을 벗기고 있다.
+  sentences:
+  - 한 남자가 수영장으로 뛰어들었다.
+  - 한 여성이 프라이팬에 노란 혼합물을 부어 넣고 있다.
+  - 두 마리의 갈색 개가 눈 속에서 서로 놀고 있다.
+- source_sentence: 버스가 바쁜 길을 따라 운전한다.
+  sentences:
+  - 우리와 같은 태양계가 은하계 밖에서 존재할 수도 있을 것입니다.
+  - 그 여자는 데이트하러 가는 중이다.
+  - 녹색 버스가 도로를 따라 내려간다.
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+metrics:
+- pearson_cosine
+- spearman_cosine
+- pearson_euclidean
+- spearman_euclidean
+- pearson_manhattan
+- spearman_manhattan
+- pearson_dot
+- spearman_dot
+- pearson_max
+- spearman_max
+model-index:
+- name: SentenceTransformer based on CocoRoF/ModernBERT-SimCSE_v02
+  results:
+  - task:
+      type: semantic-similarity
+      name: Semantic Similarity
+    dataset:
+      name: sts dev
+      type: sts_dev
+    metrics:
+    - type: pearson_cosine
+      value: 0.8219368394963247
+      name: Pearson Cosine
+    - type: spearman_cosine
+      value: 0.8227360781964935
+      name: Spearman Cosine
+    - type: pearson_euclidean
+      value: 0.7729334760561297
+      name: Pearson Euclidean
+    - type: spearman_euclidean
+      value: 0.7748490630523356
+      name: Spearman Euclidean
+    - type: pearson_manhattan
+      value: 0.7754957053869553
+      name: Pearson Manhattan
+    - type: spearman_manhattan
+      value: 0.7782586536188661
+      name: Spearman Manhattan
+    - type: pearson_dot
+      value: 0.7469111462936613
+      name: Pearson Dot
+    - type: spearman_dot
+      value: 0.7392541828806165
+      name: Spearman Dot
+    - type: pearson_max
+      value: 0.8219368394963247
+      name: Pearson Max
+    - type: spearman_max
+      value: 0.8227360781964935
+      name: Spearman Max
+---
+
+# SentenceTransformer based on CocoRoF/ModernBERT-SimCSE_v02
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [CocoRoF/ModernBERT-SimCSE_v02](https://huggingface.co/CocoRoF/ModernBERT-SimCSE_v02). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [CocoRoF/ModernBERT-SimCSE_v02](https://huggingface.co/CocoRoF/ModernBERT-SimCSE_v02) <!-- at revision de4148c764893843e15a4e0b241fe308147a9aaa -->
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 768 dimensions
+- **Similarity Function:** Cosine Similarity
+<!-- - **Training Dataset:** Unknown -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: ModernBertModel 
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Dense({'in_features': 768, 'out_features': 768, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("CocoRoF/ModernBERT-SimCSE-multitask_v03")
+# Run inference
+sentences = [
+    '버스가 바쁜 길을 따라 운전한다.',
+    '녹색 버스가 도로를 따라 내려간다.',
+    '그 여자는 데이트하러 가는 중이다.',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 768]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Semantic Similarity
+
+* Dataset: `sts_dev`
+* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
+
+| Metric             | Value      |
+|:-------------------|:-----------|
+| pearson_cosine     | 0.8219     |
+| spearman_cosine    | 0.8227     |
+| pearson_euclidean  | 0.7729     |
+| spearman_euclidean | 0.7748     |
+| pearson_manhattan  | 0.7755     |
+| spearman_manhattan | 0.7783     |
+| pearson_dot        | 0.7469     |
+| spearman_dot       | 0.7393     |
+| pearson_max        | 0.8219     |
+| **spearman_max**   | **0.8227** |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### Unnamed Dataset
+
+
+* Size: 5,749 training samples
+* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | sentence1                                                                         | sentence2                                                                         | score                                                          |
+  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                            | float                                                          |
+  | details | <ul><li>min: 7 tokens</li><li>mean: 13.52 tokens</li><li>max: 36 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 13.41 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.45</li><li>max: 1.0</li></ul> |
+* Samples:
+  | sentence1                           | sentence2                                 | score             |
+  |:------------------------------------|:------------------------------------------|:------------------|
+  | <code>비행기가 이륙하고 있다.</code>          | <code>비행기가 이륙하고 있다.</code>                | <code>1.0</code>  |
+  | <code>한 남자가 큰 플루트를 연주하고 있다.</code>  | <code>남자가 플루트를 연주하고 있다.</code>            | <code>0.76</code> |
+  | <code>한 남자가 피자에 치즈를 뿌려놓고 있다.</code> | <code>한 남자가 구운 피자에 치즈 조각을 뿌려놓고 있다.</code> | <code>0.76</code> |
+* Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:
+  ```json
+  {
+      "loss_fct": "torch.nn.modules.loss.MSELoss"
+  }
+  ```
+
+### Evaluation Dataset
+
+#### Unnamed Dataset
+
+
+* Size: 1,500 evaluation samples
+* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | sentence1                                                                         | sentence2                                                                         | score                                                          |
+  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                            | float                                                          |
+  | details | <ul><li>min: 7 tokens</li><li>mean: 20.38 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 20.52 tokens</li><li>max: 54 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |
+* Samples:
+  | sentence1                            | sentence2                           | score             |
+  |:-------------------------------------|:------------------------------------|:------------------|
+  | <code>안전모를 가진 한 남자가 춤을 추고 있다.</code> | <code>안전모를 쓴 한 남자가 춤을 추고 있다.</code> | <code>1.0</code>  |
+  | <code>어린아이가 말을 타고 있다.</code>         | <code>아이가 말을 타고 있다.</code>          | <code>0.95</code> |
+  | <code>한 남자가 뱀에게 쥐를 먹이고 있다.</code>    | <code>남자가 뱀에게 쥐를 먹이고 있다.</code>     | <code>1.0</code>  |
+* Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:
+  ```json
+  {
+      "loss_fct": "torch.nn.modules.loss.MSELoss"
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `overwrite_output_dir`: True
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `gradient_accumulation_steps`: 8
+- `learning_rate`: 1e-05
+- `num_train_epochs`: 10.0
+- `warmup_ratio`: 0.1
+- `push_to_hub`: True
+- `hub_model_id`: CocoRoF/ModernBERT-SimCSE-multitask_v03
+- `hub_strategy`: checkpoint
+- `batch_sampler`: no_duplicates
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: True
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 8
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 1e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 10.0
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: True
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: True
+- `resume_from_checkpoint`: None
+- `hub_model_id`: CocoRoF/ModernBERT-SimCSE-multitask_v03
+- `hub_strategy`: checkpoint
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: no_duplicates
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+| Epoch  | Step | Training Loss | Validation Loss | sts_dev_spearman_max |
+|:------:|:----:|:-------------:|:---------------:|:--------------------:|
+| 0.2228 | 10   | 0.0283        | -               | -                    |
+| 0.4457 | 20   | 0.0344        | -               | -                    |
+| 0.6685 | 30   | 0.0305        | 0.0310          | 0.7939               |
+| 0.8914 | 40   | 0.0489        | -               | -                    |
+| 1.1337 | 50   | 0.0382        | -               | -                    |
+| 1.3565 | 60   | 0.0271        | 0.0293          | 0.7994               |
+| 1.5794 | 70   | 0.0344        | -               | -                    |
+| 1.8022 | 80   | 0.0382        | -               | -                    |
+| 2.0446 | 90   | 0.0419        | 0.0280          | 0.8059               |
+| 2.2674 | 100  | 0.0244        | -               | -                    |
+| 2.4903 | 110  | 0.0307        | -               | -                    |
+| 2.7131 | 120  | 0.0291        | 0.0269          | 0.8108               |
+| 2.9359 | 130  | 0.038         | -               | -                    |
+| 3.1783 | 140  | 0.0269        | -               | -                    |
+| 3.4011 | 150  | 0.0268        | 0.0262          | 0.8155               |
+| 3.6240 | 160  | 0.0246        | -               | -                    |
+| 3.8468 | 170  | 0.0313        | -               | -                    |
+| 4.0891 | 180  | 0.0303        | 0.0259          | 0.8185               |
+| 4.3120 | 190  | 0.0198        | -               | -                    |
+| 4.5348 | 200  | 0.0257        | -               | -                    |
+| 4.7577 | 210  | 0.0242        | 0.0255          | 0.8202               |
+| 4.9805 | 220  | 0.0293        | -               | -                    |
+| 5.2228 | 230  | 0.0193        | -               | -                    |
+| 5.4457 | 240  | 0.0222        | 0.0254          | 0.8222               |
+| 5.6685 | 250  | 0.0184        | -               | -                    |
+| 5.8914 | 260  | 0.0243        | -               | -                    |
+| 6.1337 | 270  | 0.0204        | 0.0254          | 0.8235               |
+| 6.3565 | 280  | 0.0147        | -               | -                    |
+| 6.5794 | 290  | 0.0196        | -               | -                    |
+| 6.8022 | 300  | 0.0176        | 0.0253          | 0.8227               |
+
+
+### Framework Versions
+- Python: 3.11.10
+- Sentence Transformers: 3.3.1
+- Transformers: 4.48.0.dev0
+- PyTorch: 2.5.1+cu121
+- Accelerate: 1.1.0
+- Datasets: 3.1.0
+- Tokenizers: 0.21.0
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

last-checkpoint/config.json

@@ -0,0 +1,47 @@
+{
+  "_name_or_path": "CocoRoF/ModernBERT-SimCSE_v02",
+  "architectures": [
+    "ModernBertModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 50281,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 50281,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "embedding_dropout": 0.0,
+  "eos_token_id": 50282,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000.0,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 10000.0,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 50283,
+  "position_embedding_type": "absolute",
+  "reference_compile": false,
+  "repad_logits_with_grad": false,
+  "sep_token_id": 50282,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.0.dev0",
+  "vocab_size": 95663
+}

last-checkpoint/config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.3.1",
+    "transformers": "4.48.0.dev0",
+    "pytorch": "2.5.1+cu121"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

last-checkpoint/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0089cd9e1eb926c27edf4a3eba92fe4d1b1affb5a38446b48a4d318fab339be6
+size 735216376

last-checkpoint/modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Dense",
+    "type": "sentence_transformers.models.Dense"
+  }
+]

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6669f86a183384b3de31120ee22eb7b3339e490b7f29c07921954824f0c474c6
+size 1475248442

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba6ff2ae7c5593b8218c9e3d922c03bdd80a2dfd192d086d778ccd95adce4be2
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5986d6e6104e590e997adda30b312d4039c5eb05173d5aab3bf76131f1a91b4f
+size 1000

last-checkpoint/sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": false
+}

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,945 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "|||IP_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "1": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50254": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50255": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50256": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50257": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "|||EMAIL_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50278": {
+      "content": "|||PHONE_NUMBER|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50279": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50280": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50281": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50282": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50283": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50284": {
+      "content": "[MASK]",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "95580": {
+      "content": "[unused0]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95581": {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95582": {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95583": {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95584": {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95585": {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95586": {
+      "content": "[unused6]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95587": {
+      "content": "[unused7]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95588": {
+      "content": "[unused8]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95589": {
+      "content": "[unused9]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95590": {
+      "content": "[unused10]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95591": {
+      "content": "[unused11]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95592": {
+      "content": "[unused12]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95593": {
+      "content": "[unused13]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95594": {
+      "content": "[unused14]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95595": {
+      "content": "[unused15]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95596": {
+      "content": "[unused16]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95597": {
+      "content": "[unused17]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95598": {
+      "content": "[unused18]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95599": {
+      "content": "[unused19]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95600": {
+      "content": "[unused20]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95601": {
+      "content": "[unused21]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95602": {
+      "content": "[unused22]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95603": {
+      "content": "[unused23]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95604": {
+      "content": "[unused24]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95605": {
+      "content": "[unused25]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95606": {
+      "content": "[unused26]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95607": {
+      "content": "[unused27]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95608": {
+      "content": "[unused28]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95609": {
+      "content": "[unused29]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95610": {
+      "content": "[unused30]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95611": {
+      "content": "[unused31]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95612": {
+      "content": "[unused32]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95613": {
+      "content": "[unused33]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95614": {
+      "content": "[unused34]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95615": {
+      "content": "[unused35]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95616": {
+      "content": "[unused36]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95617": {
+      "content": "[unused37]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95618": {
+      "content": "[unused38]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95619": {
+      "content": "[unused39]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95620": {
+      "content": "[unused40]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95621": {
+      "content": "[unused41]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95622": {
+      "content": "[unused42]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95623": {
+      "content": "[unused43]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95624": {
+      "content": "[unused44]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95625": {
+      "content": "[unused45]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95626": {
+      "content": "[unused46]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95627": {
+      "content": "[unused47]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95628": {
+      "content": "[unused48]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95629": {
+      "content": "[unused49]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95630": {
+      "content": "[unused50]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95631": {
+      "content": "[unused51]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95632": {
+      "content": "[unused52]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95633": {
+      "content": "[unused53]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95634": {
+      "content": "[unused54]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95635": {
+      "content": "[unused55]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95636": {
+      "content": "[unused56]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95637": {
+      "content": "[unused57]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95638": {
+      "content": "[unused58]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95639": {
+      "content": "[unused59]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95640": {
+      "content": "[unused60]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95641": {
+      "content": "[unused61]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95642": {
+      "content": "[unused62]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95643": {
+      "content": "[unused63]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95644": {
+      "content": "[unused64]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95645": {
+      "content": "[unused65]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95646": {
+      "content": "[unused66]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95647": {
+      "content": "[unused67]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95648": {
+      "content": "[unused68]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95649": {
+      "content": "[unused69]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95650": {
+      "content": "[unused70]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95651": {
+      "content": "[unused71]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95652": {
+      "content": "[unused72]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95653": {
+      "content": "[unused73]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95654": {
+      "content": "[unused74]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95655": {
+      "content": "[unused75]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95656": {
+      "content": "[unused76]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95657": {
+      "content": "[unused77]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95658": {
+      "content": "[unused78]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95659": {
+      "content": "[unused79]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95660": {
+      "content": "[unused80]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95661": {
+      "content": "[unused81]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95662": {
+      "content": "[unused82]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "[UNK]"
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,423 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 6.802228412256268,
+  "eval_steps": 30,
+  "global_step": 300,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.22284122562674094,
+      "grad_norm": 0.2538502514362335,
+      "learning_rate": 2.785515320334262e-07,
+      "loss": 0.0283,
+      "step": 10
+    },
+    {
+      "epoch": 0.4456824512534819,
+      "grad_norm": 0.13635846972465515,
+      "learning_rate": 5.571030640668524e-07,
+      "loss": 0.0344,
+      "step": 20
+    },
+    {
+      "epoch": 0.6685236768802229,
+      "grad_norm": 0.25132906436920166,
+      "learning_rate": 8.356545961002786e-07,
+      "loss": 0.0305,
+      "step": 30
+    },
+    {
+      "epoch": 0.6685236768802229,
+      "eval_loss": 0.030975496396422386,
+      "eval_runtime": 6.5316,
+      "eval_samples_per_second": 229.652,
+      "eval_steps_per_second": 14.392,
+      "eval_sts_dev_pearson_cosine": 0.7953296758719961,
+      "eval_sts_dev_pearson_dot": 0.6855921619048916,
+      "eval_sts_dev_pearson_euclidean": 0.7647603423822984,
+      "eval_sts_dev_pearson_manhattan": 0.7662305710281121,
+      "eval_sts_dev_pearson_max": 0.7953296758719961,
+      "eval_sts_dev_spearman_cosine": 0.7938998183894888,
+      "eval_sts_dev_spearman_dot": 0.6701160606364611,
+      "eval_sts_dev_spearman_euclidean": 0.764275064463694,
+      "eval_sts_dev_spearman_manhattan": 0.7663956716038323,
+      "eval_sts_dev_spearman_max": 0.7938998183894888,
+      "step": 30
+    },
+    {
+      "epoch": 0.8913649025069638,
+      "grad_norm": 0.2590219974517822,
+      "learning_rate": 1.1142061281337048e-06,
+      "loss": 0.0489,
+      "step": 40
+    },
+    {
+      "epoch": 1.1337047353760445,
+      "grad_norm": 0.2477671355009079,
+      "learning_rate": 1.392757660167131e-06,
+      "loss": 0.0382,
+      "step": 50
+    },
+    {
+      "epoch": 1.3565459610027855,
+      "grad_norm": 0.2230578511953354,
+      "learning_rate": 1.6713091922005572e-06,
+      "loss": 0.0271,
+      "step": 60
+    },
+    {
+      "epoch": 1.3565459610027855,
+      "eval_loss": 0.02927413582801819,
+      "eval_runtime": 6.1022,
+      "eval_samples_per_second": 245.812,
+      "eval_steps_per_second": 15.404,
+      "eval_sts_dev_pearson_cosine": 0.8001627825550413,
+      "eval_sts_dev_pearson_dot": 0.7013280153939746,
+      "eval_sts_dev_pearson_euclidean": 0.7629781135707555,
+      "eval_sts_dev_pearson_manhattan": 0.7647370302448242,
+      "eval_sts_dev_pearson_max": 0.8001627825550413,
+      "eval_sts_dev_spearman_cosine": 0.7994084764965521,
+      "eval_sts_dev_spearman_dot": 0.6877298483304968,
+      "eval_sts_dev_spearman_euclidean": 0.7623008729981257,
+      "eval_sts_dev_spearman_manhattan": 0.7650295208380897,
+      "eval_sts_dev_spearman_max": 0.7994084764965521,
+      "step": 60
+    },
+    {
+      "epoch": 1.5793871866295266,
+      "grad_norm": 0.23978064954280853,
+      "learning_rate": 1.9498607242339835e-06,
+      "loss": 0.0344,
+      "step": 70
+    },
+    {
+      "epoch": 1.8022284122562673,
+      "grad_norm": 0.2269248366355896,
+      "learning_rate": 2.2284122562674097e-06,
+      "loss": 0.0382,
+      "step": 80
+    },
+    {
+      "epoch": 2.0445682451253484,
+      "grad_norm": 0.1311478465795517,
+      "learning_rate": 2.506963788300836e-06,
+      "loss": 0.0419,
+      "step": 90
+    },
+    {
+      "epoch": 2.0445682451253484,
+      "eval_loss": 0.0279527697712183,
+      "eval_runtime": 6.1525,
+      "eval_samples_per_second": 243.802,
+      "eval_steps_per_second": 15.278,
+      "eval_sts_dev_pearson_cosine": 0.8052740525083868,
+      "eval_sts_dev_pearson_dot": 0.7129779531910554,
+      "eval_sts_dev_pearson_euclidean": 0.7630256540163647,
+      "eval_sts_dev_pearson_manhattan": 0.7649555842254796,
+      "eval_sts_dev_pearson_max": 0.8052740525083868,
+      "eval_sts_dev_spearman_cosine": 0.805932936440032,
+      "eval_sts_dev_spearman_dot": 0.7013448783489886,
+      "eval_sts_dev_spearman_euclidean": 0.762706783236441,
+      "eval_sts_dev_spearman_manhattan": 0.7655443912587759,
+      "eval_sts_dev_spearman_max": 0.805932936440032,
+      "step": 90
+    },
+    {
+      "epoch": 2.267409470752089,
+      "grad_norm": 0.15666936337947845,
+      "learning_rate": 2.785515320334262e-06,
+      "loss": 0.0244,
+      "step": 100
+    },
+    {
+      "epoch": 2.4902506963788302,
+      "grad_norm": 0.14549851417541504,
+      "learning_rate": 3.064066852367688e-06,
+      "loss": 0.0307,
+      "step": 110
+    },
+    {
+      "epoch": 2.713091922005571,
+      "grad_norm": 0.20197178423404694,
+      "learning_rate": 3.3426183844011143e-06,
+      "loss": 0.0291,
+      "step": 120
+    },
+    {
+      "epoch": 2.713091922005571,
+      "eval_loss": 0.02694467455148697,
+      "eval_runtime": 6.528,
+      "eval_samples_per_second": 229.78,
+      "eval_steps_per_second": 14.4,
+      "eval_sts_dev_pearson_cosine": 0.8095317257793349,
+      "eval_sts_dev_pearson_dot": 0.7228217786137938,
+      "eval_sts_dev_pearson_euclidean": 0.7635943588878411,
+      "eval_sts_dev_pearson_manhattan": 0.7656672001584354,
+      "eval_sts_dev_pearson_max": 0.8095317257793349,
+      "eval_sts_dev_spearman_cosine": 0.8107539995821735,
+      "eval_sts_dev_spearman_dot": 0.7126247484390617,
+      "eval_sts_dev_spearman_euclidean": 0.7634838306489425,
+      "eval_sts_dev_spearman_manhattan": 0.7664168478564297,
+      "eval_sts_dev_spearman_max": 0.8107539995821735,
+      "step": 120
+    },
+    {
+      "epoch": 2.935933147632312,
+      "grad_norm": 0.2107369303703308,
+      "learning_rate": 3.6211699164345405e-06,
+      "loss": 0.038,
+      "step": 130
+    },
+    {
+      "epoch": 3.1782729805013927,
+      "grad_norm": 0.15846215188503265,
+      "learning_rate": 3.899721448467967e-06,
+      "loss": 0.0269,
+      "step": 140
+    },
+    {
+      "epoch": 3.401114206128134,
+      "grad_norm": 0.17715278267860413,
+      "learning_rate": 4.178272980501394e-06,
+      "loss": 0.0268,
+      "step": 150
+    },
+    {
+      "epoch": 3.401114206128134,
+      "eval_loss": 0.026173867285251617,
+      "eval_runtime": 6.306,
+      "eval_samples_per_second": 237.869,
+      "eval_steps_per_second": 14.906,
+      "eval_sts_dev_pearson_cosine": 0.8136326182189031,
+      "eval_sts_dev_pearson_dot": 0.7289342393989602,
+      "eval_sts_dev_pearson_euclidean": 0.7658102043154281,
+      "eval_sts_dev_pearson_manhattan": 0.7680399446033591,
+      "eval_sts_dev_pearson_max": 0.8136326182189031,
+      "eval_sts_dev_spearman_cosine": 0.8154563967795785,
+      "eval_sts_dev_spearman_dot": 0.7204276033712009,
+      "eval_sts_dev_spearman_euclidean": 0.7661516256266799,
+      "eval_sts_dev_spearman_manhattan": 0.7692973830139536,
+      "eval_sts_dev_spearman_max": 0.8154563967795785,
+      "step": 150
+    },
+    {
+      "epoch": 3.6239554317548746,
+      "grad_norm": 0.1337411254644394,
+      "learning_rate": 4.456824512534819e-06,
+      "loss": 0.0246,
+      "step": 160
+    },
+    {
+      "epoch": 3.8467966573816157,
+      "grad_norm": 0.20471176505088806,
+      "learning_rate": 4.735376044568246e-06,
+      "loss": 0.0313,
+      "step": 170
+    },
+    {
+      "epoch": 4.089136490250697,
+      "grad_norm": 0.12327426671981812,
+      "learning_rate": 5.013927576601672e-06,
+      "loss": 0.0303,
+      "step": 180
+    },
+    {
+      "epoch": 4.089136490250697,
+      "eval_loss": 0.02586401253938675,
+      "eval_runtime": 6.8399,
+      "eval_samples_per_second": 219.3,
+      "eval_steps_per_second": 13.743,
+      "eval_sts_dev_pearson_cosine": 0.8163121986548724,
+      "eval_sts_dev_pearson_dot": 0.7330841259509188,
+      "eval_sts_dev_pearson_euclidean": 0.7674859088604027,
+      "eval_sts_dev_pearson_manhattan": 0.7697974598144367,
+      "eval_sts_dev_pearson_max": 0.8163121986548724,
+      "eval_sts_dev_spearman_cosine": 0.8184908732804921,
+      "eval_sts_dev_spearman_dot": 0.7250521959658871,
+      "eval_sts_dev_spearman_euclidean": 0.7684563123887144,
+      "eval_sts_dev_spearman_manhattan": 0.7715573641686395,
+      "eval_sts_dev_spearman_max": 0.8184908732804921,
+      "step": 180
+    },
+    {
+      "epoch": 4.311977715877437,
+      "grad_norm": 0.11181030422449112,
+      "learning_rate": 5.292479108635098e-06,
+      "loss": 0.0198,
+      "step": 190
+    },
+    {
+      "epoch": 4.534818941504178,
+      "grad_norm": 0.11830934137105942,
+      "learning_rate": 5.571030640668524e-06,
+      "loss": 0.0257,
+      "step": 200
+    },
+    {
+      "epoch": 4.757660167130919,
+      "grad_norm": 0.1775977462530136,
+      "learning_rate": 5.849582172701951e-06,
+      "loss": 0.0242,
+      "step": 210
+    },
+    {
+      "epoch": 4.757660167130919,
+      "eval_loss": 0.02551957406103611,
+      "eval_runtime": 6.4245,
+      "eval_samples_per_second": 233.481,
+      "eval_steps_per_second": 14.631,
+      "eval_sts_dev_pearson_cosine": 0.8184173000480589,
+      "eval_sts_dev_pearson_dot": 0.7369533513611706,
+      "eval_sts_dev_pearson_euclidean": 0.7687482582532739,
+      "eval_sts_dev_pearson_manhattan": 0.7712300663924829,
+      "eval_sts_dev_pearson_max": 0.8184173000480589,
+      "eval_sts_dev_spearman_cosine": 0.8201930470486518,
+      "eval_sts_dev_spearman_dot": 0.7292325959243812,
+      "eval_sts_dev_spearman_euclidean": 0.7696170592602297,
+      "eval_sts_dev_spearman_manhattan": 0.7729809111066369,
+      "eval_sts_dev_spearman_max": 0.8201930470486518,
+      "step": 210
+    },
+    {
+      "epoch": 4.9805013927576605,
+      "grad_norm": 0.23354189097881317,
+      "learning_rate": 6.128133704735376e-06,
+      "loss": 0.0293,
+      "step": 220
+    },
+    {
+      "epoch": 5.222841225626741,
+      "grad_norm": 0.12718431651592255,
+      "learning_rate": 6.406685236768803e-06,
+      "loss": 0.0193,
+      "step": 230
+    },
+    {
+      "epoch": 5.445682451253482,
+      "grad_norm": 0.1111082211136818,
+      "learning_rate": 6.685236768802229e-06,
+      "loss": 0.0222,
+      "step": 240
+    },
+    {
+      "epoch": 5.445682451253482,
+      "eval_loss": 0.02539980411529541,
+      "eval_runtime": 6.3582,
+      "eval_samples_per_second": 235.915,
+      "eval_steps_per_second": 14.784,
+      "eval_sts_dev_pearson_cosine": 0.8203051470878093,
+      "eval_sts_dev_pearson_dot": 0.7391973842870876,
+      "eval_sts_dev_pearson_euclidean": 0.7710328054708023,
+      "eval_sts_dev_pearson_manhattan": 0.7734981812206646,
+      "eval_sts_dev_pearson_max": 0.8203051470878093,
+      "eval_sts_dev_spearman_cosine": 0.8222047787628998,
+      "eval_sts_dev_spearman_dot": 0.7306726496212352,
+      "eval_sts_dev_spearman_euclidean": 0.7721080064054946,
+      "eval_sts_dev_spearman_manhattan": 0.7758967012553709,
+      "eval_sts_dev_spearman_max": 0.8222047787628998,
+      "step": 240
+    },
+    {
+      "epoch": 5.6685236768802225,
+      "grad_norm": 0.167997807264328,
+      "learning_rate": 6.963788300835655e-06,
+      "loss": 0.0184,
+      "step": 250
+    },
+    {
+      "epoch": 5.891364902506964,
+      "grad_norm": 0.18360492587089539,
+      "learning_rate": 7.242339832869081e-06,
+      "loss": 0.0243,
+      "step": 260
+    },
+    {
+      "epoch": 6.133704735376044,
+      "grad_norm": 0.11399545520544052,
+      "learning_rate": 7.5208913649025075e-06,
+      "loss": 0.0204,
+      "step": 270
+    },
+    {
+      "epoch": 6.133704735376044,
+      "eval_loss": 0.025426626205444336,
+      "eval_runtime": 6.3377,
+      "eval_samples_per_second": 236.678,
+      "eval_steps_per_second": 14.832,
+      "eval_sts_dev_pearson_cosine": 0.8215923043460271,
+      "eval_sts_dev_pearson_dot": 0.7427941063103285,
+      "eval_sts_dev_pearson_euclidean": 0.7725242056053008,
+      "eval_sts_dev_pearson_manhattan": 0.7749558209132376,
+      "eval_sts_dev_pearson_max": 0.8215923043460271,
+      "eval_sts_dev_spearman_cosine": 0.8234628421089484,
+      "eval_sts_dev_spearman_dot": 0.7343279809432616,
+      "eval_sts_dev_spearman_euclidean": 0.7742054612821838,
+      "eval_sts_dev_spearman_manhattan": 0.777339758218875,
+      "eval_sts_dev_spearman_max": 0.8234628421089484,
+      "step": 270
+    },
+    {
+      "epoch": 6.3565459610027855,
+      "grad_norm": 0.14734485745429993,
+      "learning_rate": 7.799442896935934e-06,
+      "loss": 0.0147,
+      "step": 280
+    },
+    {
+      "epoch": 6.579387186629527,
+      "grad_norm": 0.14232878386974335,
+      "learning_rate": 8.07799442896936e-06,
+      "loss": 0.0196,
+      "step": 290
+    },
+    {
+      "epoch": 6.802228412256268,
+      "grad_norm": 0.12475496530532837,
+      "learning_rate": 8.356545961002787e-06,
+      "loss": 0.0176,
+      "step": 300
+    },
+    {
+      "epoch": 6.802228412256268,
+      "eval_loss": 0.025328340008854866,
+      "eval_runtime": 6.1771,
+      "eval_samples_per_second": 242.832,
+      "eval_steps_per_second": 15.217,
+      "eval_sts_dev_pearson_cosine": 0.8219368394963247,
+      "eval_sts_dev_pearson_dot": 0.7469111462936613,
+      "eval_sts_dev_pearson_euclidean": 0.7729334760561297,
+      "eval_sts_dev_pearson_manhattan": 0.7754957053869553,
+      "eval_sts_dev_pearson_max": 0.8219368394963247,
+      "eval_sts_dev_spearman_cosine": 0.8227360781964935,
+      "eval_sts_dev_spearman_dot": 0.7392541828806165,
+      "eval_sts_dev_spearman_euclidean": 0.7748490630523356,
+      "eval_sts_dev_spearman_manhattan": 0.7782586536188661,
+      "eval_sts_dev_spearman_max": 0.8227360781964935,
+      "step": 300
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 440,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 300,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 16,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2afca26b81088c945d1adc33c05fe570f26a511dc3144dc1c153eee3ff6326de
+size 5688