RK-1235/bge-base-FIR-matryoshka-BASELINE-10ep-triplet

SentenceTransformer based on BAAI/bge-base-en

This is a sentence-transformers model finetuned from BAAI/bge-base-en on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • json

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("RK-1235/bge-base-FIR-matryoshka-BASELINE-10ep-triplet")
# Run inference
sentences = [
    "(First Information Report): A plastic bottle containing approximately 10 liters of raw illegal liquor and an arrest under Section 60(1) Ex Act, Thana Asandra district Barabanki on today's date 24/01/2024, I, H/O Amar Bahadur Singh, along with H/O Ramji Yadav and H/O Bharat Sharma, departed from the police station towards the highway barrier for peacekeeping and inspection in the area. The special informant came and informed that a person from Devra Jhalia village was coming towards the new road carrying a bottle with raw illegal liquor intended for sale, and if we acted quickly, he could be caught. Believing the informant's information, we proceeded towards the address with the informant. Just before reaching Devra Jhalia village, the informant pointed out that the person coming was the one they were looking for and then left. As soon as we, the police, turned towards the village, the individual noticed us and became suspicious, trying to flee but was surrounded and caught at the Devra",
    'On what date did the incident involving the arrest under Section 60(1) Ex Act take place?',
    'What was found in the right pocket of the pants worn by Siraj Ismail during the search?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Dataset: dim_768
• Evaluated with InformationRetrievalEvaluator with these parameters:

  {
      "truncate_dim": 768
  }
  

Metric	Value
cosine_accuracy@1	0.254
cosine_accuracy@3	0.5088
cosine_accuracy@5	0.5883
cosine_accuracy@10	0.6745
cosine_precision@1	0.254
cosine_precision@3	0.1696
cosine_precision@5	0.1177
cosine_precision@10	0.0674
cosine_recall@1	0.254
cosine_recall@3	0.5088
cosine_recall@5	0.5883
cosine_recall@10	0.6745
cosine_ndcg@10	0.4691
cosine_mrr@10	0.4031
cosine_map@100	0.4124

Information Retrieval

• Dataset: dim_512
• Evaluated with InformationRetrievalEvaluator with these parameters:

  {
      "truncate_dim": 512
  }
  

Metric	Value
cosine_accuracy@1	0.2394
cosine_accuracy@3	0.5015
cosine_accuracy@5	0.5803
cosine_accuracy@10	0.662
cosine_precision@1	0.2394
cosine_precision@3	0.1672
cosine_precision@5	0.1161
cosine_precision@10	0.0662
cosine_recall@1	0.2394
cosine_recall@3	0.5015
cosine_recall@5	0.5803
cosine_recall@10	0.662
cosine_ndcg@10	0.4569
cosine_mrr@10	0.3907
cosine_map@100	0.4005

Information Retrieval

• Dataset: dim_256
• Evaluated with InformationRetrievalEvaluator with these parameters:

  {
      "truncate_dim": 256
  }
  

Metric	Value
cosine_accuracy@1	0.2409
cosine_accuracy@3	0.4839
cosine_accuracy@5	0.5642
cosine_accuracy@10	0.6445
cosine_precision@1	0.2409
cosine_precision@3	0.1613
cosine_precision@5	0.1128
cosine_precision@10	0.0645
cosine_recall@1	0.2409
cosine_recall@3	0.4839
cosine_recall@5	0.5642
cosine_recall@10	0.6445
cosine_ndcg@10	0.4469
cosine_mrr@10	0.3834
cosine_map@100	0.3929

Information Retrieval

• Dataset: dim_128
• Evaluated with InformationRetrievalEvaluator with these parameters:

  {
      "truncate_dim": 128
  }
  

Metric	Value
cosine_accuracy@1	0.2168
cosine_accuracy@3	0.4416
cosine_accuracy@5	0.5248
cosine_accuracy@10	0.608
cosine_precision@1	0.2168
cosine_precision@3	0.1472
cosine_precision@5	0.105
cosine_precision@10	0.0608
cosine_recall@1	0.2168
cosine_recall@3	0.4416
cosine_recall@5	0.5248
cosine_recall@10	0.608
cosine_ndcg@10	0.4149
cosine_mrr@10	0.353
cosine_map@100	0.3632

Information Retrieval

• Dataset: dim_64
• Evaluated with InformationRetrievalEvaluator with these parameters:

  {
      "truncate_dim": 64
  }
  

Metric	Value
cosine_accuracy@1	0.1934
cosine_accuracy@3	0.4036
cosine_accuracy@5	0.4657
cosine_accuracy@10	0.5569
cosine_precision@1	0.1934
cosine_precision@3	0.1345
cosine_precision@5	0.0931
cosine_precision@10	0.0557
cosine_recall@1	0.1934
cosine_recall@3	0.4036
cosine_recall@5	0.4657
cosine_recall@10	0.5569
cosine_ndcg@10	0.3746
cosine_mrr@10	0.3167
cosine_map@100	0.3272

Training Details

Training Dataset

json

• Dataset: json
• Size: 3,019 training samples
• Columns: positive, anchor, and negative
• Approximate statistics based on the first 1000 samples:

  	positive	anchor	negative
  type	string	string	list
  details	min: 14 tokens mean: 198.49 tokens max: 413 tokens	min: 8 tokens mean: 19.28 tokens max: 41 tokens	size: 5 elements

• Samples:

  positive	anchor	negative
  First Information Report: Officer Thanadhiksh Thahbarpur Ajmer respectfully submits that the petitioner Manju Yadav states that my husband Santosh Yadav, son Sw. Harikesh Yadav, brother-in-law Mayank Yadav, son Sw. Heetikesh Yadav, and mother-in-law Radhika, wife of Sw. Harikesh Yadav, these three people have beaten me together. If the villagers had not intervened, these three would have killed me and my son. The constant intention of my mother-in-law Radhika, wife of Sw. Harikesh Yadav, is to beat me to death. My husband Santosh Yadav and brother-in-law Mayank Yadav have been indecent with me. My husband cut off my ear while my brother-in-law Mayank Yadav tore my sari and tried to undress me, which is why I was trying to protect myself with bricks. If the villagers had not intervened, these people would have undressed and killed me. These three come every day asking for dowry from me and beat me, insult me, and abuse me. There is a danger to my life and my son's life from these three	Who are the three individuals mentioned in the report who have beaten Manju Yadav?	["for firing. An ejector is attached at the back of the barrel for loading and unloading the cartridge. The body and trigger are joined together with a steel plate. The seized cartridge is of 315 bore, made of brass, with 'KF 8 MM' written on its base. The accused's actions fall under the offense punishable under Section 3/25A Act. At around 10:45 PM, he was taken into custody and informed of the charges. Due to the sudden occurrence in an isolated area, no witnesses were available. During the arrest and seizure, strict adherence to the orders and directives of the High Court and Human Rights Commission was maintained. An arrest memo was prepared, and the accused's arrest was communicated to his brother Jitendra Rajput at mobile number 9695404597. The seized caliber and cartridge were sealed in the clothes they were found in and stamped with a seal, and a sample stamp was prepared. The report was written and read out in the light of the torch, and signatures of all concerned were", "rights were fully followed. An arrest memo was prepared, and notice of the arrest will be given to the accused's relatives at the police station. The report was read out to the accused on the spot for signature as evidence. SD HC Manoj SD Hindi Sovinder Kumar SD HC Aditya SD HC Balkishan SD English Unreadable Karmanvir Up NI Thana SI Line Munger 9.3.24 Note: A copy of the report was given to the accused, and fingerprints were taken. Signature Thumbprint Dilshad SD English Unreadable Karmanvir Up NI Thana SI Line District Mujjafarnagar Date 9.3.2024 Note: I hereby certify that the above report was copied by me. 4 N.C R.B (N.C.R.B)
  First Information Report: Officer Thanadhiksh Thahbarpur Ajmer respectfully submits that the petitioner Manju Yadav states that my husband Santosh Yadav, son Sw. Harikesh Yadav, brother-in-law Mayank Yadav, son Sw. Heetikesh Yadav, and mother-in-law Radhika, wife of Sw. Harikesh Yadav, these three people have beaten me together. If the villagers had not intervened, these three would have killed me and my son. The constant intention of my mother-in-law Radhika, wife of Sw. Harikesh Yadav, is to beat me to death. My husband Santosh Yadav and brother-in-law Mayank Yadav have been indecent with me. My husband cut off my ear while my brother-in-law Mayank Yadav tore my sari and tried to undress me, which is why I was trying to protect myself with bricks. If the villagers had not intervened, these people would have undressed and killed me. These three come every day asking for dowry from me and beat me, insult me, and abuse me. There is a danger to my life and my son's life from these three	What actions did Manju Yadav's husband Santosh Yadav and brother-in-law Mayank Yadav take against her?	['First Information Report: On receiving information, two batches of illegal raw liquor totaling 18 liters were seized, and one person was arrested under Section 60(1) EX Act at Badosaray Police Station, Barabanki district on April 4, 2024. I, Sub-Inspector Salik Ray, along with Vinay Kumar Verma, Alauk Pandey, and Deepak Kumar, set out from Thanha Hajja towards Baraulia Chauraha Cafekirav where suspicious individuals and vehicles were reported. We were discussing measures to curb the production and sale of raw liquor when an informant approached and informed us that a man carrying raw liquor in both hands was heading towards Kotwa Dhama via Amara Katohra Road towards Sita Devi Mahavidyalaya. Believing the information, we proceeded to the location indicated by the informant along with Inspector Sanjay Singh and Excise Inspector Shivani Singh Chauhan. As we turned towards Sita Devi 3 NC RB (NCRB) I.L.F.-I (Unified Investigation Form - I), we headed towards Sita Devi School. After', "First Information Report: During the course of duty, one illegal chaku was seized and one person was arrested under Section 425 Arms Act on the date 30.01.24. Today, I, No. 2079 Anmol Kumar May Hana Rahi, along with No. 2031 Shravan Kumar Chita 11, were posted and conducting an investigation and search request letter stoppage crime jail Tamil Chauki area when we, the employees, reached the Karakala Tiraha while patrolling, a special informant came to us and informed that a person who has an illegal chaku is sitting near the back wall of the small minaret and can be caught quickly if we hurry. We, the police, believing the informant's information, tried to get testimony from the people coming and going, but due to good or bad reasons, no one was willing to testify and left without giving their name and address. Then we, the police, searched each other's clothes and believed that we did not have any item related to the crime. Subsequently, we, the police, went to the place indicated by", "First Information Report: Counterfeit Case --- Seizure Report 18 bottles of country liquor and arrest of one person under Section 60 Ex Act PS Eagles Road Aligarh on today's date 17/3/24 by SI Jitendra Kumar we were checking suspicious individuals and vehicles in the police station area on Sasni Road when we were checking at Kaka Chowk, a person coming from the Eagles direction showed a plastic bag in his hand. Suspecting something, we stopped him and signaled with a torch light to stop, but he quickly started running back towards Eagles. Suspecting further, we chased him and caught him on Main Road near Eagles, about 20 steps away from Kaka Chowk at 22:30 hours. After searching him, we found 3 NC RB (N.C.R.B.) I.L.F.-I (Unified Investigation Form - I) and questioned him. He gave his name as Lal Singh, son of Gopali Ram, from village Nagla Ahivasi, Police Station Eagles Aligarh. Checking the bag in his hand, we found 13 bottles of Joshila country liquor (spiced) (quantity 200 ml) and 5", "Urmila Devi's wife Vidyadhara killed her and burned her body in the house to hide the truth. Kanhaya Lal first asked the petitioner to come to Azamgarh, then said to come to Tahbarpur, and then cut off the phone. The petitioner is giving this information. Therefore, it is requested that the FIR be registered and legal action be taken against the accused persons. Date 18.03.2024, Petitioner Sudip Yadav's son Madhuban Yadav, Gram Pashchim Patti, Thana Ahiraula, Janpad Azamgarh, signature Sudipyadav HMICM, case registered according to rules, signature illegible, Thana Adhyaksh Thana Tahbarpur, date 18.03.2024, Note verified that the copy of the document was dictated by me HEMU Anand Kumar Maurya on the CCTNS portal and typed word for word by HEMU Ram Kumar, additionally those words which were unclear.", "the railway bridge after about 50 steps at around 4:30 PM. When questioned, he identified himself as Sumit Kumar Saini, son of Rajesh Kumar, aged about 20 years, from Shahjahanpur district. In his right hand, we found a plastic container with a bottle inside containing a brownish substance. Upon opening the bottle, we smelled raw alcohol. The seized raw alcohol license was not shown when requested. The bottle was sealed on the spot and a sample seal was prepared. During the arrest, orders of the Supreme Court and Human Rights Commission were followed to treat the accused fairly. The seized goods and accused were taken into police custody. Due to the sudden arrest near the railway and public inconvenience, no witnesses were available. The FIR will be communicated to the accused's relatives upon arrival at the police station. The report was dictated and written by 02025 Rohit Kumar and signed by 02710 Gaurav Kumar at Sadar Bazaar Police Station, Shahjahanpur, on January 5, 2024. A copy"]
  to my life and my son's life from these three people. Therefore, I request you to grant justice. Petitioner Manju Yadav, husband Santosh Yadav, village Sikandarpur, post Dharmdaspur, Nawada district-Ajmer, mobile number-8423229252, date-16/02/2024 HMICM 4 N.C R.B (N.C.R.B). I.F.- (Unified Investigation Form-l) Register the case according to the rules. Signature illegible, Thanadhiksh Officer 16.2.2024	Where does the petitioner Manju Yadav reside?	['Mobile-9517003259, 83188106881 Note- I, Prabhat Kumar 3 NC RB (N.C.R.B) I.L.F.-1 (Unified Investigation Form -1) certify that the document was fed exactly as spoken by me on the computer.', 'given by the informant who then left. With the help of the Leopard team, the suspect was apprehended along with the cow at the edge of the highway near Matlupur Crossing. When questioned about his name and address, he stated his name as Kariman alias Ajay son of Kishor, residing in Ranee Thana Dakshinatola, district Maunath Bhanjan, approximately 27 years old. He was asked about the cow he was carrying, which was tied with a rope of about four feet, and he started to act suspiciously. Upon further questioning, he admitted that since vehicles carrying cows were often caught by the animal police, he had been transporting cows on foot up to Badhua Godam Border and then sending them to Bihar for slaughter. He said it was his misfortune today that he was caught by us. He claimed to be a poor man with no other means of livelihood and supported his family with this income. He requested forgiveness and promised to stop this work. The seized cow has the following characteristics: one healthy', "First Information Report: The complaint letter is written in Hindi and is final. In service, Mr. Thana Adhyaksh Gajipur Indiranagar Lucknow, it is informed that the applicant Shah Nawaz Ansari SIO Tahir Ansari is a resident of Amraai Village Indiranagar Lucknow. I came to read at Prime Plaza Library Munshi Pulia on today's date 06/03/2024 from 11:30 AM to 9:00 PM. My vehicle (Vehicle No.) UP 32 HY 7715, which was white in color, was found missing from the place after a thorough search, and no information was obtained. Therefore, it is requested that Mr. Ji kindly search for my vehicle. It would be a great favor. Date 06/03/2024 Thank you Applicant signature readable in English Shahnwaz Shah Nawaz Ansari SLO Tahir Ansari Address Amraai Village Indiranagar Lucknow Mobile No. 8853832907 Note: The above complaint was made by Vijay Kumar Tiwari with registration number 7498 and FIR was registered based on his statement by Shrikanth Tiwari (Additional technical error).", "information, I, the responsible inspector May Hamarahee, went to Shyam Singh's house near Thanha Village. I secretly verified the location and prepared a map before obtaining a search warrant from the Deputy Superintendent of Police Patiala Jan. Kasganj. After receiving the search warrant, I summoned SI Kalyan Singh and Kan. 07 Abhlaash Ch and Kan. 678 Kuldeep Kumar, gave necessary directions, and brought them to the Thanha Village site. I informed everyone about the incident and conducted a body search on the spot to ensure no related items were with us. Subsequently, I, the responsible inspector May Police Force, approached Shyam Singh's house and stood quietly near the door listening, hearing 99 pao being played. This confirmed to me that illegal gambling was taking place inside the house. Then, I entered Anil Gupta's house with May Hamrahee Force, and upon seeing us, the people inside started running around the room. At approximately 21:10, all were arrested using necessary force.", 'given to the accused. SD Order Kumar, SD No. 293 Kumar, SD Illegible SI 22/3/24 (Dhirendra Singh) Note: I, SD No. 594 Kapin Kumar, certify that the copy of the report was made word for word by me. SD No. 594 Kapin Kumar 4 N.C R.B (NCRB)

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 8
• learning_rate: 2e-05
• num_train_epochs: 50
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: True
• load_best_model_at_end: True
• optim: adamw_torch_fused
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 8
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 50
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: True
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	dim_768_cosine_ndcg@10	dim_512_cosine_ndcg@10	dim_256_cosine_ndcg@10	dim_128_cosine_ndcg@10	dim_64_cosine_ndcg@10
0.8421	10	167.2069	-	-	-	-	-
1.0	12	-	0.3228	0.3164	0.2842	0.2333	0.1615
1.6737	20	143.9168	-	-	-	-	-
2.0	24	-	0.3380	0.3376	0.3012	0.2403	0.1700
2.5053	30	112.1768	-	-	-	-	-
3.0	36	-	0.3567	0.3475	0.3115	0.2527	0.1898
3.3368	40	86.4427	-	-	-	-	-
4.0	48	-	0.3982	0.3888	0.3507	0.3100	0.2488
4.1684	50	69.2315	-	-	-	-	-
5.0	60	52.051	0.4031	0.3892	0.3601	0.3328	0.2754
5.8421	70	39.3198	-	-	-	-	-
6.0	72	-	0.4240	0.4080	0.3836	0.3636	0.3075
6.6737	80	31.2848	-	-	-	-	-
7.0	84	-	0.4445	0.4383	0.4135	0.3881	0.3332
7.5053	90	26.8355	-	-	-	-	-
8.0	96	-	0.4517	0.4400	0.4247	0.3947	0.3407
8.3368	100	22.4953	-	-	-	-	-
9.0	108	-	0.4595	0.4485	0.4335	0.4044	0.3553
9.1684	110	20.0884	-	-	-	-	-
10.0	120	16.9324	0.4637	0.4535	0.4372	0.4107	0.3577
10.8421	130	14.7747	-	-	-	-	-
11.0	132	-	0.4666	0.4562	0.4357	0.4112	0.3625
11.6737	140	12.672	-	-	-	-	-
12.0	144	-	0.4632	0.4519	0.4329	0.4073	0.3651
12.5053	150	11.6213	-	-	-	-	-
13.0	156	-	0.4660	0.4585	0.4374	0.4106	0.3701
13.3368	160	10.4717	-	-	-	-	-
14.0	168	-	0.4652	0.4528	0.4347	0.4106	0.3663
14.1684	170	9.0225	-	-	-	-	-
15.0	180	8.8606	0.4656	0.4593	0.4393	0.4150	0.3715
15.8421	190	7.6863	-	-	-	-	-
16.0	192	-	0.4636	0.4572	0.4382	0.4118	0.3705
16.6737	200	7.7795	-	-	-	-	-
17.0	204	-	0.4697	0.4590	0.4377	0.4131	0.3682
17.5053	210	6.532	-	-	-	-	-
18.0	216	-	0.4574	0.4515	0.4348	0.4079	0.3664
18.3368	220	6.0948	-	-	-	-	-
19.0	228	-	0.4632	0.4571	0.4342	0.4127	0.3714
19.1684	230	5.936	-	-	-	-	-
20.0	240	5.1483	0.4729	0.4618	0.4426	0.4126	0.3745
20.8421	250	5.2524	-	-	-	-	-
21.0	252	-	0.4623	0.4560	0.4366	0.4123	0.3674
21.6737	260	4.689	-	-	-	-	-
22.0	264	-	0.4668	0.4594	0.4364	0.4143	0.3745
22.5053	270	4.6265	-	-	-	-	-
23.0	276	-	0.4694	0.4636	0.4405	0.4160	0.3723
23.3368	280	4.3978	-	-	-	-	-
24.0	288	-	0.4659	0.4635	0.4399	0.4120	0.3718
24.1684	290	4.1499	-	-	-	-	-
25.0	300	3.8881	0.4702	0.4633	0.4400	0.4114	0.3751
25.8421	310	4.0321	-	-	-	-	-
26.0	312	-	0.4673	0.4603	0.4392	0.4142	0.3742
26.6737	320	3.4653	-	-	-	-	-
27.0	324	-	0.4670	0.4633	0.4412	0.4171	0.3745
27.5053	330	3.4544	-	-	-	-	-
28.0	336	-	0.4725	0.4607	0.4414	0.4168	0.3781
28.3368	340	3.2718	-	-	-	-	-
29.0	348	-	0.4648	0.4581	0.4385	0.4162	0.3756
29.1684	350	3.2159	-	-	-	-	-
30.0	360	3.0854	0.4673	0.4565	0.4416	0.4120	0.3740
30.8421	370	2.9308	-	-	-	-	-
31.0	372	-	0.4644	0.4541	0.4417	0.4126	0.3716
31.6737	380	2.8503	-	-	-	-	-
32.0	384	-	0.4686	0.4561	0.4408	0.4150	0.3731
32.5053	390	2.856	-	-	-	-	-
33.0	396	-	0.4729	0.4564	0.4385	0.4161	0.3733
33.3368	400	2.8083	-	-	-	-	-
34.0	408	-	0.4715	0.4558	0.4421	0.4149	0.3727
34.1684	410	2.7123	-	-	-	-	-
35.0	420	2.7888	0.4728	0.4587	0.4456	0.4154	0.3740
35.8421	430	2.6049	-	-	-	-	-
36.0	432	-	0.4727	0.4577	0.4449	0.4150	0.3728
36.6737	440	2.4939	-	-	-	-	-
37.0	444	-	0.4704	0.4560	0.4464	0.4145	0.3736
37.5053	450	2.3718	-	-	-	-	-
38.0	456	-	0.4701	0.4582	0.4474	0.4139	0.3758
38.3368	460	2.4518	-	-	-	-	-
39.0	468	-	0.4681	0.4577	0.4475	0.4158	0.3752
39.1684	470	2.2857	-	-	-	-	-
40.0	480	2.5938	0.4670	0.4573	0.4450	0.4152	0.3737
40.8421	490	2.4938	-	-	-	-	-
41.0	492	-	0.4684	0.4592	0.4459	0.4147	0.3737
41.6737	500	2.3226	-	-	-	-	-
42.0	504	-	0.4691	0.4574	0.4453	0.4156	0.3741
42.5053	510	2.2881	-	-	-	-	-
43.0	516	-	0.4690	0.4563	0.4446	0.4139	0.3744
43.3368	520	2.3356	-	-	-	-	-
44.0	528	-	0.4688	0.4582	0.4455	0.4138	0.3745
44.1684	530	2.4034	-	-	-	-	-
45.0	540	2.4925	0.4700	0.4566	0.4444	0.4133	0.3734
45.8421	550	2.4681	-	-	-	-	-
46.0	552	-	0.4694	0.4563	0.4447	0.4151	0.3751
46.6737	560	2.4209	-	-	-	-	-
47.0	564	-	0.4697	0.4594	0.4456	0.4150	0.3748
47.5053	570	2.2247	-	-	-	-	-
48.0	576	-	0.4711	0.4575	0.4458	0.4147	0.3748
48.3368	580	2.1489	-	-	-	-	-
49.0	588	-	0.4679	0.4578	0.4451	0.4149	0.3737
49.1684	590	2.1203	-	-	-	-	-
50.0	600	2.2774	0.4691	0.4569	0.4469	0.4149	0.3746

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 4.1.0
• Transformers: 4.52.2
• PyTorch: 2.6.0+cu124
• Accelerate: 1.7.0
• Datasets: 3.6.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}