--- language: - en license: apache-2.0 tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:1567 - loss:MatryoshkaLoss - loss:MultipleNegativesRankingLoss base_model: nomic-ai/modernbert-embed-base widget: - source_sentence: How many authors are listed for the trial? sentences: - 'chemotherapy and bone marrow transplantation for certain malignancies and has a long track record of safe use in adults and children. The incidence of adverse events such as fever, chills, bone pain, dyspnea, tachycardia, and hemodynamic instability was no different between GM- CSF and placebo-treated groups in controlled adult BMT studies. Rapid IV administration of' - 'clinical ICU staff in accordance with institutional practice and judgment. Child Assent Subjects who are eligible for this study will be critically ill, and child assent is typically not possible at the time of study enrollment. However, during follow up after discharge from the ICU, issues about assent become applicable. Children who are capable of giving assent' - 'Controlled Phase 2 Trial. Stroke, 49(5):1210–1216, 2018. [76] M. K. R. Somagutta, M. K. Lourdes Pormento, P. Hamid, A. Hamdan, M. A. Khan, R. Desir, R. Vijayan, S. Shirke, R. Jeyakumar, Z. Dogar, S. S. Makkar, P. Guntipalli, N. N. Ngardig, M. S. Nagineni, T. Paul, E. Luvsannyam, C. Riddick, and M. A. Sanchez-' - source_sentence: What type of event can lead to the suspension of enrollment in the study? sentences: - 'and data generated by this study must be available for inspection upon request by representatives (when applicable) of the Food and Drug Administration (FDA), NIH, other Federal funders or study sponsors, and the Institutional Review Board (IRB) for each study site. 9 Protection of Human Subjects 9.1 Risks to Human Subjects 9.1.1 Human Subjects Involvement and Characteristics' - 'two consecutive days while receiving study drug, the drug will be discontinued. Adverse events will be monitored as described in Section 10.2.6 on page 61. The medical monitor has the authority to suspend enrollment in the event of an unexpected, study-related serious adverse event that is judged to change the risk/benefit of subject participation.' - 'innate immune system is common and measurable in pediatric sepsis. Innate immune cells such as monocytes and neutrophils serve critical functions including migration to sites of infection, phagocytosis of pathogens, promotion of microbial killing, antigen presentation, and production of immunomodulatory cytokines. We have repeatedly shown that severe reduction in the ability' - source_sentence: When will the reviews start? sentences: - 'mg/kg/day given for three days by continuous infusion was used.23, 63 Despite its apparent safety in adults, this dose is substantially higher than what has been used in children with HLH/MAS or adults with COVID-19. In the largest (to date) published study of anakinra in hospitalized, hyper-inflamed adults with COVID-19 (N=392), a dose of 10 mg/kg/day IV divided every 12 hours (infused over 1' - 'data are required for Federal reporting purposes to delineate subject accrual by race, ethnicity, and gender. For purposes of the DCC handling potential protected health information (PHI) and pro- ducing the de–identified research data sets that will be used for analyses, all study sites have been offered a Business Associate Agreement with the University of Utah. Copies of executed' - 'empirically whether these patients differ from those remaining in the study for the scheduled treatment and follow-up time. Missingness for primary, secondary, exploratory, and safety outcomes will be reviewed in aggregate and by site. Reviews will start as soon as enrollment opens and will be regulatory monitored so missing data problems can be addressed early in the study.' - source_sentence: What type of results will be communicated to the Data Coordinating Center and clinical site investigator? sentences: - 'ing of a medical condition that was present at the time of randomization will be considered a new adverse event and reported. After patient randomization all adverse events (including serious adverse events) will be recorded according to relatedness, severity, and expectedness, as well as their duration and' - '12.2 Health Insurance Portability and Accountability Act Data elements collected include the date of birth and date of admission. Prior to statistical analyses, dates will be used to calculate patient age at the time of the study events. Data elements for race, ethnicity, and gender are also being collected. These demographic' - 'The Collaborative Pediatric Critical Care Research NetworkPage 34 of 76 Protocol 90 (Hall, Zuppa and Mourani) 4.5 Randomization Upon determination of a subject’s immunophenotype, Dr. Hall or his designee will notify the Data Coordinating Center and the clinical site investigator of the laboratory results. Subjects' - source_sentence: What age groups will be enrolled in the study? sentences: - 'have mild to moderate inflammation (i.e. a serum ferritin level <2,000 ng/ml) from the TRIPS trial. Those subjects will be instead entered into a completely distinct clinical trial of immune stimulation with GM-CSF (GRACE-2) that is covered by a separate IND (#112277). PRECISE Protocol Version 1.07 Protocol Version Date: June 16, 2023' - 'Subject Population to be Studied Participating sites will enroll infants, children and adoles- cent patients who are admitted to a Pediatric or Cardiac Intensive Care Unit with sepsis-induced multiple organ dysfunction syndrome (MODS). The goal is to determine if personalized im- munomodulation is an effective strategy to reduce mortality and morbidity from sepsis-induced' - 'Loosdregt, N. M. Wulffraat, S. de Roock, and S. J. Vastert. Treatment to target using recombinant interleukin-1 receptor antagonist as first-line monotherapy in new-onset systemic juvenile idiopathic arthritis: Results from a five-year follow-up study. Arthritis Rheumatol, 71(7):1163–1173, 2019. [78] R. K. Thakkar, R. Devine, J. Popelka, J. Hensley, R. Fabia, J. A. Muszynski, and M. W.' pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 model-index: - name: Fine-tuned with [QuicKB](https://github.com/ALucek/QuicKB) results: - task: type: information-retrieval name: Information Retrieval dataset: name: dim 768 type: dim_768 metrics: - type: cosine_accuracy@1 value: 0.5714285714285714 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.7828571428571428 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.8114285714285714 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.8742857142857143 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.5714285714285714 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.2609523809523809 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.16228571428571423 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.08742857142857141 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.5714285714285714 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.7828571428571428 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.8114285714285714 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.8742857142857143 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.7304617900805063 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.6836485260770975 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.6898282619821292 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 512 type: dim_512 metrics: - type: cosine_accuracy@1 value: 0.5485714285714286 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.7885714285714286 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.8285714285714286 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.8685714285714285 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.5485714285714286 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.2628571428571428 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.16571428571428568 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.08685714285714283 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.5485714285714286 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.7885714285714286 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.8285714285714286 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.8685714285714285 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.7172419802927883 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.6675759637188208 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.6741729815259775 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 256 type: dim_256 metrics: - type: cosine_accuracy@1 value: 0.5485714285714286 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.76 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.84 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.9085714285714286 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.5485714285714286 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.2533333333333333 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.16799999999999995 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09085714285714283 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.5485714285714286 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.76 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.84 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.9085714285714286 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.7268936400245406 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.6687596371882085 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.6719911574054431 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 128 type: dim_128 metrics: - type: cosine_accuracy@1 value: 0.49142857142857144 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.7028571428571428 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.7885714285714286 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.8685714285714285 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.49142857142857144 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.23428571428571424 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.15771428571428567 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.08685714285714284 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.49142857142857144 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.7028571428571428 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.7885714285714286 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.8685714285714285 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.6778419592624233 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.6168730158730158 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.6219971103464577 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 64 type: dim_64 metrics: - type: cosine_accuracy@1 value: 0.38285714285714284 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.5714285714285714 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.6571428571428571 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.7885714285714286 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.38285714285714284 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.19047619047619044 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.1314285714285714 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.07885714285714283 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.38285714285714284 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.5714285714285714 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.6571428571428571 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.7885714285714286 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.5697625172066919 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.5015079365079367 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.5090522718083348 name: Cosine Map@100 --- # Fine-tuned with [QuicKB](https://github.com/ALucek/QuicKB) This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [nomic-ai/modernbert-embed-base](https://huggingface.co/nomic-ai/modernbert-embed-base). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [nomic-ai/modernbert-embed-base](https://huggingface.co/nomic-ai/modernbert-embed-base) - **Maximum Sequence Length:** 1024 tokens - **Output Dimensionality:** 768 dimensions - **Similarity Function:** Cosine Similarity - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False}) with Transformer model: ModernBertModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("Mdean77/modernbert-embed-quickb") # Run inference sentences = [ 'What age groups will be enrolled in the study?', 'Subject Population to be Studied Participating sites will enroll infants, children and adoles-\ncent patients who are admitted to a Pediatric or Cardiac Intensive Care Unit with sepsis-induced\nmultiple organ dysfunction syndrome (MODS). The goal is to determine if personalized im-\nmunomodulation is an effective strategy to reduce mortality and morbidity from sepsis-induced', 'have mild to moderate inflammation (i.e. a serum ferritin level <2,000 ng/ml) from the TRIPS\ntrial. Those subjects will be instead entered into a completely distinct clinical trial of immune\nstimulation with GM-CSF (GRACE-2) that is covered by a separate IND (#112277).\nPRECISE Protocol Version 1.07\nProtocol Version Date: June 16, 2023', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Datasets: `dim_768`, `dim_512`, `dim_256`, `dim_128` and `dim_64` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | dim_768 | dim_512 | dim_256 | dim_128 | dim_64 | |:--------------------|:-----------|:-----------|:-----------|:-----------|:-----------| | cosine_accuracy@1 | 0.5714 | 0.5486 | 0.5486 | 0.4914 | 0.3829 | | cosine_accuracy@3 | 0.7829 | 0.7886 | 0.76 | 0.7029 | 0.5714 | | cosine_accuracy@5 | 0.8114 | 0.8286 | 0.84 | 0.7886 | 0.6571 | | cosine_accuracy@10 | 0.8743 | 0.8686 | 0.9086 | 0.8686 | 0.7886 | | cosine_precision@1 | 0.5714 | 0.5486 | 0.5486 | 0.4914 | 0.3829 | | cosine_precision@3 | 0.261 | 0.2629 | 0.2533 | 0.2343 | 0.1905 | | cosine_precision@5 | 0.1623 | 0.1657 | 0.168 | 0.1577 | 0.1314 | | cosine_precision@10 | 0.0874 | 0.0869 | 0.0909 | 0.0869 | 0.0789 | | cosine_recall@1 | 0.5714 | 0.5486 | 0.5486 | 0.4914 | 0.3829 | | cosine_recall@3 | 0.7829 | 0.7886 | 0.76 | 0.7029 | 0.5714 | | cosine_recall@5 | 0.8114 | 0.8286 | 0.84 | 0.7886 | 0.6571 | | cosine_recall@10 | 0.8743 | 0.8686 | 0.9086 | 0.8686 | 0.7886 | | **cosine_ndcg@10** | **0.7305** | **0.7172** | **0.7269** | **0.6778** | **0.5698** | | cosine_mrr@10 | 0.6836 | 0.6676 | 0.6688 | 0.6169 | 0.5015 | | cosine_map@100 | 0.6898 | 0.6742 | 0.672 | 0.622 | 0.5091 | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 1,567 training samples * Columns: anchor and positive * Approximate statistics based on the first 1000 samples: | | anchor | positive | |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | anchor | positive | |:-----------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | How many terabytes of data are referenced? | over 125 terabytes of data.
Information systems are available 24/7/365 unless a scheduled maintenance period or
mitigation of an unexpected event is required. Critical systems availability has exceeded 99.9%
for the past 5 years.
7.2.3 Security, Support, Encryption, and Confidentiality
The data center coordinates the network infrastructure and security with University Information | | What regulation allows single parent permission for the study? | for their child in the study. Single parent permission is permitted under 45 CFR §46.405. The
parent or legal guardian will be informed about the objectives of the study and the potential
risks and benefits of their child’s participation. If the parent or legal guardian refuses permission
for their child to participate, then all clinical management will continue to be provided by the | | What is included in the follow-up plan for non-compliant sites? | planned site visits, criteria for focused visits, additional visits or remote monitoring, a plan for
chart review and a follow up plan for non-compliant sites. The monitoring plan also describes
the type of monitoring that will take place (e.g., sample of all subjects within a site; key data or
all data), the schedule of visits, how they are reported and a time frame to resolve any issues
found. | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 768, 512, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: epoch - `per_device_train_batch_size`: 16 - `gradient_accumulation_steps`: 16 - `learning_rate`: 2e-05 - `num_train_epochs`: 4 - `lr_scheduler_type`: cosine - `warmup_ratio`: 0.1 - `tf32`: False - `load_best_model_at_end`: True - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: epoch - `prediction_loss_only`: True - `per_device_train_batch_size`: 16 - `per_device_eval_batch_size`: 8 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 16 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 4 - `max_steps`: -1 - `lr_scheduler_type`: cosine - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: False - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: None - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `eval_use_gather_object`: False - `average_tokens_across_devices`: False - `prompts`: None - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | Training Loss | dim_768_cosine_ndcg@10 | dim_512_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 | |:----------:|:------:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:| | 1.0 | 7 | - | 0.6698 | 0.6606 | 0.6458 | 0.6146 | 0.5049 | | 1.4898 | 10 | 55.7211 | - | - | - | - | - | | 2.0 | 14 | - | 0.7210 | 0.7080 | 0.7183 | 0.6653 | 0.5621 | | 2.9796 | 20 | 26.9161 | - | - | - | - | - | | 3.0 | 21 | - | 0.7309 | 0.7172 | 0.7262 | 0.6762 | 0.5694 | | **3.4898** | **24** | **-** | **0.7305** | **0.7172** | **0.7269** | **0.6778** | **0.5698** | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.12.3 - Sentence Transformers: 3.4.1 - Transformers: 4.49.0 - PyTorch: 2.6.0 - Accelerate: 1.4.0 - Datasets: 3.3.2 - Tokenizers: 0.21.0 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```