README.md

---
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:1567
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
base_model: nomic-ai/modernbert-embed-base
widget:
- source_sentence: How many authors are listed for the trial?
  sentences:
  - 'chemotherapy and bone marrow transplantation for certain malignancies and has
    a long track

    record of safe use in adults and children. The incidence of adverse events such
    as fever, chills,

    bone pain, dyspnea, tachycardia, and hemodynamic instability was no different
    between GM-

    CSF and placebo-treated groups in controlled adult BMT studies. Rapid IV administration
    of'
  - 'clinical ICU staff in accordance with institutional practice and judgment.

    Child Assent Subjects who are eligible for this study will be critically ill,
    and child assent is

    typically not possible at the time of study enrollment. However, during follow
    up after discharge

    from the ICU, issues about assent become applicable. Children who are capable
    of giving assent'
  - 'Controlled Phase 2 Trial. Stroke, 49(5):1210–1216, 2018.

    [76] M. K. R. Somagutta, M. K. Lourdes Pormento, P. Hamid, A. Hamdan, M. A. Khan,

    R. Desir, R. Vijayan, S. Shirke, R. Jeyakumar, Z. Dogar, S. S. Makkar, P. Guntipalli,

    N. N. Ngardig, M. S. Nagineni, T. Paul, E. Luvsannyam, C. Riddick, and M. A. Sanchez-'
- source_sentence: What type of event can lead to the suspension of enrollment in
    the study?
  sentences:
  - 'and data generated by this study must be available for inspection upon request
    by representatives

    (when applicable) of the Food and Drug Administration (FDA), NIH, other Federal
    funders or

    study sponsors, and the Institutional Review Board (IRB) for each study site.

    9 Protection of Human Subjects

    9.1 Risks to Human Subjects

    9.1.1 Human Subjects Involvement and Characteristics'
  - 'two consecutive days while receiving study drug, the drug will be discontinued.

    Adverse events will be monitored as described in Section 10.2.6 on page 61. The
    medical

    monitor has the authority to suspend enrollment in the event of an unexpected,
    study-related

    serious adverse event that is judged to change the risk/benefit of subject participation.'
  - 'innate immune system is common and measurable in pediatric sepsis. Innate immune
    cells such

    as monocytes and neutrophils serve critical functions including migration to sites
    of infection,

    phagocytosis of pathogens, promotion of microbial killing, antigen presentation,
    and production

    of immunomodulatory cytokines. We have repeatedly shown that severe reduction
    in the ability'
- source_sentence: When will the reviews start?
  sentences:
  - 'mg/kg/day given for three days by continuous infusion was used.23, 63 Despite
    its apparent safety

    in adults, this dose is substantially higher than what has been used in children
    with HLH/MAS

    or adults with COVID-19.

    In the largest (to date) published study of anakinra in hospitalized, hyper-inflamed
    adults

    with COVID-19 (N=392), a dose of 10 mg/kg/day IV divided every 12 hours (infused
    over 1'
  - 'data are required for Federal reporting purposes to delineate subject accrual
    by race, ethnicity,

    and gender.

    For purposes of the DCC handling potential protected health information (PHI)
    and pro-

    ducing the de–identified research data sets that will be used for analyses, all
    study sites have

    been offered a Business Associate Agreement with the University of Utah. Copies
    of executed'
  - 'empirically whether these patients differ from those remaining in the study for
    the scheduled

    treatment and follow-up time. Missingness for primary, secondary, exploratory,
    and safety

    outcomes will be reviewed in aggregate and by site. Reviews will start as soon
    as enrollment

    opens and will be regulatory monitored so missing data problems can be addressed
    early in the

    study.'
- source_sentence: What type of results will be communicated to the Data Coordinating
    Center and clinical site investigator?
  sentences:
  - 'ing of a medical condition that was present at the time of randomization will
    be considered a

    new adverse event and reported.

    After patient randomization all adverse events (including serious adverse events)
    will be

    recorded according to relatedness, severity, and expectedness, as well as their
    duration and'
  - '12.2 Health Insurance Portability and Accountability Act

    Data elements collected include the date of birth and date of admission. Prior
    to statistical

    analyses, dates will be used to calculate patient age at the time of the study
    events.

    Data elements for race, ethnicity, and gender are also being collected. These
    demographic'
  - 'The Collaborative Pediatric Critical Care Research NetworkPage 34 of 76 Protocol
    90 (Hall, Zuppa and Mourani)

    4.5 Randomization

    Upon determination of a subject’s immunophenotype, Dr. Hall or his designee will
    notify the

    Data Coordinating Center and the clinical site investigator of the laboratory
    results. Subjects'
- source_sentence: What age groups will be enrolled in the study?
  sentences:
  - 'have mild to moderate inflammation (i.e. a serum ferritin level <2,000 ng/ml)
    from the TRIPS

    trial. Those subjects will be instead entered into a completely distinct clinical
    trial of immune

    stimulation with GM-CSF (GRACE-2) that is covered by a separate IND (#112277).

    PRECISE Protocol Version 1.07

    Protocol Version Date: June 16, 2023'
  - 'Subject Population to be Studied Participating sites will enroll infants, children
    and adoles-

    cent patients who are admitted to a Pediatric or Cardiac Intensive Care Unit with
    sepsis-induced

    multiple organ dysfunction syndrome (MODS). The goal is to determine if personalized
    im-

    munomodulation is an effective strategy to reduce mortality and morbidity from
    sepsis-induced'
  - 'Loosdregt, N. M. Wulffraat, S. de Roock, and S. J. Vastert. Treatment to target
    using

    recombinant interleukin-1 receptor antagonist as first-line monotherapy in new-onset

    systemic juvenile idiopathic arthritis: Results from a five-year follow-up study.
    Arthritis

    Rheumatol, 71(7):1163–1173, 2019.

    [78] R. K. Thakkar, R. Devine, J. Popelka, J. Hensley, R. Fabia, J. A. Muszynski,
    and M. W.'
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
model-index:
- name: Fine-tuned with [QuicKB](https://github.com/ALucek/QuicKB)
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 768
      type: dim_768
    metrics:
    - type: cosine_accuracy@1
      value: 0.5714285714285714
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.7828571428571428
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8114285714285714
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.8742857142857143
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.5714285714285714
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2609523809523809
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.16228571428571423
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.08742857142857141
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.5714285714285714
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.7828571428571428
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8114285714285714
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.8742857142857143
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.7304617900805063
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.6836485260770975
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.6898282619821292
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 512
      type: dim_512
    metrics:
    - type: cosine_accuracy@1
      value: 0.5485714285714286
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.7885714285714286
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8285714285714286
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.8685714285714285
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.5485714285714286
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2628571428571428
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.16571428571428568
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.08685714285714283
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.5485714285714286
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.7885714285714286
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8285714285714286
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.8685714285714285
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.7172419802927883
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.6675759637188208
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.6741729815259775
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 256
      type: dim_256
    metrics:
    - type: cosine_accuracy@1
      value: 0.5485714285714286
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.76
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.84
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9085714285714286
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.5485714285714286
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2533333333333333
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.16799999999999995
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09085714285714283
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.5485714285714286
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.76
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.84
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9085714285714286
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.7268936400245406
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.6687596371882085
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.6719911574054431
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 128
      type: dim_128
    metrics:
    - type: cosine_accuracy@1
      value: 0.49142857142857144
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.7028571428571428
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.7885714285714286
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.8685714285714285
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.49142857142857144
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.23428571428571424
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.15771428571428567
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.08685714285714284
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.49142857142857144
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.7028571428571428
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.7885714285714286
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.8685714285714285
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.6778419592624233
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.6168730158730158
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.6219971103464577
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 64
      type: dim_64
    metrics:
    - type: cosine_accuracy@1
      value: 0.38285714285714284
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.5714285714285714
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.6571428571428571
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.7885714285714286
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.38285714285714284
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.19047619047619044
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.1314285714285714
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.07885714285714283
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.38285714285714284
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.5714285714285714
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.6571428571428571
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.7885714285714286
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.5697625172066919
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.5015079365079367
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.5090522718083348
      name: Cosine Map@100
---

# Fine-tuned with [QuicKB](https://github.com/ALucek/QuicKB)

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [nomic-ai/modernbert-embed-base](https://huggingface.co/nomic-ai/modernbert-embed-base). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [nomic-ai/modernbert-embed-base](https://huggingface.co/nomic-ai/modernbert-embed-base) <!-- at revision d556a88e332558790b210f7bdbe87da2fa94a8d8 -->
- **Maximum Sequence Length:** 1024 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
- **Language:** en
- **License:** apache-2.0

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False}) with Transformer model: ModernBertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Mdean77/modernbert-embed-quickb")
# Run inference
sentences = [
    'What age groups will be enrolled in the study?',
    'Subject Population to be Studied Participating sites will enroll infants, children and adoles-\ncent patients who are admitted to a Pediatric or Cardiac Intensive Care Unit with sepsis-induced\nmultiple organ dysfunction syndrome (MODS). The goal is to determine if personalized im-\nmunomodulation is an effective strategy to reduce mortality and morbidity from sepsis-induced',
    'have mild to moderate inflammation (i.e. a serum ferritin level <2,000 ng/ml) from the TRIPS\ntrial. Those subjects will be instead entered into a completely distinct clinical trial of immune\nstimulation with GM-CSF (GRACE-2) that is covered by a separate IND (#112277).\nPRECISE Protocol Version 1.07\nProtocol Version Date: June 16, 2023',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval

* Datasets: `dim_768`, `dim_512`, `dim_256`, `dim_128` and `dim_64`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | dim_768    | dim_512    | dim_256    | dim_128    | dim_64     |
|:--------------------|:-----------|:-----------|:-----------|:-----------|:-----------|
| cosine_accuracy@1   | 0.5714     | 0.5486     | 0.5486     | 0.4914     | 0.3829     |
| cosine_accuracy@3   | 0.7829     | 0.7886     | 0.76       | 0.7029     | 0.5714     |
| cosine_accuracy@5   | 0.8114     | 0.8286     | 0.84       | 0.7886     | 0.6571     |
| cosine_accuracy@10  | 0.8743     | 0.8686     | 0.9086     | 0.8686     | 0.7886     |
| cosine_precision@1  | 0.5714     | 0.5486     | 0.5486     | 0.4914     | 0.3829     |
| cosine_precision@3  | 0.261      | 0.2629     | 0.2533     | 0.2343     | 0.1905     |
| cosine_precision@5  | 0.1623     | 0.1657     | 0.168      | 0.1577     | 0.1314     |
| cosine_precision@10 | 0.0874     | 0.0869     | 0.0909     | 0.0869     | 0.0789     |
| cosine_recall@1     | 0.5714     | 0.5486     | 0.5486     | 0.4914     | 0.3829     |
| cosine_recall@3     | 0.7829     | 0.7886     | 0.76       | 0.7029     | 0.5714     |
| cosine_recall@5     | 0.8114     | 0.8286     | 0.84       | 0.7886     | 0.6571     |
| cosine_recall@10    | 0.8743     | 0.8686     | 0.9086     | 0.8686     | 0.7886     |
| **cosine_ndcg@10**  | **0.7305** | **0.7172** | **0.7269** | **0.6778** | **0.5698** |
| cosine_mrr@10       | 0.6836     | 0.6676     | 0.6688     | 0.6169     | 0.5015     |
| cosine_map@100      | 0.6898     | 0.6742     | 0.672      | 0.622      | 0.5091     |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset

* Size: 1,567 training samples
* Columns: <code>anchor</code> and <code>positive</code>
* Approximate statistics based on the first 1000 samples:
  |         | anchor                                                                            | positive                                                                            |
  |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
  | type    | string                                                                            | string                                                                              |
  | details | <ul><li>min: 8 tokens</li><li>mean: 15.03 tokens</li><li>max: 30 tokens</li></ul> | <ul><li>min: 57 tokens</li><li>mean: 90.85 tokens</li><li>max: 185 tokens</li></ul> |
* Samples:
  | anchor                                                                       | positive                                                                                                                                                                                                                                                                                                                                                                                                                                |
  |:-----------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>How many terabytes of data are referenced?</code>                      | <code>over 125 terabytes of data.<br>Information systems are available 24/7/365 unless a scheduled maintenance period or<br>mitigation of an unexpected event is required. Critical systems availability has exceeded 99.9%<br>for the past 5 years.<br>7.2.3 Security, Support, Encryption, and Confidentiality<br>The data center coordinates the network infrastructure and security with University Information</code>              |
  | <code>What regulation allows single parent permission for the study?</code>  | <code>for their child in the study. Single parent permission is permitted under 45 CFR §46.405. The<br>parent or legal guardian will be informed about the objectives of the study and the potential<br>risks and benefits of their child’s participation. If the parent or legal guardian refuses permission<br>for their child to participate, then all clinical management will continue to be provided by the</code>                |
  | <code>What is included in the follow-up plan for non-compliant sites?</code> | <code>planned site visits, criteria for focused visits, additional visits or remote monitoring, a plan for<br>chart review and a follow up plan for non-compliant sites. The monitoring plan also describes<br>the type of monitoring that will take place (e.g., sample of all subjects within a site; key data or<br>all data), the schedule of visits, how they are reported and a time frame to resolve any issues<br>found.</code> |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: epoch
- `per_device_train_batch_size`: 16
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 4
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `tf32`: False
- `load_best_model_at_end`: True
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 8
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 4
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: False
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch      | Step   | Training Loss | dim_768_cosine_ndcg@10 | dim_512_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 |
|:----------:|:------:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|
| 1.0        | 7      | -             | 0.6698                 | 0.6606                 | 0.6458                 | 0.6146                 | 0.5049                |
| 1.4898     | 10     | 55.7211       | -                      | -                      | -                      | -                      | -                     |
| 2.0        | 14     | -             | 0.7210                 | 0.7080                 | 0.7183                 | 0.6653                 | 0.5621                |
| 2.9796     | 20     | 26.9161       | -                      | -                      | -                      | -                      | -                     |
| 3.0        | 21     | -             | 0.7309                 | 0.7172                 | 0.7262                 | 0.6762                 | 0.5694                |
| **3.4898** | **24** | **-**         | **0.7305**             | **0.7172**             | **0.7269**             | **0.6778**             | **0.5698**            |

* The bold row denotes the saved checkpoint.

### Framework Versions
- Python: 3.12.3
- Sentence Transformers: 3.4.1
- Transformers: 4.49.0
- PyTorch: 2.6.0
- Accelerate: 1.4.0
- Datasets: 3.3.2
- Tokenizers: 0.21.0

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->