README.md

---
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:10356
- loss:MultipleNegativesRankingLoss
base_model: intfloat/multilingual-e5-large
widget:
- source_sentence: Horn band legwearis a type oflegwear, oftenthighhighs, with ahornedcharacter
    design along the upper band.
  sentences:
  - horn band legwear
  - head out of frame
  - sweatpants
- source_sentence: When a character is looping the laces of theiruntied shoelacesinto
    a sturdy bow.
  sentences:
  - hair tie
  - tying footwear
  - loose necktie
- source_sentence: Use this tag if the person's eyewear isremovedfrom their usual
    place and carried in the hands. If it still rests on the bridge of the nose or
    head, seeadjusting eyewearand its related tags.
  sentences:
  - cow costume
  - sarong
  - holding removed eyewear
- source_sentence: When both of a character's hands are on another character'sthighs.
  sentences:
  - baking
  - triplets
  - hands on another's thighs
- source_sentence: A long appendage protruding from the lower back. Often covered
    in fur or scales. A common feature of animal girls.
  sentences:
  - tail
  - grey-framed eyewear
  - stomach day
datasets:
- meandyou200175/word_embedding
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@2
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_accuracy@100
- cosine_precision@1
- cosine_precision@2
- cosine_precision@5
- cosine_precision@10
- cosine_precision@100
- cosine_recall@1
- cosine_recall@2
- cosine_recall@5
- cosine_recall@10
- cosine_recall@100
- cosine_ndcg@10
- cosine_mrr@1
- cosine_mrr@2
- cosine_mrr@5
- cosine_mrr@10
- cosine_mrr@100
- cosine_map@100
model-index:
- name: SentenceTransformer based on intfloat/multilingual-e5-large
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: Unknown
      type: unknown
    metrics:
    - type: cosine_accuracy@1
      value: 0.9073359073359073
      name: Cosine Accuracy@1
    - type: cosine_accuracy@2
      value: 0.9739382239382239
      name: Cosine Accuracy@2
    - type: cosine_accuracy@5
      value: 0.9942084942084942
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.999034749034749
      name: Cosine Accuracy@10
    - type: cosine_accuracy@100
      value: 1.0
      name: Cosine Accuracy@100
    - type: cosine_precision@1
      value: 0.9073359073359073
      name: Cosine Precision@1
    - type: cosine_precision@2
      value: 0.48696911196911197
      name: Cosine Precision@2
    - type: cosine_precision@5
      value: 0.19884169884169883
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.0999034749034749
      name: Cosine Precision@10
    - type: cosine_precision@100
      value: 0.010000000000000002
      name: Cosine Precision@100
    - type: cosine_recall@1
      value: 0.9073359073359073
      name: Cosine Recall@1
    - type: cosine_recall@2
      value: 0.9739382239382239
      name: Cosine Recall@2
    - type: cosine_recall@5
      value: 0.9942084942084942
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.999034749034749
      name: Cosine Recall@10
    - type: cosine_recall@100
      value: 1.0
      name: Cosine Recall@100
    - type: cosine_ndcg@10
      value: 0.9601842774877813
      name: Cosine Ndcg@10
    - type: cosine_mrr@1
      value: 0.9073359073359073
      name: Cosine Mrr@1
    - type: cosine_mrr@2
      value: 0.9406370656370656
      name: Cosine Mrr@2
    - type: cosine_mrr@5
      value: 0.9462837837837839
      name: Cosine Mrr@5
    - type: cosine_mrr@10
      value: 0.946988570202856
      name: Cosine Mrr@10
    - type: cosine_mrr@100
      value: 0.9470763202906061
      name: Cosine Mrr@100
    - type: cosine_map@100
      value: 0.9470763202906061
      name: Cosine Map@100
---

# SentenceTransformer based on intfloat/multilingual-e5-large

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [intfloat/multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [intfloat/multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) <!-- at revision 0dc5580a448e4284468b8909bae50fa925907bc5 -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 1024 dimensions
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("meandyou200175/e5_large_finetune_word")
# Run inference
sentences = [
    'A long appendage protruding from the lower back. Often covered in fur or scales. A common feature of animal girls.',
    'tail',
    'stomach day',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval

* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric               | Value      |
|:---------------------|:-----------|
| cosine_accuracy@1    | 0.9073     |
| cosine_accuracy@2    | 0.9739     |
| cosine_accuracy@5    | 0.9942     |
| cosine_accuracy@10   | 0.999      |
| cosine_accuracy@100  | 1.0        |
| cosine_precision@1   | 0.9073     |
| cosine_precision@2   | 0.487      |
| cosine_precision@5   | 0.1988     |
| cosine_precision@10  | 0.0999     |
| cosine_precision@100 | 0.01       |
| cosine_recall@1      | 0.9073     |
| cosine_recall@2      | 0.9739     |
| cosine_recall@5      | 0.9942     |
| cosine_recall@10     | 0.999      |
| cosine_recall@100    | 1.0        |
| **cosine_ndcg@10**   | **0.9602** |
| cosine_mrr@1         | 0.9073     |
| cosine_mrr@2         | 0.9406     |
| cosine_mrr@5         | 0.9463     |
| cosine_mrr@10        | 0.947      |
| cosine_mrr@100       | 0.9471     |
| cosine_map@100       | 0.9471     |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset

* Size: 10,356 training samples
* Columns: <code>query</code> and <code>positive</code>
* Approximate statistics based on the first 1000 samples:
  |         | query                                                                              | positive                                                                        |
  |:--------|:-----------------------------------------------------------------------------------|:--------------------------------------------------------------------------------|
  | type    | string                                                                             | string                                                                          |
  | details | <ul><li>min: 3 tokens</li><li>mean: 36.54 tokens</li><li>max: 177 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 5.3 tokens</li><li>max: 13 tokens</li></ul> |
* Samples:
  | query                                                                                                                                           | positive                           |
  |:------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------|
  | <code>Eyewear shaped like a semicircle.</code>                                                                                                  | <code>semi-circular eyewear</code> |
  | <code>A handheld electric appliance used fordryingand styling hair.</code>                                                                      | <code>hair dryer</code>            |
  | <code>When onebreastis exposed while the other remains covered or confined by clothing. Seebreasts outfor when both breasts are exposed.</code> | <code>one breast out</code>        |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  ```

### Evaluation Dataset

#### word_embedding

* Dataset: [word_embedding](https://huggingface.co/datasets/meandyou200175/word_embedding) at [af76b11](https://huggingface.co/datasets/meandyou200175/word_embedding/tree/af76b11c1d93542ca76e864a60b1744d5e02b099)
* Size: 1,036 evaluation samples
* Columns: <code>query</code> and <code>positive</code>
* Approximate statistics based on the first 1000 samples:
  |         | query                                                                              | positive                                                                         |
  |:--------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
  | type    | string                                                                             | string                                                                           |
  | details | <ul><li>min: 4 tokens</li><li>mean: 35.89 tokens</li><li>max: 164 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 5.38 tokens</li><li>max: 14 tokens</li></ul> |
* Samples:
  | query                                                                                                                                                                                                                                                                                         | positive                  |
  |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------|
  | <code>A machine that manipulates data according to a list of instructions. The ability to store and execute lists of instructions called programs make computers extremely versatile. On Danbooru's images they are most often used fordrawing,playing gamesand accessing theinternet.</code> | <code>computer</code>     |
  | <code>Aplaying cardwith twoclubs.</code>                                                                                                                                                                                                                                                      | <code>two of clubs</code> |
  | <code>Yebisu (ヱビス, Ebisu) is a beer produced bySapporo Breweries. It is one of Japan's oldest brands, first being brewed in Tokyo in 1890 by the Japan Beer Brewery Company.</code>                                                                                                           | <code>yebisu</code>       |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: steps
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 5
- `warmup_ratio`: 0.1
- `fp16`: True
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 5
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: True
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `tp_size`: 0
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch  | Step | Training Loss | Validation Loss | cosine_ndcg@10 |
|:------:|:----:|:-------------:|:---------------:|:--------------:|
| -1     | -1   | -             | -               | 0.7166         |
| 0.1543 | 100  | 0.9191        | -               | -              |
| 0.3086 | 200  | 0.1876        | -               | -              |
| 0.4630 | 300  | 0.1547        | -               | -              |
| 0.6173 | 400  | 0.1556        | -               | -              |
| 0.7716 | 500  | 0.179         | -               | -              |
| 0.9259 | 600  | 0.1234        | -               | -              |
| 1.0802 | 700  | 0.087         | -               | -              |
| 1.2346 | 800  | 0.0576        | -               | -              |
| 1.3889 | 900  | 0.0564        | -               | -              |
| 1.5432 | 1000 | 0.0583        | 0.0271          | 0.9198         |
| 1.6975 | 1100 | 0.0764        | -               | -              |
| 1.8519 | 1200 | 0.0493        | -               | -              |
| 2.0062 | 1300 | 0.0481        | -               | -              |
| 2.1605 | 1400 | 0.0222        | -               | -              |
| 2.3148 | 1500 | 0.0234        | -               | -              |
| 2.4691 | 1600 | 0.0283        | -               | -              |
| 2.6235 | 1700 | 0.0236        | -               | -              |
| 2.7778 | 1800 | 0.026         | -               | -              |
| 2.9321 | 1900 | 0.0217        | -               | -              |
| 3.0864 | 2000 | 0.0193        | 0.0061          | 0.9534         |
| 3.2407 | 2100 | 0.0135        | -               | -              |
| 3.3951 | 2200 | 0.0162        | -               | -              |
| 3.5494 | 2300 | 0.0109        | -               | -              |
| 3.7037 | 2400 | 0.0107        | -               | -              |
| 3.8580 | 2500 | 0.0105        | -               | -              |
| 4.0123 | 2600 | 0.0095        | -               | -              |
| 4.1667 | 2700 | 0.0146        | -               | -              |
| 4.3210 | 2800 | 0.0102        | -               | -              |
| 4.4753 | 2900 | 0.0108        | -               | -              |
| 4.6296 | 3000 | 0.01          | 0.0061          | 0.9602         |
| 4.7840 | 3100 | 0.008         | -               | -              |
| 4.9383 | 3200 | 0.0117        | -               | -              |


### Framework Versions
- Python: 3.11.11
- Sentence Transformers: 3.4.1
- Transformers: 4.51.1
- PyTorch: 2.5.1+cu124
- Accelerate: 1.3.0
- Datasets: 3.5.0
- Tokenizers: 0.21.0

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->