--- tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:10356 - loss:MultipleNegativesRankingLoss base_model: intfloat/multilingual-e5-large widget: - source_sentence: Horn band legwearis a type oflegwear, oftenthighhighs, with ahornedcharacter design along the upper band. sentences: - horn band legwear - head out of frame - sweatpants - source_sentence: When a character is looping the laces of theiruntied shoelacesinto a sturdy bow. sentences: - hair tie - tying footwear - loose necktie - source_sentence: Use this tag if the person's eyewear isremovedfrom their usual place and carried in the hands. If it still rests on the bridge of the nose or head, seeadjusting eyewearand its related tags. sentences: - cow costume - sarong - holding removed eyewear - source_sentence: When both of a character's hands are on another character'sthighs. sentences: - baking - triplets - hands on another's thighs - source_sentence: A long appendage protruding from the lower back. Often covered in fur or scales. A common feature of animal girls. sentences: - tail - grey-framed eyewear - stomach day datasets: - meandyou200175/word_embedding pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy@1 - cosine_accuracy@2 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_accuracy@100 - cosine_precision@1 - cosine_precision@2 - cosine_precision@5 - cosine_precision@10 - cosine_precision@100 - cosine_recall@1 - cosine_recall@2 - cosine_recall@5 - cosine_recall@10 - cosine_recall@100 - cosine_ndcg@10 - cosine_mrr@1 - cosine_mrr@2 - cosine_mrr@5 - cosine_mrr@10 - cosine_mrr@100 - cosine_map@100 model-index: - name: SentenceTransformer based on intfloat/multilingual-e5-large results: - task: type: information-retrieval name: Information Retrieval dataset: name: Unknown type: unknown metrics: - type: cosine_accuracy@1 value: 0.9073359073359073 name: Cosine Accuracy@1 - type: cosine_accuracy@2 value: 0.9739382239382239 name: Cosine Accuracy@2 - type: cosine_accuracy@5 value: 0.9942084942084942 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.999034749034749 name: Cosine Accuracy@10 - type: cosine_accuracy@100 value: 1.0 name: Cosine Accuracy@100 - type: cosine_precision@1 value: 0.9073359073359073 name: Cosine Precision@1 - type: cosine_precision@2 value: 0.48696911196911197 name: Cosine Precision@2 - type: cosine_precision@5 value: 0.19884169884169883 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.0999034749034749 name: Cosine Precision@10 - type: cosine_precision@100 value: 0.010000000000000002 name: Cosine Precision@100 - type: cosine_recall@1 value: 0.9073359073359073 name: Cosine Recall@1 - type: cosine_recall@2 value: 0.9739382239382239 name: Cosine Recall@2 - type: cosine_recall@5 value: 0.9942084942084942 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.999034749034749 name: Cosine Recall@10 - type: cosine_recall@100 value: 1.0 name: Cosine Recall@100 - type: cosine_ndcg@10 value: 0.9601842774877813 name: Cosine Ndcg@10 - type: cosine_mrr@1 value: 0.9073359073359073 name: Cosine Mrr@1 - type: cosine_mrr@2 value: 0.9406370656370656 name: Cosine Mrr@2 - type: cosine_mrr@5 value: 0.9462837837837839 name: Cosine Mrr@5 - type: cosine_mrr@10 value: 0.946988570202856 name: Cosine Mrr@10 - type: cosine_mrr@100 value: 0.9470763202906061 name: Cosine Mrr@100 - type: cosine_map@100 value: 0.9470763202906061 name: Cosine Map@100 --- # SentenceTransformer based on intfloat/multilingual-e5-large This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [intfloat/multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [intfloat/multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 1024 dimensions - **Similarity Function:** Cosine Similarity ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("meandyou200175/e5_large_finetune_word") # Run inference sentences = [ 'A long appendage protruding from the lower back. Often covered in fur or scales. A common feature of animal girls.', 'tail', 'stomach day', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 1024] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:---------------------|:-----------| | cosine_accuracy@1 | 0.9073 | | cosine_accuracy@2 | 0.9739 | | cosine_accuracy@5 | 0.9942 | | cosine_accuracy@10 | 0.999 | | cosine_accuracy@100 | 1.0 | | cosine_precision@1 | 0.9073 | | cosine_precision@2 | 0.487 | | cosine_precision@5 | 0.1988 | | cosine_precision@10 | 0.0999 | | cosine_precision@100 | 0.01 | | cosine_recall@1 | 0.9073 | | cosine_recall@2 | 0.9739 | | cosine_recall@5 | 0.9942 | | cosine_recall@10 | 0.999 | | cosine_recall@100 | 1.0 | | **cosine_ndcg@10** | **0.9602** | | cosine_mrr@1 | 0.9073 | | cosine_mrr@2 | 0.9406 | | cosine_mrr@5 | 0.9463 | | cosine_mrr@10 | 0.947 | | cosine_mrr@100 | 0.9471 | | cosine_map@100 | 0.9471 | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 10,356 training samples * Columns: query and positive * Approximate statistics based on the first 1000 samples: | | query | positive | |:--------|:-----------------------------------------------------------------------------------|:--------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | query | positive | |:------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------| | Eyewear shaped like a semicircle. | semi-circular eyewear | | A handheld electric appliance used fordryingand styling hair. | hair dryer | | When onebreastis exposed while the other remains covered or confined by clothing. Seebreasts outfor when both breasts are exposed. | one breast out | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim" } ``` ### Evaluation Dataset #### word_embedding * Dataset: [word_embedding](https://huggingface.co/datasets/meandyou200175/word_embedding) at [af76b11](https://huggingface.co/datasets/meandyou200175/word_embedding/tree/af76b11c1d93542ca76e864a60b1744d5e02b099) * Size: 1,036 evaluation samples * Columns: query and positive * Approximate statistics based on the first 1000 samples: | | query | positive | |:--------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | query | positive | |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------| | A machine that manipulates data according to a list of instructions. The ability to store and execute lists of instructions called programs make computers extremely versatile. On Danbooru's images they are most often used fordrawing,playing gamesand accessing theinternet. | computer | | Aplaying cardwith twoclubs. | two of clubs | | Yebisu (ヱビス, Ebisu) is a beer produced bySapporo Breweries. It is one of Japan's oldest brands, first being brewed in Tokyo in 1890 by the Japan Beer Brewery Company. | yebisu | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim" } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 16 - `per_device_eval_batch_size`: 16 - `learning_rate`: 2e-05 - `num_train_epochs`: 5 - `warmup_ratio`: 0.1 - `fp16`: True - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 16 - `per_device_eval_batch_size`: 16 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 5 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: True - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `tp_size`: 0 - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: None - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `eval_use_gather_object`: False - `average_tokens_across_devices`: False - `prompts`: None - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | Training Loss | Validation Loss | cosine_ndcg@10 | |:------:|:----:|:-------------:|:---------------:|:--------------:| | -1 | -1 | - | - | 0.7166 | | 0.1543 | 100 | 0.9191 | - | - | | 0.3086 | 200 | 0.1876 | - | - | | 0.4630 | 300 | 0.1547 | - | - | | 0.6173 | 400 | 0.1556 | - | - | | 0.7716 | 500 | 0.179 | - | - | | 0.9259 | 600 | 0.1234 | - | - | | 1.0802 | 700 | 0.087 | - | - | | 1.2346 | 800 | 0.0576 | - | - | | 1.3889 | 900 | 0.0564 | - | - | | 1.5432 | 1000 | 0.0583 | 0.0271 | 0.9198 | | 1.6975 | 1100 | 0.0764 | - | - | | 1.8519 | 1200 | 0.0493 | - | - | | 2.0062 | 1300 | 0.0481 | - | - | | 2.1605 | 1400 | 0.0222 | - | - | | 2.3148 | 1500 | 0.0234 | - | - | | 2.4691 | 1600 | 0.0283 | - | - | | 2.6235 | 1700 | 0.0236 | - | - | | 2.7778 | 1800 | 0.026 | - | - | | 2.9321 | 1900 | 0.0217 | - | - | | 3.0864 | 2000 | 0.0193 | 0.0061 | 0.9534 | | 3.2407 | 2100 | 0.0135 | - | - | | 3.3951 | 2200 | 0.0162 | - | - | | 3.5494 | 2300 | 0.0109 | - | - | | 3.7037 | 2400 | 0.0107 | - | - | | 3.8580 | 2500 | 0.0105 | - | - | | 4.0123 | 2600 | 0.0095 | - | - | | 4.1667 | 2700 | 0.0146 | - | - | | 4.3210 | 2800 | 0.0102 | - | - | | 4.4753 | 2900 | 0.0108 | - | - | | 4.6296 | 3000 | 0.01 | 0.0061 | 0.9602 | | 4.7840 | 3100 | 0.008 | - | - | | 4.9383 | 3200 | 0.0117 | - | - | ### Framework Versions - Python: 3.11.11 - Sentence Transformers: 3.4.1 - Transformers: 4.51.1 - PyTorch: 2.5.1+cu124 - Accelerate: 1.3.0 - Datasets: 3.5.0 - Tokenizers: 0.21.0 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```