Add new CrossEncoder model

README.md

@@ -0,0 +1,503 @@
+---
+language:
+- en
+tags:
+- sentence-transformers
+- cross-encoder
+- generated_from_trainer
+- dataset_size:78704
+- loss:PListMLELoss
+base_model: microsoft/MiniLM-L12-H384-uncased
+datasets:
+- microsoft/ms_marco
+pipeline_tag: text-ranking
+library_name: sentence-transformers
+metrics:
+- map
+- mrr@10
+- ndcg@10
+model-index:
+- name: CrossEncoder based on microsoft/MiniLM-L12-H384-uncased
+  results:
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoMSMARCO R100
+      type: NanoMSMARCO_R100
+    metrics:
+    - type: map
+      value: 0.5257
+      name: Map
+    - type: mrr@10
+      value: 0.5139
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.5778
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoNFCorpus R100
+      type: NanoNFCorpus_R100
+    metrics:
+    - type: map
+      value: 0.3387
+      name: Map
+    - type: mrr@10
+      value: 0.5921
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.366
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoNQ R100
+      type: NanoNQ_R100
+    metrics:
+    - type: map
+      value: 0.5581
+      name: Map
+    - type: mrr@10
+      value: 0.5648
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.6325
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-nano-beir
+      name: Cross Encoder Nano BEIR
+    dataset:
+      name: NanoBEIR R100 mean
+      type: NanoBEIR_R100_mean
+    metrics:
+    - type: map
+      value: 0.4742
+      name: Map
+    - type: mrr@10
+      value: 0.5569
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.5254
+      name: Ndcg@10
+---
+
+# CrossEncoder based on microsoft/MiniLM-L12-H384-uncased
+
+This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) on the [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) dataset using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Cross Encoder
+- **Base model:** [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) <!-- at revision 44acabbec0ef496f6dbc93adadea57f376b7c0ec -->
+- **Maximum Sequence Length:** 512 tokens
+- **Number of Output Labels:** 1 label
+- **Training Dataset:**
+    - [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco)
+- **Language:** en
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import CrossEncoder
+
+# Download from the 🤗 Hub
+model = CrossEncoder("yjoonjang/reranker-msmarco-v1.1-MiniLM-L12-H384-uncased-plistmle-normalize-temperature-2")
+# Get scores for pairs of texts
+pairs = [
+    ['How many calories in an egg', 'There are on average between 55 and 80 calories in an egg depending on its size.'],
+    ['How many calories in an egg', 'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.'],
+    ['How many calories in an egg', 'Most of the calories in an egg come from the yellow yolk in the center.'],
+]
+scores = model.predict(pairs)
+print(scores.shape)
+# (3,)
+
+# Or rank different texts based on similarity to a single text
+ranks = model.rank(
+    'How many calories in an egg',
+    [
+        'There are on average between 55 and 80 calories in an egg depending on its size.',
+        'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.',
+        'Most of the calories in an egg come from the yellow yolk in the center.',
+    ]
+)
+# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Cross Encoder Reranking
+
+* Datasets: `NanoMSMARCO_R100`, `NanoNFCorpus_R100` and `NanoNQ_R100`
+* Evaluated with [<code>CrossEncoderRerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderRerankingEvaluator) with these parameters:
+  ```json
+  {
+      "at_k": 10,
+      "always_rerank_positives": true
+  }
+  ```
+
+| Metric      | NanoMSMARCO_R100     | NanoNFCorpus_R100    | NanoNQ_R100          |
+|:------------|:---------------------|:---------------------|:---------------------|
+| map         | 0.5257 (+0.0362)     | 0.3387 (+0.0777)     | 0.5581 (+0.1385)     |
+| mrr@10      | 0.5139 (+0.0364)     | 0.5921 (+0.0923)     | 0.5648 (+0.1381)     |
+| **ndcg@10** | **0.5778 (+0.0374)** | **0.3660 (+0.0410)** | **0.6325 (+0.1319)** |
+
+#### Cross Encoder Nano BEIR
+
+* Dataset: `NanoBEIR_R100_mean`
+* Evaluated with [<code>CrossEncoderNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderNanoBEIREvaluator) with these parameters:
+  ```json
+  {
+      "dataset_names": [
+          "msmarco",
+          "nfcorpus",
+          "nq"
+      ],
+      "rerank_k": 100,
+      "at_k": 10,
+      "always_rerank_positives": true
+  }
+  ```
+
+| Metric      | Value                |
+|:------------|:---------------------|
+| map         | 0.4742 (+0.0841)     |
+| mrr@10      | 0.5569 (+0.0889)     |
+| **ndcg@10** | **0.5254 (+0.0701)** |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### ms_marco
+
+* Dataset: [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) at [a47ee7a](https://huggingface.co/datasets/microsoft/ms_marco/tree/a47ee7aae8d7d466ba15f9f0bfac3b3681087b3a)
+* Size: 78,704 training samples
+* Columns: <code>query</code>, <code>docs</code>, and <code>labels</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                                          | docs                                                                                   | labels                                                                                 |
+  |:--------|:-----------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|
+  | type    | string                                                                                         | list                                                                                   | list                                                                                   |
+  | details | <ul><li>min: 12 characters</li><li>mean: 33.99 characters</li><li>max: 98 characters</li></ul> | <ul><li>min: 3 elements</li><li>mean: 6.50 elements</li><li>max: 10 elements</li></ul> | <ul><li>min: 3 elements</li><li>mean: 6.50 elements</li><li>max: 10 elements</li></ul> |
+* Samples:
+  | query                                                       | docs                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | labels                            |
+  |:------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------|
+  | <code>what does a business development consultant do</code> | <code>['Duties and Responsibilities. An organizational development consultant is a person called in to a company, be it a large corporation or a small business, to evaluate how it operates and make recommendations for improvement.', 'Many sales businesses use business development consultants to help generate leads and show them how to do so. In a business such as sales, lead generation can make or break a company. Having someone show a business owner how to successfully acquire this key piece of information is very important.', 'Development of a marketing strategy is another area covered by a business development consultant. Many businesses struggle with devising ways to effectively market their business to prospective clients.', 'A Good Business Consultant Has Extensive Experience. A good Business Consultant has experience working in and working with a broad range of businesses. It is the accumulated business history of a Business Consultant which makes the consultant valuable.', "A busines...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>did soren kjeldsen ever play in the masters</code>    | <code>["Recent News. Soeren Søren kjeldsen stalled on the back nine and finished joint-runner up in The British masters At woburn with a-2-under par-32=37. 69 The, dane who Captured'may S Irish, open looked good for his second win of the season when shooting four birdies against a single bogey on The' marquess'course s front. nine", "Latest News. Soeren Søren kjeldsen stalled on the back nine and finished joint-runner up in The British masters At woburn with a-2-under par-32=37. 69 The, dane who Captured'may S Irish, open looked good for his second win of the season when shooting four birdies against a single bogey on The' marquess'course s front. nine", "Soren Kjeldsen of Denmark (L) celebrates winning the Irish Open with Austria's Bernd Wiesberger …. It is amazing to be holding the trophy but then I felt good coming into the tournament, he said. I played well in my last two tournaments and while I was not in contention I had the chance today to change all that.", "Denmark. Soeren Søren kje...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>what is a sell limit order</code>                     | <code>["Sell Limit Order. A Sell Limit Order is an order to sell a specified number of shares of a stock that you own at a designated price or higher, at a price that is above the current market price. This is your limit price, in other words, the minimum price you are willing to accept to sell your shares. The main benefit of a Sell Limit Order is that you may be able to sell the shares that you own at a minimum price that you specify IF the stock's price raises to that price. Sell Limit Orders are great for maximizing profit-taking.", 'Stop-Limit Order. A stop-limit order is an order to buy or sell a stock that combines the features of a stop order and a limit order. Once the stop price is reached, a stop-limit order becomes a limit order that will be executed at a specified price (or better)', "You place a Sell Limit Order @ $50 on 100 shares of TGT. Now suppose the price trades up to $50. As long as the price remains above $50 per share, your shares would then be sold at the next best av...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+* Loss: [<code>PListMLELoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#plistmleloss) with these parameters:
+  ```json
+  {
+      "lambda_weight": "sentence_transformers.cross_encoder.losses.PListMLELoss.PListMLELambdaWeight",
+      "activation_fct": "torch.nn.modules.linear.Identity",
+      "mini_batch_size": null,
+      "respect_input_order": true
+  }
+  ```
+
+### Evaluation Dataset
+
+#### ms_marco
+
+* Dataset: [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) at [a47ee7a](https://huggingface.co/datasets/microsoft/ms_marco/tree/a47ee7aae8d7d466ba15f9f0bfac3b3681087b3a)
+* Size: 1,000 evaluation samples
+* Columns: <code>query</code>, <code>docs</code>, and <code>labels</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                                           | docs                                                                                   | labels                                                                                 |
+  |:--------|:------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|
+  | type    | string                                                                                          | list                                                                                   | list                                                                                   |
+  | details | <ul><li>min: 11 characters</li><li>mean: 33.46 characters</li><li>max: 108 characters</li></ul> | <ul><li>min: 2 elements</li><li>mean: 6.00 elements</li><li>max: 10 elements</li></ul> | <ul><li>min: 2 elements</li><li>mean: 6.00 elements</li><li>max: 10 elements</li></ul> |
+* Samples:
+  | query                                               | docs                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | labels                            |
+  |:----------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------|
+  | <code>how much to spend on wordpress hosting</code> | <code>['Shared server. This will cost as little as $3 per month to around $10 per month depending on how you want to pay for it (by the month or by the year). The performance of your site will suffer from shared hosting. It’s a good choice for a personal blog or for getting you started. 1 Courses – You can take courses online that are free for extremely basic information or spend up to $200 or more for mid to advanced topics. 2  Some courses cost from $20 – $50 for a monthly subscription. 3  This allows you to pay for as much training as you want. 4  This can still cost hundreds of dollars', 'You may also choose to invest in customization, SEO or other factors along the way. If your interest is simply to start a blog on WordPress, you can start with a minimal cost of $60 for unlimited hosting and free domain with Bluehost. You can learn all about which hosting service is best for WordPress here. Domain: Cost – $10. The first element you need to shop for is a domain. Having a domain name is g...</code> | <code>[1, 1, 0, 0, 0, ...]</code> |
+  | <code>what type blood is the universal donor</code> | <code>["At one time, type O negative blood was considered the universal blood donor type. This implied that anyone — regardless of blood type — could receive type O negative blood without risking a transfusion reaction. Even then, small samples of the recipient's and donor's blood are mixed to check compatibility in a process known as crossmatching. In an emergency, however, type O negative red blood cells may be given to anyone — especially if the situation is life-threatening or the matching blood type is in short supply.", 'People with type O Rh D negative blood are often called universal donors. O Rh D negative is the universal donor because it does not contain any antigens (markers). When you … get donated blood that has antigens that are not the same as those of the recipient the blood will clot in the body. AB is a universal acceptor because RBC (red blood cells) contain the A and B antigen (simply put, it is a marker on the cell) so the body a … ccepts any blood type because it recog...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>dental crown costs average</code>             | <code>['The prices for dental crowns range from $500 to $2,500 per crown and are dependent upon the materials used, location of tooth and geographic location. The average cost of a crown is $825, with or without dental insurance coverage. The cheapest cost of a dental crown is $500 for a simple metal crown. Dental crowns are specifically shaped shells that fit over damaged or broken teeth for either cosmetic or structural purposes. 1 People with insurance typically paid $520 – $1,140 out of pocket with an average of $882 per crown. 2  Those without insurance generally paid between $830 and $2,465 per crown with an average cost of $1,350.', '1 All-porcelain crowns require a higher level of skill and take more time to install than metal or porcelain-fused-to-metal crowns, and can cost $800-$3,000 or more per tooth. 2  CostHelper readers without insurance report paying $860-$3,000, at an average cost of $1,430. 1 CostHelper readers without insurance report paying $860-$3,000, at an average cost...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+* Loss: [<code>PListMLELoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#plistmleloss) with these parameters:
+  ```json
+  {
+      "lambda_weight": "sentence_transformers.cross_encoder.losses.PListMLELoss.PListMLELambdaWeight",
+      "activation_fct": "torch.nn.modules.linear.Identity",
+      "mini_batch_size": null,
+      "respect_input_order": true
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `learning_rate`: 2e-05
+- `num_train_epochs`: 1
+- `warmup_ratio`: 0.1
+- `seed`: 12
+- `bf16`: True
+- `load_best_model_at_end`: True
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 2e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 12
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: True
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: True
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+| Epoch      | Step     | Training Loss | Validation Loss | NanoMSMARCO_R100_ndcg@10 | NanoNFCorpus_R100_ndcg@10 | NanoNQ_R100_ndcg@10  | NanoBEIR_R100_mean_ndcg@10 |
+|:----------:|:--------:|:-------------:|:---------------:|:------------------------:|:-------------------------:|:--------------------:|:--------------------------:|
+| -1         | -1       | -             | -               | 0.0375 (-0.5029)         | 0.2604 (-0.0646)          | 0.0219 (-0.4788)     | 0.1066 (-0.3488)           |
+| 0.0002     | 1        | 1713.1071     | -               | -                        | -                         | -                    | -                          |
+| 0.0508     | 250      | 1833.4537     | -               | -                        | -                         | -                    | -                          |
+| 0.1016     | 500      | 1790.301      | 1707.9830       | 0.1182 (-0.4222)         | 0.2072 (-0.1178)          | 0.3276 (-0.1730)     | 0.2177 (-0.2377)           |
+| 0.1525     | 750      | 1775.4549     | -               | -                        | -                         | -                    | -                          |
+| 0.2033     | 1000     | 1716.7897     | 1638.4917       | 0.5203 (-0.0201)         | 0.3349 (+0.0099)          | 0.6145 (+0.1138)     | 0.4899 (+0.0345)           |
+| 0.2541     | 1250     | 1734.1811     | -               | -                        | -                         | -                    | -                          |
+| 0.3049     | 1500     | 1707.1166     | 1619.5133       | 0.5134 (-0.0270)         | 0.3245 (-0.0005)          | 0.6225 (+0.1218)     | 0.4868 (+0.0314)           |
+| 0.3558     | 1750     | 1715.8994     | -               | -                        | -                         | -                    | -                          |
+| 0.4066     | 2000     | 1682.5393     | 1630.9360       | 0.5278 (-0.0127)         | 0.3434 (+0.0184)          | 0.5907 (+0.0900)     | 0.4873 (+0.0319)           |
+| 0.4574     | 2250     | 1705.7818     | -               | -                        | -                         | -                    | -                          |
+| **0.5082** | **2500** | **1650.1962** | **1599.1906**   | **0.5778 (+0.0374)**     | **0.3660 (+0.0410)**      | **0.6325 (+0.1319)** | **0.5254 (+0.0701)**       |
+| 0.5591     | 2750     | 1651.8559     | -               | -                        | -                         | -                    | -                          |
+| 0.6099     | 3000     | 1677.6405     | 1594.7935       | 0.5657 (+0.0253)         | 0.3514 (+0.0263)          | 0.6304 (+0.1298)     | 0.5158 (+0.0605)           |
+| 0.6607     | 3250     | 1690.9901     | -               | -                        | -                         | -                    | -                          |
+| 0.7115     | 3500     | 1647.8661     | 1597.9960       | 0.5553 (+0.0149)         | 0.3582 (+0.0331)          | 0.6342 (+0.1335)     | 0.5159 (+0.0605)           |
+| 0.7624     | 3750     | 1657.8038     | -               | -                        | -                         | -                    | -                          |
+| 0.8132     | 4000     | 1670.0114     | 1591.1512       | 0.5429 (+0.0025)         | 0.3617 (+0.0367)          | 0.6377 (+0.1370)     | 0.5141 (+0.0587)           |
+| 0.8640     | 4250     | 1678.4298     | -               | -                        | -                         | -                    | -                          |
+| 0.9148     | 4500     | 1687.3654     | 1587.0916       | 0.5427 (+0.0023)         | 0.3549 (+0.0299)          | 0.6317 (+0.1310)     | 0.5098 (+0.0544)           |
+| 0.9656     | 4750     | 1645.7461     | -               | -                        | -                         | -                    | -                          |
+| -1         | -1       | -             | -               | 0.5778 (+0.0374)         | 0.3660 (+0.0410)          | 0.6325 (+0.1319)     | 0.5254 (+0.0701)           |
+
+* The bold row denotes the saved checkpoint.
+
+### Framework Versions
+- Python: 3.11.11
+- Sentence Transformers: 3.5.0.dev0
+- Transformers: 4.49.0
+- PyTorch: 2.6.0+cu124
+- Accelerate: 1.5.2
+- Datasets: 3.4.0
+- Tokenizers: 0.21.1
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### PListMLELoss
+```bibtex
+@inproceedings{lan2014position,
+  title={Position-Aware ListMLE: A Sequential Learning Process for Ranking.},
+  author={Lan, Yanyan and Zhu, Yadong and Guo, Jiafeng and Niu, Shuzi and Cheng, Xueqi},
+  booktitle={UAI},
+  volume={14},
+  pages={449--458},
+  year={2014}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,34 @@
+{
+  "_name_or_path": "microsoft/MiniLM-L12-H384-uncased",
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "sentence_transformers": {
+    "activation_fn": "torch.nn.modules.activation.Sigmoid"
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.49.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5471f2a74a673a0200c71249f8469ca8347a5292a36523077fe89e10f476eb7f
+size 133464836

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}