Add new SparseEncoder model

README.md

@@ -8,34 +8,45 @@ tags:
 - sparse
 - splade
 - generated_from_trainer
-- dataset_size:800000
+- dataset_size:496123
 - loss:SpladeLoss
-- loss:SparseMultipleNegativesRankingLoss
+- loss:SparseMarginMSELoss
 - loss:FlopsLoss
 base_model: prajjwal1/bert-tiny
 widget:
-- text: how much do private cleaners charge per hour
-- text: atlantic ocean air currents affects climate
-- text: >-
-    RNA polymerase is the core enzyme in transcription which needs proteins
-    known as transcription factors to bind to the DNA promoter. also DNA
-    plymerase... can't remember what about it though. DNA polymerase isn't
-    involved in DNA transcription. However, DNA polymerase IS involved in DNA
-    REPLICATION.
-- text: >-
-    Exploit:JS/Axpergle.E Virus is a threatening Trojan horse which gets itself
-    loaded when you turn on your computer and eats up lots of system resources.
-    Once this Exploit:JS/Axpergle.E virus successfully enters your operating
-    system, your computer will be subjected to a variety of errors and drive you
-    mad.) Exploit:JS/Axpergle.E Virus corrupts the data and files saved on your
-    computer hard drive terribly. 2) Exploit:JS/Axpergle.E Virus changes the
-    registry entry to get itself launched at system startup.
-- text: >-
-    --No depreciation deduction shall be allowed under this section (and no
-    depreciation or amortization deduction shall be allowed under any other
-    provision of this subtitle) to the taxpayer for any term interest in
-    property for any period during which the remainder interest in such property
-    is held (directly or indirectly) by a related person.
+- text: Hurley doesn't just want to be your go-to for surf gear, but the be the brand
+    that represents your lifestyle. Of course you have your pick up board shorts,
+    tanks and a Hurley hat while you're on the beach, but you can also look at graphic
+    tees, sandals, and accessories when you're on the street.
+- text: 'Electric field of a positive and a negative point charge. Electric charge
+    is the physical property of matter that causes it to experience a force when placed
+    in an electromagnetic field.There are two types of electric charges: positive
+    and negative.lectric charge is a characteristic property of many subatomic particles.
+    The charges of free-standing particles are integer multiples of the elementary
+    charge e; we say that electric charge is quantized. Michael Faraday, in his electrolysis
+    experiments, was the first to note the discrete nature of electric charge.'
+- text: The term mechanical digestion refers to the physical breakdown of large pieces
+    of food into smaller pieces which can subsequently be accessed by digestive enzymes.
+    In chemical digestion, enzymes break down food into the small molecules the body
+    can use.
+- text: Kids and Quick Solutions. Children learn to put away their clothes when they
+    can reach the hanging rods. This is actually fun for little ones -- they may spend
+    a long stretch of time putting hangers on and taking them off the rods -- as long
+    as the rods are child-height.So take your stand against piles of clothes on the
+    floor of the teen's bedroom early by re-sizing the closet to fit the kid.his is
+    actually fun for little ones -- they may spend a long stretch of time putting
+    hangers on and taking them off the rods -- as long as the rods are child-height.
+    So take your stand against piles of clothes on the floor of the teen's bedroom
+    early by re-sizing the closet to fit the kid.
+- text: About EUS (endoscopic ultrasound). An EUS, or endoscopic ultrasound, is an
+    outpatient procedure used to closely examine the tissues in the digestive tract.
+    The procedure is done using a standard endoscope and a tiny ultrasound device.The
+    ultrasound sensor sends back visual images of the digestive tract to a screen,
+    allowing the physician to see deeper into the tissues and the organs beneath the
+    surface of the intestines.. In general, an EUS is a very safe procedure. If your
+    procedure is being done on the upper GI tract, you may have a sore throat for
+    a few days. As a result of the sedation, you should not drive, operate heavy machinery
+    or make any important decisions for up to six hours following the procedure.
 pipeline_tag: feature-extraction
 library_name: sentence-transformers
 metrics:
@@ -59,7 +70,7 @@ metrics:
 - corpus_active_dims
 - corpus_sparsity_ratio
 model-index:
-- name: SPLADE-BERT-Tiny
+- name: SPLADE-BERT-Tiny-Distil
   results:
   - task:
       type: sparse-information-retrieval
@@ -69,67 +80,65 @@ model-index:
       type: unknown
     metrics:
     - type: dot_accuracy@1
-      value: 0.457
+      value: 0.4602
       name: Dot Accuracy@1
     - type: dot_accuracy@3
-      value: 0.7572
+      value: 0.7768
       name: Dot Accuracy@3
     - type: dot_accuracy@5
-      value: 0.8574
+      value: 0.885
       name: Dot Accuracy@5
     - type: dot_accuracy@10
-      value: 0.929
+      value: 0.9548
       name: Dot Accuracy@10
     - type: dot_precision@1
-      value: 0.457
+      value: 0.4602
       name: Dot Precision@1
     - type: dot_precision@3
-      value: 0.25906666666666667
+      value: 0.2653333333333333
       name: Dot Precision@3
     - type: dot_precision@5
-      value: 0.178
+      value: 0.18391999999999997
       name: Dot Precision@5
     - type: dot_precision@10
-      value: 0.09714
+      value: 0.10024
       name: Dot Precision@10
     - type: dot_recall@1
-      value: 0.44155
+      value: 0.4461833333333334
       name: Dot Recall@1
     - type: dot_recall@3
-      value: 0.7427833333333334
+      value: 0.7631166666666666
       name: Dot Recall@3
     - type: dot_recall@5
-      value: 0.8471666666666666
+      value: 0.8761
       name: Dot Recall@5
     - type: dot_recall@10
-      value: 0.9223
+      value: 0.9500333333333334
       name: Dot Recall@10
     - type: dot_ndcg@10
-      value: 0.6931598312411338
+      value: 0.7094495794736737
       name: Dot Ndcg@10
     - type: dot_mrr@10
-      value: 0.6234866666666686
+      value: 0.6344716666666689
       name: Dot Mrr@10
     - type: dot_map@100
-      value: 0.6191148055389254
+      value: 0.6306882016403095
       name: Dot Map@100
     - type: query_active_dims
-      value: 21.215999603271484
+      value: 16.77560043334961
       name: Query Active Dims
     - type: query_sparsity_ratio
-      value: 0.9993048948429568
+      value: 0.9994503767632085
       name: Query Sparsity Ratio
     - type: corpus_active_dims
-      value: 159.5419082486014
+      value: 102.47956598021874
       name: Corpus Active Dims
     - type: corpus_sparsity_ratio
-      value: 0.99477288813811
+      value: 0.9966424360795421
       name: Corpus Sparsity Ratio
-datasets:
-- microsoft/ms_marco
 ---
 
-# SPLADE-BERT-Tiny
+# SPLADE-BERT-Tiny-Distil
 
 This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [prajjwal1/bert-tiny](https://huggingface.co/prajjwal1/bert-tiny) using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space   and can be used for semantic search and sparse retrieval.
 ## Model Details
@@ -175,15 +184,15 @@ Then you can load this model and run inference.
 from sentence_transformers import SparseEncoder
 
 # Download from the 🤗 Hub
-model = SparseEncoder("rasyosef/SPLADE-BERT-Tiny")
+model = SparseEncoder("yosefw/SPLADE-BERT-Tiny-distil-msmarco")
 # Run inference
 queries = [
-    "what code section is depreciation",
+    "what is eus appointment",
 ]
 documents = [
-    'Section 179 depreciation deduction. Section 179 of the United States Internal Revenue Code (26 U.S.C. § 179), allows a taxpayer to elect to deduct the cost of certain types of property on their income taxes as an expense, rather than requiring the cost of the property to be capitalized and depreciated.',
-    '--No depreciation deduction shall be allowed under this section (and no depreciation or amortization deduction shall be allowed under any other provision of this subtitle) to the taxpayer for any term interest in property for any period during which the remainder interest in such property is held (directly or indirectly) by a related person.',
-    'Depreciation - Amortization Code. Refer to the IRS Instructions for Form 4562, Line 42, for the amortization code.',
+    "Endoscopic Ultrasound (EUS). You've been referred to have an endoscopic ultrasound, or EUS, which will help your doctor, evaluate or treat your condition. This brochure will give you a basic understanding of the procedure-how it is performed, how it can help, and what side effects you might experience.our doctor can use EUS to diagnose the cause of conditions such as abdominal pain or abnormal weight loss. Or, if your doctor has ruled out certain conditions, EUS can confirm your diagnosis and give you a clean bill of health.",
+    'About EUS (endoscopic ultrasound). An EUS, or endoscopic ultrasound, is an outpatient procedure used to closely examine the tissues in the digestive tract. The procedure is done using a standard endoscope and a tiny ultrasound device.The ultrasound sensor sends back visual images of the digestive tract to a screen, allowing the physician to see deeper into the tissues and the organs beneath the surface of the intestines.. In general, an EUS is a very safe procedure. If your procedure is being done on the upper GI tract, you may have a sore throat for a few days. As a result of the sedation, you should not drive, operate heavy machinery or make any important decisions for up to six hours following the procedure.',
+    'Endoscopic Ultrasound (EUS) allows your doctor to examine the lining and the walls of your upper and lower gastrointestinal tract.The upper tract is the esophagus, stomach, and duodenum; the lower tract includes your colon and rectum.Doctors also use EUS to study internal organs that lie next to the gastrointestinal tract, such as the gall bladder and the pancreas. Your endoscopist will use a thin, flexible tube called an endoscope.he upper tract is the esophagus, stomach, and duodenum; the lower tract includes your colon and rectum. Doctors also use EUS to study internal organs that lie next to the gastrointestinal tract, such as the gall bladder and the pancreas.',
 ]
 query_embeddings = model.encode_query(queries)
 document_embeddings = model.encode_document(documents)
@@ -193,7 +202,7 @@ print(query_embeddings.shape, document_embeddings.shape)
 # Get the similarity scores for the embeddings
 similarities = model.similarity(query_embeddings, document_embeddings)
 print(similarities)
-# tensor([[17.0167, 11.4943, 13.8083]])
+# tensor([[12.9370, 14.3277, 12.9725]])
 ```
 
 <!--
@@ -230,25 +239,25 @@ You can finetune this model on your own dataset.
 
 | Metric                | Value      |
 |:----------------------|:-----------|
-| dot_accuracy@1        | 0.457      |
-| dot_accuracy@3        | 0.7572     |
-| dot_accuracy@5        | 0.8574     |
-| dot_accuracy@10       | 0.929      |
-| dot_precision@1       | 0.457      |
-| dot_precision@3       | 0.2591     |
-| dot_precision@5       | 0.178      |
-| dot_precision@10      | 0.0971     |
-| dot_recall@1          | 0.4415     |
-| dot_recall@3          | 0.7428     |
-| dot_recall@5          | 0.8472     |
-| dot_recall@10         | 0.9223     |
-| **dot_ndcg@10**       | **0.6932** |
-| dot_mrr@10            | 0.6235     |
-| dot_map@100           | 0.6191     |
-| query_active_dims     | 21.216     |
-| query_sparsity_ratio  | 0.9993     |
-| corpus_active_dims    | 159.5419   |
-| corpus_sparsity_ratio | 0.9948     |
+| dot_accuracy@1        | 0.4602     |
+| dot_accuracy@3        | 0.7768     |
+| dot_accuracy@5        | 0.885      |
+| dot_accuracy@10       | 0.9548     |
+| dot_precision@1       | 0.4602     |
+| dot_precision@3       | 0.2653     |
+| dot_precision@5       | 0.1839     |
+| dot_precision@10      | 0.1002     |
+| dot_recall@1          | 0.4462     |
+| dot_recall@3          | 0.7631     |
+| dot_recall@5          | 0.8761     |
+| dot_recall@10         | 0.95       |
+| **dot_ndcg@10**       | **0.7094** |
+| dot_mrr@10            | 0.6345     |
+| dot_map@100           | 0.6307     |
+| query_active_dims     | 16.7756    |
+| query_sparsity_ratio  | 0.9995     |
+| corpus_active_dims    | 102.4796   |
+| corpus_sparsity_ratio | 0.9966     |
 
 <!--
 ## Bias, Risks and Limitations
@@ -268,25 +277,25 @@ You can finetune this model on your own dataset.
 
 #### Unnamed Dataset
 
-* Size: 800,000 training samples
-* Columns: <code>query</code>, <code>positive</code>, <code>negative_1</code>, and <code>negative_2</code>
+* Size: 496,123 training samples
+* Columns: <code>query</code>, <code>positive</code>, <code>negative_1</code>, <code>negative_2</code>, <code>negative_3</code>, <code>negative_4</code>, and <code>label</code>
 * Approximate statistics based on the first 1000 samples:
-  |         | query                                                                            | positive                                                                            | negative_1                                                                          | negative_2                                                                          |
-  |:--------|:---------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
-  | type    | string                                                                           | string                                                                              | string                                                                              | string                                                                              |
-  | details | <ul><li>min: 4 tokens</li><li>mean: 9.03 tokens</li><li>max: 30 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 81.92 tokens</li><li>max: 220 tokens</li></ul> | <ul><li>min: 22 tokens</li><li>mean: 78.63 tokens</li><li>max: 227 tokens</li></ul> | <ul><li>min: 18 tokens</li><li>mean: 78.11 tokens</li><li>max: 236 tokens</li></ul> |
+  |         | query                                                                            | positive                                                                            | negative_1                                                                          | negative_2                                                                         | negative_3                                                                          | negative_4                                                                         | label                              |
+  |:--------|:---------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------|
+  | type    | string                                                                           | string                                                                              | string                                                                              | string                                                                             | string                                                                              | string                                                                             | list                               |
+  | details | <ul><li>min: 4 tokens</li><li>mean: 9.09 tokens</li><li>max: 37 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 80.68 tokens</li><li>max: 215 tokens</li></ul> | <ul><li>min: 20 tokens</li><li>mean: 78.57 tokens</li><li>max: 238 tokens</li></ul> | <ul><li>min: 18 tokens</li><li>mean: 77.8 tokens</li><li>max: 253 tokens</li></ul> | <ul><li>min: 18 tokens</li><li>mean: 76.46 tokens</li><li>max: 248 tokens</li></ul> | <ul><li>min: 16 tokens</li><li>mean: 75.9 tokens</li><li>max: 190 tokens</li></ul> | <ul><li>size: 4 elements</li></ul> |
 * Samples:
-  | query                                                  | positive                                                                                                                                                                                                                                                                                                                                      | negative_1                                                                                                                                                                                                                                                                                                                         | negative_2                                                                                                                                                                                                                                                                                                |
-  |:-------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-  | <code>definition of vas deferens</code>                | <code>Vas deferens: The tube that connects the testes with the urethra. The vas deferens is a coiled duct that conveys sperm from the epididymis to the ejaculatory duct and the urethra.</code>                                                                                                                                              | <code>For further discussion of the vas deferens within the context of the structures and functions of reproduction and sexuality, please see the overview section “The Reproductive System.”. See also FERTILITY; TESTICLES; VASECTOMY.</code>                                                                                    | <code>1 Testicular cancer symptoms include a painless lump or swelling in a testicle, testicle or scrotum pain, a dull ache in the abdomen, back, or groin, and. 2  Urinary Tract Infections (UTIs)  A urinary tract infection (UTI) is an infection of the bladder, kidneys, ureters, or urethra.</code> |
-  | <code>how old is kieron williamson</code>              | <code>Kieron Williamson – the latest artist to be part of GoGoDragons! April 21, 2015. A 12-year-old artist, nicknamed Mini-Monet, is to unveil a sculpture of a dragon he has painted for GoGoDragons. Kieron Williamson, from Norfolk, who has so far earned about £2m, painted the 5ft-tall (1.5m) dragon for the event in Norwich.</code> | <code>8-year-old artist: Don't call me Monet. London, England (CNN) -- He has the deft brush strokes of a seasoned artist, but Kieron Williamson is just eight years old. The boy from Norfolk, in eastern England, has been hailed by the British press as a mini Monet, a reference to the famous French impressionist.</code>   | <code>Needless to say, this site does not tell you much about his football career (yet!), but the website will tell you everything there is to know about Kieron Williamson’s passion for oil, watercolour and pastel,</code>                                                                             |
-  | <code>when do you start showing third pregnancy</code> | <code>Yes | No Thank you! I am pregnant with my third child and I am definitly showing at 10 weeks. I am starting to wear some maternity clothes. My low low rise pre-pregnancy jeans still work. My biggest problem is shirts, but fortunately the style right now is loose shirts that look maternity.</code>                               | <code>Some women do not start to show until they are well into their second trimester or even the start of their third trimester. If you are overweight at the start of your pregnancy, you may not gain as much weight during your pregnancy and may not begin to show until later into your pregnancy. Average: 3.591215.</code> | <code>There isn't a set time when moms-to-be start sporting an obviously pregnant belly – every woman is different. Some women keep their pre-pregnancy belly far into the second trimester, while others start showing in the first trimester.</code>                                                    |
+  | query                                            | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        | negative_1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | negative_2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | negative_3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                | negative_4                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | label                                                                                       |
+  |:-------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------|
+  | <code>could Nexium antacid cause sweating</code> | <code>Summary: Sweating-excessive is found among people who take Nexium, especially for people who are 60+ old, have been taking the drug for.Personalized health information: on eHealthMe you can find out what patients like me (same gender, age) reported their drugs and conditions on FDA and social media since 1977. I am a 56 year old female who has been taking Nexium for 13 years and has been plagued by shingles.. 2  Support group for people who have Sweating-Excessive.  3 Been on warfarin for 6 days and having sweating at times.</code> | <code>More questions for: Nexium, Sweating-excessive. You may be interested at these reviews (Write a review): 1  Xarelto caused shortness of breath. 2  After taking Xarelto for 3 years I suddently experienced shortness of breath, sweating and pain in my arms. 3  Myrbetriq & hyperhidrosis (night sweats). I am a 56 year old female who has been taking Nexium for 13 years and has been plagued by shingles.. 2  Support group for people who have Sweating-Excessive.  3 Been on warfarin for 6 days and having sweating at times.</code> | <code>NEXIUM may help your acid-related symptoms, but you could still have serious stomach problems. Talk with your doctor. NEXIUM can cause serious side effects, including: 1  Diarrhea. 2  NEXIUM may increase your risk of getting severe diarrhea.3  This diarrhea may be caused by an infection (Clostridium difficile) in your intestines.EXIUM can cause serious side effects, including: 1  Diarrhea. 2  NEXIUM may increase your risk of getting severe diarrhea. 3  This diarrhea may be caused by an infection (Clostridium difficile) in your intestines.</code> | <code>Treatment for sweating. The treatment you have will depend on the cause of your sweating. If you have an infection, antibiotics will treat the infection and stop the sweating. If your sweating is due to cancer, treating the cancer can get rid of the sweating.If you have sweating because treatment has changed your hormone levels, it may settle down after a few weeks or months, once your body is used to the treatment. Talk to your doctor or nurse about your sweats.nfection. Infection is one of the most common causes of sweating in people who have cancer. Infection can give you a high temperature and your body sweats to try and reduce it. Treating the infection can control or stop the sweating.</code> | <code>Esomeprazole is used to treat certain stomach and esophagus problems (such as acid reflux, ulcers). It works by decreasing the amount of acid your stomach makes.ide Effects. See also Precautions section. Headache or abdominal pain may occur. If any of these effects persist or worsen, tell your doctor or pharmacist promptly. Remember that your doctor has prescribed this medication because he or she has judged that the benefit to you is greater than the risk of side effects.</code> | <code>[0.5, 6.390576362609863, 11.97206974029541, 16.409034729003906]</code>                |
+  | <code>what is electronic document access</code>  | <code>Electronic Document Access (EDA) is a web-based system that provides secure online access, storage, and retrieval of contracts, contract modifications, Government Bills of Lading (GBLs), DFAS Transactions for Others (E110), vouchers, and Contract Deficiency Reports (CDR) to authorized users throughout the Department of Defense (DoD).</code>                                                                                                                                                                                                    | <code>An electronic document management system (EDMS) is a software system for organizing and storing different kinds of documents. This type of system is a more particular kind of document management system, a more general type of storage system that helps users to organize and store paper or digital documents.</code>                                                                                                                                                                                                                    | <code>In many cases, the specific documentation for original storage protocols is a major part of what makes an electronic document management system so valuable to a business or organization.</code>                                                                                                                                                                                                                                                                                                                                                                       | <code>Benefits derived from DoD EDA include: 1  Single-source, timely information. 2  Electronic search and retrieval – 24/7 access/retrieval capability. 3  Increased visibility of all procurement & payment actions.  Reduction in data entry/human 1  error. Lower postage, handling, retention and document management costs.</code>                                                                                                                                                                                                                                                                                                                                                                                                 | <code>If YES, go to www.docusign.net and log in with your email and password. On the DocuSign Web Application, select the Documents tab. Your documents are listed there. If NO, you can access the document by opening the DocuSign Completed email. This email is sent to you once you have finished signing a DocuSign document. See the instructions below. Note: In some cases, your documents might be attached to the Completed email. 1. Open the DocuSign Completed email.</code>                 | <code>[4.681269645690918, 9.322907447814941, 14.813400268554688, 20.356698989868164]</code> |
+  | <code>does hpv cause uti</code>                  | <code>So now you get in the acidic environment can hpv cause urinary tract infection for the area of the blockage of the fruits and fiber as a completely eliminate urinate at all. Spending money on prescription of antibiotics will kill all of the bacterial infection keeps happening to your veterinarian will work to cure the condition.</code>                                                                                                                                                                                                         | <code>HPV & Urinary Tract Infections. Human Papillomavirus (HPV) is a group of viruses that can cause warts and cancers of the cervix, anus and genitals. Urinary tract infection (UTI) occurs when bacteria multiply within the bladder, causing pain and urinary urgency. (Thomas Northcut/Digital Vision/Getty Images) Other People Are Reading.</code>                                                                                                                                                                                          | <code>Some types of the HPV virus can infect the genital epithelial cells (skin and mucous membranes). Some types of HPV virus cause warts that appear on the genitals (vagina, vulva, penis, etc.) and anus of women and men.</code>                                                                                                                                                                                                                                                                                                                                         | <code>Most women with HPV have no signs of infection. Since most HPV infections go away on their own within two years, many women never know they had an infection. Some HPV infections cause genital warts that can be seen or felt. The only way to know if you have HPV is to ask your health care provider to do an HPV test.</code>                                                                                                                                                                                                                                                                                                                                                                                                  | <code>Genital warts are caused by low-risk types of human papillomavirus (HPV). These viruses may not cause warts in everyone. Women can get genital warts from sexual contact with someone who has HPV. Genital warts are spread by skin-to-skin contact, usually from contact with the warts. It can be spread by vaginal, anal, oral, or handgenital sexual contact. Genital warts will spread HPV while visible, and after recent treatment.</code>                                                    | <code>[0.5, 2.4958395957946777, 3.76273775100708, 4.114340305328369]</code>                 |
 * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
   ```json
   {
-      "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')",
-      "document_regularizer_weight": 0.003,
-      "query_regularizer_weight": 0.005
+      "loss": "SparseMarginMSELoss",
+      "document_regularizer_weight": 0.3,
+      "query_regularizer_weight": 0.5
   }
   ```
 
@@ -294,18 +303,15 @@ You can finetune this model on your own dataset.
 #### Non-Default Hyperparameters
 
 - `eval_strategy`: epoch
-- `per_device_train_batch_size`: 16
-- `per_device_eval_batch_size`: 16
-- `gradient_accumulation_steps`: 4
-- `learning_rate`: 6e-05
+- `per_device_train_batch_size`: 48
+- `per_device_eval_batch_size`: 48
+- `learning_rate`: 8e-05
 - `num_train_epochs`: 6
 - `lr_scheduler_type`: cosine
 - `warmup_ratio`: 0.025
 - `fp16`: True
 - `load_best_model_at_end`: True
 - `optim`: adamw_torch_fused
-- `push_to_hub`: True
-- `batch_sampler`: no_duplicates
 
 #### All Hyperparameters
 <details><summary>Click to expand</summary>
@@ -314,14 +320,14 @@ You can finetune this model on your own dataset.
 - `do_predict`: False
 - `eval_strategy`: epoch
 - `prediction_loss_only`: True
-- `per_device_train_batch_size`: 16
-- `per_device_eval_batch_size`: 16
+- `per_device_train_batch_size`: 48
+- `per_device_eval_batch_size`: 48
 - `per_gpu_train_batch_size`: None
 - `per_gpu_eval_batch_size`: None
-- `gradient_accumulation_steps`: 4
+- `gradient_accumulation_steps`: 1
 - `eval_accumulation_steps`: None
 - `torch_empty_cache_steps`: None
-- `learning_rate`: 6e-05
+- `learning_rate`: 8e-05
 - `weight_decay`: 0.0
 - `adam_beta1`: 0.9
 - `adam_beta2`: 0.999
@@ -388,7 +394,7 @@ You can finetune this model on your own dataset.
 - `dataloader_persistent_workers`: False
 - `skip_memory_metrics`: True
 - `use_legacy_prediction_loop`: False
-- `push_to_hub`: True
+- `push_to_hub`: False
 - `resume_from_checkpoint`: None
 - `hub_model_id`: None
 - `hub_strategy`: every_save
@@ -423,7 +429,7 @@ You can finetune this model on your own dataset.
 - `eval_use_gather_object`: False
 - `average_tokens_across_devices`: False
 - `prompts`: None
-- `batch_sampler`: no_duplicates
+- `batch_sampler`: batch_sampler
 - `multi_dataset_batch_sampler`: proportional
 - `router_mapping`: {}
 - `learning_rate_mapping`: {}
@@ -433,23 +439,23 @@ You can finetune this model on your own dataset.
 ### Training Logs
 | Epoch   | Step      | Training Loss | dot_ndcg@10 |
 |:-------:|:---------:|:-------------:|:-----------:|
-| 1.0     | 12500     | 11.5771       | 0.6587      |
-| 2.0     | 25000     | 0.7888        | 0.6810      |
-| 3.0     | 37500     | 0.7271        | 0.6884      |
-| 4.0     | 50000     | 0.6774        | 0.6920      |
-| 5.0     | 62500     | 0.6436        | 0.6912      |
-| **6.0** | **75000** | **0.6274**    | **0.6932**  |
+| 1.0     | 10336     | 16309.8824    | 0.6698      |
+| 2.0     | 20672     | 14.4047       | 0.6920      |
+| 3.0     | 31008     | 13.0742       | 0.7004      |
+| 4.0     | 41344     | 11.8023       | 0.7060      |
+| 5.0     | 51680     | 11.0464       | 0.7085      |
+| **6.0** | **62016** | **10.6766**   | **0.7094**  |
 
 * The bold row denotes the saved checkpoint.
 
 ### Framework Versions
-- Python: 3.11.11
+- Python: 3.11.13
 - Sentence Transformers: 5.0.0
-- Transformers: 4.53.1
+- Transformers: 4.53.2
 - PyTorch: 2.6.0+cu124
-- Accelerate: 1.5.2
-- Datasets: 3.6.0
-- Tokenizers: 0.21.1
+- Accelerate: 1.8.1
+- Datasets: 4.0.0
+- Tokenizers: 0.21.2
 
 ## Citation
 
@@ -481,15 +487,15 @@ You can finetune this model on your own dataset.
 }
 ```
 
-#### SparseMultipleNegativesRankingLoss
+#### SparseMarginMSELoss
 ```bibtex
-@misc{henderson2017efficient,
-    title={Efficient Natural Language Response Suggestion for Smart Reply},
-    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
-    year={2017},
-    eprint={1705.00652},
+@misc{hofstätter2021improving,
+    title={Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation},
+    author={Sebastian Hofstätter and Sophia Althammer and Michael Schröder and Mete Sertkan and Allan Hanbury},
+    year={2021},
+    eprint={2010.02666},
     archivePrefix={arXiv},
-    primaryClass={cs.CL}
+    primaryClass={cs.IR}
 }
 ```
 
@@ -519,4 +525,4 @@ You can finetune this model on your own dataset.
 ## Model Card Contact
 
 *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--->
+--> 

config.json

@@ -17,7 +17,7 @@
   "pad_token_id": 0,
   "position_embedding_type": "absolute",
   "torch_dtype": "float32",
-  "transformers_version": "4.53.1",
+  "transformers_version": "4.53.2",
   "type_vocab_size": 2,
   "use_cache": true,
   "vocab_size": 30522

config_sentence_transformers.json

@@ -2,7 +2,7 @@
   "model_type": "SparseEncoder",
   "__version__": {
     "sentence_transformers": "5.0.0",
-    "transformers": "4.53.1",
+    "transformers": "4.53.2",
     "pytorch": "2.6.0+cu124"
   },
   "prompts": {

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6b234281fbc722918304ccf3e74f1b216cc2deb2c3197034a6bf90f2a6b7e3fe
+oid sha256:c5578e5c58d8ff1c071f9ef9a555c2694c08a5b4c196697e4e199218dcc64ff0
 size 17671560