Hugging Face's logo

RK-1235
/
bge-base-FIR-matryoshka-BASELINE-10epo-triplet

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
Generated from Trainer
dataset_size:801
loss:MultipleNegativesRankingLoss
Eval Results
text-embeddings-inference
Model card Files Community
bge-base-FIR-matryoshka-BASELINE-10epo-triplet / README.md
RK-1235's picture
RK-1235
Add new SentenceTransformer model
795ee69 verified
preview code
|
raw
history blame contribute delete
83.1 kB
metadata
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:801
  - loss:MultipleNegativesRankingLoss
base_model: BAAI/bge-base-en
widget:
  - source_sentence: >-
      Who signed the case report and provided their signature at the end of the
      document?
    sentences:
      - >-
        First Information Report: The arrest of 30 quarts of country liquor
        occurred under Section 60 of the Excise Act by a team at Nawabad Police
        Station, Jhansi district. Today on the date 24/02/24, I, Mr. Brajesh
        Kumar Bhargava, along with Constable Lokendra Singh, filed report number
        31 at 10:04 AM. We were patrolling the Binavar Care Area when we
        received special intelligence from the senior officer that an individual
        carrying white liquor was standing near the toilet behind the bus stand
        at Haveli Market. If acted quickly, he could be apprehended. Following
        the senior officer's information, I, Mr. May Hamrahi, accompanied by
        Constable Lokendra Singh, proceeded towards the bus stand in Ambedkar
        Nagar and moved towards the toilet inside Tala Pura. The senior officer
        signaled from a distance and returned. About 15 steps before the toilet,
        a person was seen standing with a white-colored bottle in his hand. As
        soon as the police approached him, he tried to run away but was caught
        on the
      - >-
        upon reaching the police station. Signatures: H/O Upendra Signature
        Navratan Singh U/N Signature Bhim Singh Signature English Unreadable SI
        18/3/24 (Navin Tomar) U/N Police Station No. 113 Gautambuddhanagar Note:
        A copy of the case was given to the accused Mizruul above. Fingerprint
        Accused LTI Mizruul Signature English Unreadable SI 18/3/24 (Navin
        Tomar) U/N Police Station No. 113 Gautambuddhanagar 5 NC R.B (N.C.R.B)
        I.L.F.-I (Unified Investigation Form -1)
      - I.l.F.-I (Unified Investigation Form -1) verified and found true.
  - source_sentence: Who prepared and signed the arrest report?
    sentences:
      - >-
        First Information Report: Counterfeit letter service officer Shri
        Prabhari Inspector Sir Thana Khesra District Si. City Sir, respectfully
        informed that I am the petitioner Anurag Shukla SLO Raakesh Shukla
        village Mirwapur PO Beluakh Si. city resident today dated 28/02/24 my
        father Shri Raakesh Shukla Marvatia Market was returning home on a
        motorbike when around 5 PM in Basdia village near Pinto's tea shop as
        they reached there an old Ranjis regarding Raju alias Tira son
        Ramchandra alias Kallu Ajay son Avadhash Sanee Devl son Rajesh and some
        other people gathered together started beating my father with abuses
        calling him mother sister with sticks and slaps which caused my father
        to fall injured some people intervened and saved my father then the
        above-mentioned people left after threatening to kill my father we took
        our father to CHC Beloha where the doctor referred him to Naugadh due to
        severe injuries where his treatment is ongoing. Therefore, it is
        requested to Shri Man Ji to take
      - >-
        First Information Report: Counterfeit currency—seizure and search
        operation. Cash seizure of 380 rupees, one pen, one person arrested,
        charged under Section 13 G ACT, Thana Sadar Bazaar, Shahjahanpur
        District, today's date 20/3/2024. Officer ID 195, Devendra Kumar, along
        with colleague Officer ID 1774, Somveer, accompanied by informant, left
        from Haja police station and stopped at Ashfaqnagar area for routine
        checks, crime prevention, and peacekeeping measures. When the police
        approached near Baliji Temple, they received special intelligence from
        an informant that a person standing near Chamakni Police Station was
        involved in counterfeit currency transactions and could be caught
        quickly. The police believed the informant and conducted searches on
        each other to ensure no one had criminal material. Accompanied by the
        informant, they proceeded to the location indicated by the informant.
        Upon reaching Sharab Bhatti Sahibaznagar Road, the informant pointed out
        from a distance that someone
      - >-
        and giving testimony, and they left without providing their names due to
        fear. Throughout the entire process, orders and guidelines of the
        Honorable Supreme Court and the National Human Rights Commission were
        fully followed. The arrest notice will be given to the concerned parties
        through appropriate means at the police station. This report was
        prepared by speaking and writing by me, Pradeep, and written by Ka 1952
        Susant, read out and signed by all concerned
  - source_sentence: >-
      Where did the accident involving Ramesh Singh Kushwaha occur relative to
      the police station?
    sentences:
      - >-
        First Information Report: The complaint is in Hindi. Mr. Thana Adhyaksh
        Mahodaya Thana Birnora Janpad Gajipur Mahodaya is respectfully requested
        that the applicant Rajkumar Singh Kushwaha son of Chandrajit Singh
        Kushwaha, originally from village Araji Odasun, Thana Birnora, Janpad
        Gajipur, on today's date 12.03.2024, around 7:30 AM, my brother Ramesh
        Singh son of Chandrajit was going to check the fields from Gajipur Road
        when, about 200 meters from the police station, near Singh Kushwaha's
        house, in the western direction towards Ajmer, vehicle number UPSOABSO99
        carelessly hit Ramesh Singh Kushwaha's rider from behind with great
        force, resulting in my brother Ramesh Singh Kushwaha dying on the spot.
        Therefore, it is requested that Mr. Ji kindly write a report (register)
        and take necessary action. Applicant's signature Rajkumar Singh Kushwaha
        son of Chandrajit Singh Kushwaha, village Araji Odasun, Thana Birnora,
        Janpad Gajipur, mobile 9532127579, date 12.03.2024. Note: I, Kanu Amit
        Kumar
      - Kumar Praja Pit 4 NC RB (NCRB) I.l.F.- (Unified Investigation Form-l)
      - >-
        attached with screws. The hammer is made of iron, the total length being
        2 cm (1 inch). The trigger guard is made of iron with a screw for
        opening and closing attached to an iron plate on the other side. The
        seized firearm is a brass cartridge marked KFSMM on its base. This act
        falls under the chargeable offense under Section 3/25 A Act. At around
        10:10 am, while informing about the arrest, the orders and guidelines of
        the Human Rights Commission and the Supreme Court were followed, and
        Himanshu Singh son of Veerendra Singh, residing in Paradhan, Kotwali
        town, Maun district, was taken into custody under Section 3/25 A Act.
        One motorcycle vehicle numbered UPS4 AB7344 was seized from the scene,
        and when asked for registration, he could not show any, which is an
        offense under 207 M.V. Act. A report has been filed under 207 MV Act.
        The seized illegal firearm and cartridge were placed in a transparent
        plastic box brought by Constable Ka0 Manu Kumar and sealed with a sample
        seal. The case
  - source_sentence: How much money was given to Rajkumar Kanaujia on February 2, 2019?
    sentences:
      - >-
        cost Rupees 5 lakhs. Thus, under the pretext of getting the applicant
        and his brother Mo Arju's government jobs in the Panchayat Raj
        Department, a total of Rupees 10 lakhs was given to the aforementioned
        Rajkumar Kanaujia. The details are as follows: On February 2, 2019,
        Rupees 10,000 cash; on February 6, 2019, Rupees 40,000 cash; on August
        17, 2019, Rupees 1,00,000 cash; on October 17, 2019, Rupees 49,000 cash;
        on July 23, 2020, Rupees 1,00,000 cash; on March 8, 2021, Rupees 50,000
        cash; on March 26, 2021, Rupees 50,000 cash. The applicant gave this
        money directly to Rajkumar Kanaujia and withdrew from his
        sister-in-law's account. On February 27, 2020, Rupees 22,000 cash; on
        September 18, 2020, through NEFT, Rupees 1,00,000/-. Check No. 908768,
        Total Rupees 5,21,000/-. On October 22, 2021, Rupees 1,00,000 was given
        to Puran's wife Sony, who was sent by Rajkumar Kanaujia. The remaining
        Rupees 3,79,000/- was given in cash by the applicant to Rajkumar
        Kanaujia after borrowing from
      - >-
        First Information Report: The complaint is in Hindi. Mr. Thana Adhyaksh
        Mahodaya Thana Birnora Janpad Gajipur Mahodaya is respectfully requested
        that the applicant Rajkumar Singh Kushwaha son of Chandrajit Singh
        Kushwaha, originally from village Araji Odasun, Thana Birnora, Janpad
        Gajipur, on today's date 12.03.2024, around 7:30 AM, my brother Ramesh
        Singh son of Chandrajit was going to check the fields from Gajipur Road
        when, about 200 meters from the police station, near Singh Kushwaha's
        house, in the western direction towards Ajmer, vehicle number UPSOABSO99
        carelessly hit Ramesh Singh Kushwaha's rider from behind with great
        force, resulting in my brother Ramesh Singh Kushwaha dying on the spot.
        Therefore, it is requested that Mr. Ji kindly write a report (register)
        and take necessary action. Applicant's signature Rajkumar Singh Kushwaha
        son of Chandrajit Singh Kushwaha, village Araji Odasun, Thana Birnora,
        Janpad Gajipur, mobile 9532127579, date 12.03.2024. Note: I, Kanu Amit
        Kumar
      - >-
        lid was opened. When asked for a license to carry raw liquor, he could
        not show it and requested forgiveness for his crime. The accused was
        taken to the custody of the police around 16:35. The lid of the seized
        bottle was sealed and stamped according to the Supreme Court order and
        instructions. No witnesses were available due to various reasons, and
        they left without giving their names and addresses. The incident report
        was written by me, Constable Shivdatt, and signed by my colleague
        Constable Somveer. The arrest information will be communicated to the
        public through appropriate channels upon reaching the police station.
        The signature is illegible in English by Constable Shivdatt, serial
        number 2175, Shahjahanpur Sadar Bazaar Police Station, dated 23.01.2024.
        The signature is legible in Hindi by Constable Somveer. A carbon copy of
        the report was given to the accused for acknowledgment. Note: I,
        Amarjeet Kumar, the main writer, certify that the copy of the report has
        been fed word by
  - source_sentence: Under which section of the law was Rahul Shishupal apprehended?
    sentences:
      - >-
        Mann son Saudan residing Kanhor Thana Pahadi Bharatpur Rajasthan, 5.
        Mustak son Hariya residing Ghagwadi Thana Kathwada Bharatpur Rajasthan,
        6. Ikram son Iliaas residing Jamat Thana Punhana Noonu Mevata Haryana,
        7. Nasir son Raju residing Bisru Thana Punhana Noonu Mevata Haryana, 8.
        Sahun son Rahmuddin residing Dhaulpur Thana Dhaulpur Faridabad Haryana,
        9. Hakku alias Hakmu alias Hakmuddin son Nasru alias Nasrudeen residing
        Bichhor Thana Bichhor Noonu Mevata Haryana, 10. Kala alias Kala Minister
        son Kanura alias Kalua residing Kanhor Thana Pahadi Bharatpur Rajasthan
        are active members. The main activity of this gang is robbery, looting,
        and illegal smuggling which involves illegal money making, earning money
        to support themselves and their families, causing considerable fear and
        terror in the area. Due to the fear of this gang, no person from the
        public is willing to give testimony or write reports against them. These
        are habitual criminals described under Chapter 17 of the IPC. It is
      - >-
        2023, Mr. Chakrapani Katiyar son of Ishwardayal Katiyar Balaji Hospital
        and SCAASTAKA Diagnostic Center Hardoi reported that the gang leader and
        gang members had created a fake web portal and spread false news through
        various news channels to the public and demanded 50,000 rupees per month
        from hospital staff and forcibly took 5,000 rupees from hospital staff
        on May 25, 2023. A case FIR No. 360/2023 under sections 34/386/506 IPC
        was registered at Kotwali City Police Station. After investigation,
        based on evidence, a charge sheet was sent to the court against the gang
        leader and members which is pending in the court. On May 28, 2023, Mr.
        Sudhanshu Mishra son of OP Mishra residing Bahra Saudagar Lucknow Road
        Thana Kotwali City Hardoi reported that he was beaten, abused, and
        threatened with death and demanded 5,000 rupees by the gang leader DD
        Shukla and gang members Mohit Mishra, Ankrit Mishra, and their other
        accomplices. A case FIR No. 35
      - >-
        raw liquor near the temple in the forest. The informant then left. We
        hid our two motorcycles and approached the person stealthily. When we
        reached the person, we noticed a large crowd around him. Seeing us, the
        crowd dispersed. We apprehended the person selling liquor with a show of
        force. After questioning the apprehended individual about his name and
        address and conducting a search, he identified himself as Rahul
        Shishupal, aged approximately 28 years, from Nanakganj Jhala village,
        Kotwali Police Station, Hardeo. During the search, a plastic bottle
        containing approximately 10 liters of raw liquor was seized. Opening the
        lid of the bottle, we smelled a pungent odor of raw liquor. The act of
        the apprehended individual falls under Section 60 ex.act. The accused
        was informed of his offense under Section 60 ex.act. When asked for
        authorization documents regarding the possession and sale of illegal raw
        liquor, he was unable to produce them and requested forgiveness for his
        mistake. The
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
model-index:
  - name: SentenceTransformer based on BAAI/bge-base-en
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 1024
          type: dim_1024
        metrics:
          - type: cosine_accuracy@1
            value: 0.3888888888888889
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.6823529411764706
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.7300653594771241
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7973856209150327
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.3888888888888889
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.22745098039215686
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1460130718954248
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07973856209150326
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.3888888888888889
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.6823529411764706
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.7300653594771241
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.7973856209150327
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.6081898113831302
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.5461826953003428
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.5512418200825733
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.3888888888888889
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.6823529411764706
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.7300653594771241
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7973856209150327
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.3888888888888889
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.22745098039215686
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1460130718954248
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07973856209150326
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.3888888888888889
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.6823529411764706
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.7300653594771241
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.7973856209150327
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.6081898113831302
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.5461826953003428
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.5512418200825733
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.3843137254901961
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.6718954248366014
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.7287581699346405
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7875816993464052
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.3843137254901961
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2239651416122004
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.14575163398692811
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.0787581699346405
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.3843137254901961
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.6718954248366014
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.7287581699346405
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.7875816993464052
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.6013957196390363
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.5402334267040153
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.5455156504447409
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.3718954248366013
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.6431372549019608
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.7026143790849673
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7679738562091504
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.3718954248366013
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.21437908496732025
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.14052287581699346
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07679738562091502
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.3718954248366013
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.6431372549019608
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.7026143790849673
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.7679738562091504
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5813509127848285
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.5204315800394235
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.5256057776792838
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy@1
            value: 0.3366013071895425
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5915032679738562
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6601307189542484
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7169934640522876
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.3366013071895425
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.1971677559912854
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.13202614379084965
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07169934640522875
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.3366013071895425
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5915032679738562
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.6601307189542484
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.7169934640522876
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5365811801879968
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.47775236020334133
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.4842596807377994
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy@1
            value: 0.2718954248366013
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.4849673202614379
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.5496732026143791
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.6209150326797386
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.2718954248366013
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.1616557734204793
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.10993464052287581
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.06209150326797386
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.2718954248366013
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.4849673202614379
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.5496732026143791
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.6209150326797386
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4494736762212422
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.39436922917315087
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.4024049717422007
            name: Cosine Map@100

SentenceTransformer based on BAAI/bge-base-en

This is a sentence-transformers model finetuned from BAAI/bge-base-en on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: BAAI/bge-base-en
  • Maximum Sequence Length: 512 tokens
  • Output Dimensionality: 768 dimensions
  • Similarity Function: Cosine Similarity
  • Training Dataset:
    • json

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("RK-1235/bge-base-FIR-matryoshka-BASELINE-10epo-triplet")
# Run inference
sentences = [
    'Under which section of the law was Rahul Shishupal apprehended?',
    'raw liquor near the temple in the forest. The informant then left. We hid our two motorcycles and approached the person stealthily. When we reached the person, we noticed a large crowd around him. Seeing us, the crowd dispersed. We apprehended the person selling liquor with a show of force. After questioning the apprehended individual about his name and address and conducting a search, he identified himself as Rahul Shishupal, aged approximately 28 years, from Nanakganj Jhala village, Kotwali Police Station, Hardeo. During the search, a plastic bottle containing approximately 10 liters of raw liquor was seized. Opening the lid of the bottle, we smelled a pungent odor of raw liquor. The act of the apprehended individual falls under Section 60 ex.act. The accused was informed of his offense under Section 60 ex.act. When asked for authorization documents regarding the possession and sale of illegal raw liquor, he was unable to produce them and requested forgiveness for his mistake. The',
    'Mann son Saudan residing Kanhor Thana Pahadi Bharatpur Rajasthan, 5. Mustak son Hariya residing Ghagwadi Thana Kathwada Bharatpur Rajasthan, 6. Ikram son Iliaas residing Jamat Thana Punhana Noonu Mevata Haryana, 7. Nasir son Raju residing Bisru Thana Punhana Noonu Mevata Haryana, 8. Sahun son Rahmuddin residing Dhaulpur Thana Dhaulpur Faridabad Haryana, 9. Hakku alias Hakmu alias Hakmuddin son Nasru alias Nasrudeen residing Bichhor Thana Bichhor Noonu Mevata Haryana, 10. Kala alias Kala Minister son Kanura alias Kalua residing Kanhor Thana Pahadi Bharatpur Rajasthan are active members. The main activity of this gang is robbery, looting, and illegal smuggling which involves illegal money making, earning money to support themselves and their families, causing considerable fear and terror in the area. Due to the fear of this gang, no person from the public is willing to give testimony or write reports against them. These are habitual criminals described under Chapter 17 of the IPC. It is',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Metric Value
cosine_accuracy@1 0.3889
cosine_accuracy@3 0.6824
cosine_accuracy@5 0.7301
cosine_accuracy@10 0.7974
cosine_precision@1 0.3889
cosine_precision@3 0.2275
cosine_precision@5 0.146
cosine_precision@10 0.0797
cosine_recall@1 0.3889
cosine_recall@3 0.6824
cosine_recall@5 0.7301
cosine_recall@10 0.7974
cosine_ndcg@10 0.6082
cosine_mrr@10 0.5462
cosine_map@100 0.5512

Information Retrieval

Metric Value
cosine_accuracy@1 0.3889
cosine_accuracy@3 0.6824
cosine_accuracy@5 0.7301
cosine_accuracy@10 0.7974
cosine_precision@1 0.3889
cosine_precision@3 0.2275
cosine_precision@5 0.146
cosine_precision@10 0.0797
cosine_recall@1 0.3889
cosine_recall@3 0.6824
cosine_recall@5 0.7301
cosine_recall@10 0.7974
cosine_ndcg@10 0.6082
cosine_mrr@10 0.5462
cosine_map@100 0.5512

Information Retrieval

Metric Value
cosine_accuracy@1 0.3843
cosine_accuracy@3 0.6719
cosine_accuracy@5 0.7288
cosine_accuracy@10 0.7876
cosine_precision@1 0.3843
cosine_precision@3 0.224
cosine_precision@5 0.1458
cosine_precision@10 0.0788
cosine_recall@1 0.3843
cosine_recall@3 0.6719
cosine_recall@5 0.7288
cosine_recall@10 0.7876
cosine_ndcg@10 0.6014
cosine_mrr@10 0.5402
cosine_map@100 0.5455

Information Retrieval

Metric Value
cosine_accuracy@1 0.3719
cosine_accuracy@3 0.6431
cosine_accuracy@5 0.7026
cosine_accuracy@10 0.768
cosine_precision@1 0.3719
cosine_precision@3 0.2144
cosine_precision@5 0.1405
cosine_precision@10 0.0768
cosine_recall@1 0.3719
cosine_recall@3 0.6431
cosine_recall@5 0.7026
cosine_recall@10 0.768
cosine_ndcg@10 0.5814
cosine_mrr@10 0.5204
cosine_map@100 0.5256

Information Retrieval

Metric Value
cosine_accuracy@1 0.3366
cosine_accuracy@3 0.5915
cosine_accuracy@5 0.6601
cosine_accuracy@10 0.717
cosine_precision@1 0.3366
cosine_precision@3 0.1972
cosine_precision@5 0.132
cosine_precision@10 0.0717
cosine_recall@1 0.3366
cosine_recall@3 0.5915
cosine_recall@5 0.6601
cosine_recall@10 0.717
cosine_ndcg@10 0.5366
cosine_mrr@10 0.4778
cosine_map@100 0.4843

Information Retrieval

Metric Value
cosine_accuracy@1 0.2719
cosine_accuracy@3 0.485
cosine_accuracy@5 0.5497
cosine_accuracy@10 0.6209
cosine_precision@1 0.2719
cosine_precision@3 0.1617
cosine_precision@5 0.1099
cosine_precision@10 0.0621
cosine_recall@1 0.2719
cosine_recall@3 0.485
cosine_recall@5 0.5497
cosine_recall@10 0.6209
cosine_ndcg@10 0.4495
cosine_mrr@10 0.3944
cosine_map@100 0.4024

Training Details

Training Dataset

json

  • Dataset: json
  • Size: 801 training samples
  • Columns: anchor, positive, and negative
  • Approximate statistics based on the first 801 samples:
    anchor positive negative
    type string string list
    details
    • min: 9 tokens
    • mean: 19.53 tokens
    • max: 41 tokens
    • min: 16 tokens
    • mean: 197.47 tokens
    • max: 410 tokens
    • size: 5 elements
  • Samples:
    anchor positive negative
    What were the clothing details of the person identified by the special informant near the storage shed? was ready to come forward as a witness. Out of necessity, we police officials searched each other to ensure none of us had any incriminating items. Subsequently, accompanied by the special informant, we proceeded to the location indicated. Upon reaching the storage shed near the crematorium ghat, the special informant pointed out a person wearing black and slate-colored jersey and jeans standing behind the wall of the storage shed. We police officials parked our motorcycles nearby, hid ourselves, and approached the back wall where the person was standing. Seeing us, the individual started running, and we police officials caught him after using necessary force. The arrested person's name and address were asked, and he stated his name as Shahvez son of Mowlai No.Mo., Kakra Khurd Thana Kotwali, Shahjahanpur, age approximately 25 years. A search of his person revealed a homemade indigenous firearm 315 bore hidden in the left pocket of his pants, with a total length of one ballist three ['Bablu alias Umesh Kumar Kushwaha, son of Nathoo Prasad Kushwaha, residing in Gram Pachokhar, Than Ahra, Bandha, was conducted. Opening the white plastic bag held in his right hand revealed marijuana. The weight was measured using an electronic scale borrowed from Constable Vijay Singh in the town of Attara, which showed the weight of the marijuana to be 1 kilogram 500 grams. When asked for a license to keep and sell it, he could not show it. The act of Bablu alias Umesh Kumar Kushwaha as mentioned above constitutes a punishable offense under Section 8/20 NDPS Act. Being informed of the offense, he was taken into custody at the Gram Garganpurawa police station from the Garganpurawa turnoff, about 20 steps from Garganpurawa on the river embankment road. The seized marijuana without license of 1 kilogram 500 grams was taken into police custody. A sample of 50 grams was separated from the seized marijuana, and the remaining 1.45 kilograms of marijuana and 50 grams of sample were kept', 'First Information Report: A DCM vehicle numbered UP19T5287 loaded with large animal bones and two accused under Sections 420/278 IPC, Kotwali police station, Basti district, today on the date 01.09.23, while Inspector Rajendra Yadav was touring the area with officers Akhilesh Yadav and Indrajit Paswan, along with employees of Hamaraa Chitra 18, reached near the Toll Plaza Basti when a DCM vehicle numbered UP19T5287 passed by from Gorakhpur, emitting a strong foul smell. Due to the vehicle slowing down near the toll plaza, the driver was signaled to stop on the left side of the road. The names and addresses of the driver and the person sitting with him were asked. The driver introduced himself as Mintu Kumar, son of Tellu Ram, residing in Aldi Police Station, Shamli district, age 3 NC RB (N.C.R.B) I.L.F.-I (Unified Investigation Form - I), 24 years old. The name and address of the person sitting with the driver were also asked, who introduced himself as Deshpal SIO Krishnapal, residing', '(Unified Investigation Form -1), Purva Case, Tikra Village, Asandra Police Station, Barabanki District. A search of his person revealed a plastic bottle filled with liquid substance in his right hand. Opening the cap of the bottle, we smelled a strong odor of raw liquor. When asked for a license to carry liquor, he refused to show it and requested forgiveness for his mistake. The above-mentioned person was taken into custody under Section 60(1) of the Act around 6:05 PM. During the arrest, strict adherence was given to the orders and directives of the Human Rights Commission and the Honorable Supreme Court. The bottle cap was resealed, and witnesses were sought from the public without revealing their names. The seized bottle was brought to the police station and sealed with a serial number seal. Notice of arrest will be given to the relative. The statement will be written, read, and heard before being signed by the concerned parties. Premchand Rawat, signature HC. Ali Ahmad, signature', 'police post. Upon questioning the arrested person for name and address, 3 NC R.B (N.C.R.B) I.L.F.-I (Unified Investigation Form - I) was filled, and a body search was conducted. The arrested person stated his name as Sanjay Rajput, son of Ramasare, residing in Jhinjhak Vasti Ward No. 5 Durgais Nagar Police Station Mangalpur District Dehat Kanipalwar, age approximately 35 years. During the body search, 15 units of country liquor were seized from the right hand held plastic bag. When questioned about the illegal country liquor, he stated that he buys liquor from contractors and sells it village to village for more money. When asked for a license to sell, he failed to show one and started asking for forgiveness for the mistake. The act of the arrested person falls under the ambit of Section 60 Excise Act. The arrested person was informed of the charge under Section 60 EXACT.KS and was arrested around 10:22 PM. The illegal country liquor was seized by the police. At the time of arrest,', 'First Information Report: Oral Statement - It was stated that I am SHO Sanjeev Kumar Dubey, there are 20 numbered cartridges with Dal 1443 Dalveer Singh, one each; 20 numbered cartridges with 1504 Sandeep Kumar, one each; driving a government jeep UP 85 AG 0415. Departed from Jayprakash on the day of the incident Imroz, stopped for investigation and check and ATM and the suspect vehicle and search desired criminals and investigation gang chart approved by Shri District Magistrate Mathura, Gang Leader 1. Amir Khan son Rahmat Khan residing Akeda Thana Sadar Noonu Mevata Haryana was informed that people reported in various places that Amir Khan residing Akeda Thana Sadar Noonu Mevata Haryana is part of an organized group, its members are 2. Muslim alias Hakimi son Masood alias Sunder Jamat Thana Punhana Noonu Mevata Haryana, 3. Ajriddin alias Ajru alias Ajju son Azad residing Ghagwadi Thana Kathwada Bharatpur Rajasthan, 4. Manish alias Mann son Saudan residing Kanhor Thana Pahadi']
    How did the police officials manage to catch the individual after he started running? was ready to come forward as a witness. Out of necessity, we police officials searched each other to ensure none of us had any incriminating items. Subsequently, accompanied by the special informant, we proceeded to the location indicated. Upon reaching the storage shed near the crematorium ghat, the special informant pointed out a person wearing black and slate-colored jersey and jeans standing behind the wall of the storage shed. We police officials parked our motorcycles nearby, hid ourselves, and approached the back wall where the person was standing. Seeing us, the individual started running, and we police officials caught him after using necessary force. The arrested person's name and address were asked, and he stated his name as Shahvez son of Mowlai No.Mo., Kakra Khurd Thana Kotwali, Shahjahanpur, age approximately 25 years. A search of his person revealed a homemade indigenous firearm 315 bore hidden in the left pocket of his pants, with a total length of one ballist three ['hid behind motorcycles and approached stealthily, arresting the individual named Sunil, son of Harishchandra, residing in Rudapurwa Majra, Nayagaon, Mubarakpur, Kotwali Police Station, Dehat Hardoi, approximately 36 years old. A search of his person revealed a plastic container with about 14 liters of liquid. Upon questioning, he admitted it was raw alcohol. Opening the container lid, a strong smell of raw alcohol was detected. When asked for a license to sell and possess raw alcohol, he refused to show one and requested forgiveness for his mistake. His actions constituted an offense under Section 60 ExAct. He was informed of the charges and taken into custody at 7:25 PM. The seized container was prepared with a memo, sealed, and stamped. During the arrest, orders and guidelines from the Supreme Court and National Commission were strictly followed. Some people came during the arrest but left without giving their names or addresses after being asked for witness statements. This report', 'First Information Report: This is a handwritten complaint submitted in the Hindi service, addressed to Shri Inspector Sir, Thana Motipur, Bahraich district, Sir, it is stated that the complainant Poonam, wife of Ramabhilakh, residing in Jayarampurwa, Kudwaa, Thana Motipur, Bahraich, am living there. On the date 11.01.2024, at 4 PM, Dr. Urff Ramesh, son of Bala Lodhi, residing in Kaalapurwa, Mo Nagar, Thana Ramgaon, Bahraich district, took away my daughter Nisha, aged about 14 years, daughter of Ramabhilakh. Today, my daughter told me that Dr. Urff Ramesh forcibly took her away. Due to which I have come to the police station with my daughter. Therefore, I request Mr. Sir to kindly write my case and take appropriate action against the accused. Ref No. Poonam 3 N.C.R.B (N.C.R.B) I.L.F.-1 (Unified Investigation Form -1) Complainant Poonam, wife of Ramabhilakh, Village Jayarampurwa, Kudwaa, Thana Motipur, Bahraich, Mobile No. 7052270342 Date 17.01 2024 The registration of the case was done', 'Barabanki, approximately 23 years old. From his body search, a total of 1120 rupees was recovered, including two 500 rupee notes, two 50 rupee notes, and one 20 rupee note. The second individual identified himself as Muhammad Imran, son of Shahid Ali, residing in Dhakauli Thana, Kotwali Nagar district, Barabanki, approximately 25 years old. From his body search, a total of 1030 rupees was recovered, including two 500 rupee notes, one 20 rupee note, and one 10 rupee note. The third individual identified himself as Radhe, son of Sohki Lal, residing in Dhakauli Thana, Kotwali Nagar district, Barabanki, approximately 19 years old. From his body search, a total of 720 rupees was recovered, including one 500 rupee note, two 100 rupee notes, and one 20 rupee note. The fourth individual identified himself as Sanjay, son of Kanaku, residing in Dhakauli Thana, Kotwali Nagar district, Barabanki, approximately 20 years old. From his body search, a total of 750 rupees was recovered, including one', "neck and cheeks with the intention of killing her, which put the petitioner in a critical condition and blood started flowing from her neck. He also attacked Munni Devi, a neighbor of the petitioner. Then Munni Devi started shouting loudly, so he ran downstairs with the knife. Some people downstairs caught him and called the police by dialing 112. Then the police took him to the concerned police station, and the petitioner's treatment is ongoing at Motilal Nehru Medical College where her condition has become serious. The opponent is aggressive and violent and wants to kill the petitioner. If a case is not registered against him under severe sections, an accident can happen to the petitioner at any time. The opponent is a criminal type of person from before, and his brother is also supporting him. Therefore, respectfully requested to Mr. Sir that kindly register a case against the opponent for attempted murder and kindly register the case under severe sections and kindly register the", '(First Information Report): A fake case report of 10 liters of raw liquor and arrest of one person under Section 60 of the Excise Act today on the date 22/3/24 by me, Shivraj Singh, UNI, May, UNI Shivraj Singh May, UNI Shaitan Singh HC 231 Sanjay Kumar from Haja Police Station with report No. 06 at time 1:06 PM was on duty for verification and checking of individuals, vehicles, and investigation in the area. When we reached near Samaspur Gate, we saw a sudden panic when a person holding a plastic bottle suddenly ran away quickly upon seeing us, which made us suspicious. We chased him and caught him near Samaspur Gate about 20 steps away at around 3:00 PM. Upon questioning the arrested individual, his name was Thakur Das, son of Thakur Singh RIO, police station Chatari, Vikroori, district Bulandshahr, age approximately 40 years. He was holding a white plastic can of about 10 liters in his right hand. Opening the lid of the plastic can, we smelled the odor of raw liquor. When asked for a']
    What specific details were provided about the homemade indigenous firearm found on the arrested person? pants, with a total length of one ballist three inches, barrel iron approximately eight inches, stock iron approximately four inches, and hammer iron approximately four inches, with wooden plates screwed on both sides, trigger hammer operational, and a pin with a cap to open and close the barrel with a spring. Additionally, a live cartridge 315 bore was found in his right pocket, with 8 mm KF written on its base. The arrested person complained of pain in the lower abdomen, stating it was due to an injury from falling in the morning. When asked for a license for the firearm and cartridge, Shahvez son of Mowlai could not show it and requested forgiveness for his mistake. Since his act falls under Section 3/25 A. Act, which is a punishable offense, the NAFKAL took him into custody at 12:40 PM, citing the reason. The seized firearm and cartridge were kept in a cloth bag, sealed, and stamped with a sample seal. Arrest and seizure memos were prepared. During the arrest and seizure, the ['attached with screws. The hammer is made of iron, the total length being 2 cm (1 inch). The trigger guard is made of iron with a screw for opening and closing attached to an iron plate on the other side. The seized firearm is a brass cartridge marked KFSMM on its base. This act falls under the chargeable offense under Section 3/25 A Act. At around 10:10 am, while informing about the arrest, the orders and guidelines of the Human Rights Commission and the Supreme Court were followed, and Himanshu Singh son of Veerendra Singh, residing in Paradhan, Kotwali town, Maun district, was taken into custody under Section 3/25 A Act. One motorcycle vehicle numbered UPS4 AB7344 was seized from the scene, and when asked for registration, he could not show any, which is an offense under 207 M.V. Act. A report has been filed under 207 MV Act. The seized illegal firearm and cartridge were placed in a transparent plastic box brought by Constable Ka0 Manu Kumar and sealed with a sample seal. The case', "under Section 4/25 AACT. Therefore, the accused Jitendra Pal was informed of his crime under Section 4/25 AACT and was taken into custody at 23:15 hours at the police station. The seized sword was also taken into custody, wrapped in white cloth, sealed, and a sample seal was prepared. Requests for witness statements were made to those present, but due to various reasons, everyone left without giving their names and addresses. During the arrest and seizure, strict adherence to the orders and instructions of the Supreme Court and the Human Rights Commission was maintained. The accused's arrest was informed to his family members, and the FIR was written by talking to the local residents, including Ankrit Gupta, and read out to them. Handwritten notes in Hindi and English are illegible. The notes were attested by Balveer Singh PNO 062250708 and typed by Sonu on the computer.", "and began pleading for forgiveness. At around 19:15, the apprehended individual was taken into custody under Section 60(1) of the Excise Act. A sample of 01 liter was extracted from the plastic container for examination and placed in a plastic bottle. The container and bottle were sealed with a white cloth and stamped as evidence. An arrest memo was prepared on the spot. During the seizure and arrest, strict adherence to the orders and guidelines of the Supreme Court and the Human Rights Commission was maintained. Due to the urgency of the situation, no public witness was present. The report was written and read out loud by Vivek Kumar in sufficient light from a mobile phone and torch, and all concerned parties were informed. The arrested individual's family will be notified through appropriate means upon arrival at the police station. Note: A copy of the report was given to the accused on the spot but was destroyed. Note: Copy of the original report - H.M. Ajit Kumar
  • Loss: MultipleNegativesRankingLoss with these parameters:
    {
    "scale": 20.0,
    "similarity_fct": "cos_sim"
}

Training Hyperparameters

Non-Default Hyperparameters

  • eval_strategy: epoch
  • per_device_train_batch_size: 32
  • per_device_eval_batch_size: 16
  • gradient_accumulation_steps: 8
  • learning_rate: 2e-05
  • num_train_epochs: 50
  • lr_scheduler_type: cosine
  • warmup_ratio: 0.1
  • bf16: True
  • tf32: True
  • load_best_model_at_end: True
  • optim: adamw_torch_fused
  • batch_sampler: no_duplicates

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: epoch
  • prediction_loss_only: True
  • per_device_train_batch_size: 32
  • per_device_eval_batch_size: 16
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 8
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 2e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 50
  • max_steps: -1
  • lr_scheduler_type: cosine
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.1
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: True
  • fp16: False
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: True
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: True
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch_fused
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: None
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • include_for_metrics: []
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • use_liger_kernel: False
  • eval_use_gather_object: False
  • average_tokens_across_devices: False
  • prompts: None
  • batch_sampler: no_duplicates
  • multi_dataset_batch_sampler: proportional

Training Logs

Epoch Step Training Loss dim_1024_cosine_ndcg@10 dim_768_cosine_ndcg@10 dim_512_cosine_ndcg@10 dim_256_cosine_ndcg@10 dim_128_cosine_ndcg@10 dim_64_cosine_ndcg@10
1.0 4 - 0.5307 0.5307 0.5140 0.4598 0.3836 0.2600
2.0 8 - 0.5500 0.5500 0.5329 0.4736 0.3953 0.2692
2.6154 10 21.2344 - - - - - -
3.0 12 - 0.5777 0.5777 0.5644 0.5024 0.4182 0.2931
4.0 16 - 0.6006 0.6006 0.5897 0.5327 0.4507 0.3240
5.0 20 15.3903 0.6073 0.6073 0.5992 0.5528 0.4884 0.3596
6.0 24 - 0.5985 0.5985 0.5978 0.5574 0.5060 0.3928
7.0 28 - 0.5997 0.5997 0.5919 0.5598 0.5199 0.4097
7.6154 30 11.3886 - - - - - -
8.0 32 - 0.5996 0.5996 0.5934 0.5659 0.5254 0.4136
9.0 36 - 0.6068 0.6068 0.5969 0.5743 0.5326 0.4163
10.0 40 6.435 0.6084 0.6084 0.6003 0.5792 0.5376 0.4281
11.0 44 - 0.6133 0.6133 0.6087 0.5801 0.5446 0.4370
12.0 48 - 0.6207 0.6207 0.6102 0.5898 0.5459 0.4448
12.6154 50 4.2559 - - - - - -
13.0 52 - 0.6285 0.6285 0.6171 0.5947 0.5488 0.4461
14.0 56 - 0.6236 0.6236 0.6111 0.5925 0.5483 0.4556
15.0 60 2.2983 0.6216 0.6216 0.6144 0.5899 0.5492 0.4556
16.0 64 - 0.6177 0.6177 0.6116 0.5883 0.5504 0.4552
17.0 68 - 0.6178 0.6178 0.6095 0.5866 0.5433 0.4508
17.6154 70 1.7678 - - - - - -
18.0 72 - 0.6127 0.6127 0.6030 0.5812 0.5392 0.4491
19.0 76 - 0.6071 0.6071 0.6018 0.5805 0.5381 0.4479
20.0 80 1.1051 0.6082 0.6082 0.5988 0.5781 0.5338 0.4438
21.0 84 - 0.6085 0.6085 0.6001 0.5788 0.5340 0.4448
22.0 88 - 0.6119 0.6119 0.6050 0.5826 0.5396 0.4483
22.6154 90 0.8751 - - - - - -
23.0 92 - 0.6143 0.6143 0.6088 0.5858 0.5413 0.4467
24.0 96 - 0.6156 0.6156 0.6101 0.5886 0.5427 0.4510
25.0 100 0.6431 0.6157 0.6157 0.6103 0.5882 0.5464 0.4530
26.0 104 - 0.6100 0.6100 0.6041 0.5820 0.5407 0.4479
27.0 108 - 0.6074 0.6074 0.6001 0.5814 0.5377 0.4472
27.6154 110 0.6123 - - - - - -
28.0 112 - 0.6069 0.6069 0.6026 0.5797 0.5362 0.4481
29.0 116 - 0.6068 0.6068 0.6006 0.5806 0.5355 0.4447
30.0 120 0.4621 0.6085 0.6085 0.5993 0.5809 0.5348 0.4441
31.0 124 - 0.6076 0.6076 0.5980 0.5791 0.5342 0.4464
32.0 128 - 0.6081 0.6081 0.5946 0.5781 0.5305 0.4424
32.6154 130 0.4917 - - - - - -
33.0 132 - 0.6057 0.6057 0.5966 0.5746 0.5324 0.4436
34.0 136 - 0.6048 0.6048 0.5939 0.5756 0.5307 0.4458
35.0 140 0.3875 0.6059 0.6059 0.5980 0.5776 0.5336 0.4475
36.0 144 - 0.6061 0.6061 0.5966 0.5763 0.5347 0.4465
37.0 148 - 0.6079 0.6079 0.5937 0.5761 0.5316 0.4444
37.6154 150 0.4292 - - - - - -
38.0 152 - 0.6063 0.6063 0.5927 0.5752 0.5298 0.4427
39.0 156 - 0.6078 0.6078 0.5956 0.5756 0.5304 0.4436
40.0 160 0.3469 0.6064 0.6064 0.5950 0.5748 0.5314 0.4426
41.0 164 - 0.6092 0.6092 0.5982 0.5782 0.5305 0.4450
42.0 168 - 0.6084 0.6084 0.5988 0.5783 0.5330 0.4466
42.6154 170 0.3876 - - - - - -
43.0 172 - 0.6084 0.6084 0.6014 0.5791 0.5360 0.4495
44.0 176 - 0.6095 0.6095 0.6008 0.5800 0.5379 0.4489
45.0 180 0.358 0.6102 0.6102 0.6019 0.5809 0.5371 0.4486
46.0 184 - 0.6098 0.6098 0.6009 0.5783 0.5391 0.4479
47.0 188 - 0.6083 0.6083 0.6009 0.5794 0.5369 0.4487
47.6154 190 0.3588 - - - - - -
48.0 192 - 0.6087 0.6087 0.6031 0.5799 0.5358 0.4480
49.0 196 - 0.6100 0.6100 0.6020 0.5802 0.5362 0.4482
50.0 200 0.2932 0.6082 0.6082 0.6014 0.5814 0.5366 0.4495
  • The bold row denotes the saved checkpoint.

Framework Versions

  • Python: 3.10.12
  • Sentence Transformers: 4.1.0
  • Transformers: 4.52.2
  • PyTorch: 2.6.0+cu124
  • Accelerate: 1.7.0
  • Datasets: 3.6.0
  • Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss 

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}