dnth/ssf-retriever-modernbert-embed-base-v4

SentenceTransformer based on nomic-ai/modernbert-embed-base

This is a sentence-transformers model finetuned from nomic-ai/modernbert-embed-base on the ssf-train-valid-v4 dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: nomic-ai/modernbert-embed-base
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • ssf-train-valid-v4

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("dnth/ssf-retriever-modernbert-embed-base-v4")
# Run inference
sentences = [
    'The Reserving Actuarial Executive is responsible for analysing, monitoring and reporting insurance products in the market as well as managing internal product pricing and reinsurance negotiations. He/She conducts periodic reviews of reserving parameters and prepares reporting in adherence to regulatory requirements. He facilitates reinsurance administration to ensure it is completed in an accurate and timely manner. The Reserving Actuarial Executive works with relevant key stakeholders and handling data and information. He possesses strong analytical abilities with a good understanding of actuarial techniques, standards and assumptions. He also has strong knowledge of insurance operations and is able to convey complex information clearly and concisely.',
    'The Reserving Actuarial Executive is tasked with evaluating, overseeing, and communicating insights on insurance products within the marketplace, alongside managing internal pricing strategies and reinsurance agreements. This role involves regularly reviewing reserving assumptions and generating reports compliant with regulatory standards. The executive ensures efficient handling of reinsurance processes, collaborating closely with key stakeholders and managing relevant data. A strong analytical mindset, comprehensive grasp of actuarial methodologies, and familiarity with insurance operations are essential, as is the ability to present complex actuarial information in a clear and succinct manner.',
    'The Reinsurance Underwriting Executive is responsible for assessing and approving reinsurance applications, setting underwriting guidelines, and negotiating terms with external reinsurers. This role focuses on risk evaluation, policy acceptance criteria, and claims review within the insurance industry. The executive collaborates with sales and claims teams to ensure underwriting practices align with company objectives and regulatory requirements. Strong risk assessment skills and expertise in underwriting principles are necessary, along with excellent communication skills to explain underwriting decisions to clients and internal partners.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9130, 0.5998],
#         [0.9130, 1.0000, 0.6592],
#         [0.5998, 0.6592, 1.0000]])

Training Details

Training Dataset

ssf-train-valid-v4

• Dataset: ssf-train-valid-v4 at 7becddc
• Size: 4,524 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 57 tokens mean: 168.91 tokens max: 380 tokens	min: 61 tokens mean: 137.16 tokens max: 271 tokens	min: 45 tokens mean: 107.96 tokens max: 192 tokens

• Samples:

  anchor	positive	negative
  The Store Assistant performs handling, storing and rotating of stock, and is responsible for updating stock levels in the inventory data system. He/She demonstrates awareness of the importance of inventory control and maintains adequate stock levels to avoid overstocking and obsolete or aged lots. He is conversant with the store layout and ensures proper housekeeping. He is expected to adhere to the organisation's standard operating procedures (SOPs), and safety, health and quality systems. He supports in implementation of continuous improvement initiatives in the workplace. He works in a warehouse or store environment and is responsible for the safe and efficient operation of the material handling equipment. He should be systematic, orderly and detail-oriented. He is expected to coordinate work with internal and external stakeholders to accomplish his work.	The Store Assistant is responsible for managing stock by handling, storing, and rotating inventory while accurately updating stock quantities in the inventory management system. This role requires a strong understanding of inventory control principles to maintain optimal stock levels, preventing excess or outdated items. Familiarity with the store layout and commitment to maintaining cleanliness and organization are essential. The Store Assistant follows established standard operating procedures (SOPs) and complies with safety, health, and quality guidelines. They also contribute to continuous improvement efforts within the workplace. Operating within a warehouse or storeroom setting, the role involves safely and efficiently handling material handling equipment. The ideal candidate is methodical, detail-conscious, and coordinates effectively with both internal teams and external partners to achieve operational goals.	The Store Clerk manages customer service activities by assisting shoppers, processing sales transactions, and organizing promotional displays within a retail environment. This role emphasizes strong interpersonal and communication skills to enhance the shopping experience, rather than inventory control. The Store Clerk is responsible for maintaining clean and attractive store aisles and ensuring merchandise is well-presented. They adhere to company policies on customer interaction and sales protocols but are not involved in material handling or warehouse operations. The position requires a friendly demeanor, quick problem-solving abilities, and collaboration with sales associates and suppliers to support store sales targets.
  The Vessel Operations Manager advises on safe and efficient vessel and port operations within the confines of regulatory requirements and/or industry practices. He/She analyses operational data to optimise voyage performance, reduce fuel consumption and turnaround times in ports across a fleet of vessels within a region. He is an excellent communicator, has strong problem-solving and analytical skills, and may be required to work irregular hours.	The Vessel Operations Manager is responsible for overseeing safe and efficient vessel and port activities in compliance with regulatory standards and industry best practices. This role involves analyzing operational metrics to enhance voyage efficiency, minimize fuel usage, and decrease port turnaround durations for a fleet operating within a specific region. The manager must possess strong analytical abilities, effective communication skills, and problem-solving expertise, often requiring flexibility to work beyond standard hours.	The Port Logistics Coordinator manages the scheduling and allocation of cargo shipments within the port terminal, focusing on coordinating warehouse storage and transport logistics. This role emphasizes inventory management, shipment tracking, and collaboration with trucking companies to ensure timely delivery. Strong organizational skills and experience in supply chain software are essential, with standard working hours during weekdays.
  The Senior Engineering Service Engineer/Senior Technical Service Engineer (Aircraft Maintenance) is responsible for provision of technical expertise to optimise engineering solutions for aircraft maintenance works using advanced engineering problem-solving techniques. He/She refines scope of maintenance based on customer requirements, and leads technical and programme reviews with customers and suppliers. He reviews special process control plans, leads inspections and functional checks for conformance of maintenance works to technical specifications and verifies technical reports and documentation. He reviews compliance of aircraft maintenance works with airworthiness and legislative requirements, while proposing enhancements to the organisation's standard operating procedures (SOPs), and safety, health and quality systems. He proactively contributes to the development of lean and sustainability practices, and conducts research and digital innovation in targeted areas for continuous pr...	The Senior Engineering Service Engineer/Senior Technical Service Engineer (Aircraft Maintenance) delivers expert technical guidance to enhance engineering strategies for aircraft upkeep by employing sophisticated problem-solving methods. This role involves tailoring maintenance scopes to meet client specifications and spearheading technical and program evaluations with both customers and suppliers. Responsibilities include assessing special process control plans, directing inspections and functional verifications to ensure maintenance aligns with technical standards, and validating technical documentation. The engineer ensures that aircraft maintenance complies with airworthiness regulations and legal mandates, while recommending improvements to the company’s SOPs and safety, health, and quality frameworks. Additionally, the role supports the advancement of lean methodologies and sustainability initiatives and leads research and digital innovation projects aimed at continuous process o...	The Senior Aircraft Maintenance Planner is responsible for coordinating and scheduling aircraft maintenance activities to ensure timely completion and minimal operational disruption. This role focuses on logistics management, resource allocation, and liaising with vendors to secure parts and services. The planner develops maintenance timelines based on regulatory requirements and operational priorities but does not engage in technical inspections or engineering problem-solving. Instead, emphasis is placed on administrative oversight, compliance documentation, and cost control measures. The role involves frequent collaboration with operations and supply chain teams and requires strong organizational and communication skills rather than deep technical expertise. The Senior Aircraft Maintenance Planner works mainly in an office environment, managing workflows and reporting on maintenance progress to senior management.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

ssf-train-valid-v4

• Dataset: ssf-train-valid-v4 at 7becddc
• Size: 1,131 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 58 tokens mean: 169.54 tokens max: 403 tokens	min: 60 tokens mean: 138.68 tokens max: 268 tokens	min: 53 tokens mean: 108.4 tokens max: 196 tokens

• Samples:

  anchor	positive	negative
  The Baker is responsible for making a wide range of baked goods. He/She prepares ingredients needed for daily production of baked goods, operates baking equipment and tools during food preparation, and guides finishing touches on products. He also supports staffs compliance with hygiene, safety and other standards, and performs food and beverage operational tasks. He is expected to execute continuous improvement activities within the workstation, and follow customer service standards to escalate feedback on service challenges through proper escalation channels. Meticulous and service-oriented, he works independently with minimal supervision, and performs as a team player in both a kitchen and commercial setting. He should be comfortable working in hot and noisy environments and meeting strict production schedules.	The Baker is tasked with producing a diverse assortment of baked items by accurately preparing ingredients and skillfully using baking tools and machinery. This role involves overseeing the final presentation of products, ensuring adherence to hygiene and safety protocols, and contributing to food and beverage operations. The Baker actively participates in continuous process enhancements within their work area and upholds customer service standards by appropriately reporting any service-related issues. Demonstrating precision and a customer-focused approach, the Baker operates autonomously with limited oversight while collaborating effectively within a team environment. The position requires comfort with working in warm, noisy conditions and the ability to meet demanding production deadlines.	The Baker is responsible for managing inventory control and supply chain logistics within a food manufacturing environment. This role focuses on coordinating material procurement, optimizing storage solutions, and supervising distribution schedules to ensure timely delivery of raw ingredients. The Baker must possess strong organizational and analytical skills, maintain compliance with regulatory standards related to inventory management, and lead a team of warehouse staff. Operating primarily in an office and warehouse setting, the role demands proficiency in inventory software and the ability to work under pressure to meet operational targets.
  The Water Quality Control Principal Chemist provides technical leadership on the daily operations of the laboratory and resolves water quality issues based on investigations conducted. He/She oversees the management of the inventory of resources such as de-mineralised water and chemicals, formulates remedial actions based on review of the laboratory test equipment troubleshooting, and reviews regulatory compliance reports and other prepared documentation to perform trending of laboratory test results and implement follow-up actions. He reviews water quality control laboratory policies and processes and improves Standard Operating Procedures (SOPs) to uphold high water quality standards. He leads technical capability development programmes, including on-the-job training and coaching, and formulates the technical training and development plans for the teams. He manages the Permits-to-Work for the team, and establishes Safe System of Work (SSoW) frameworks and practices for his area of wo...	The Water Quality Control Principal Chemist leads the laboratory’s daily functions by providing expert guidance on water quality investigations and problem resolution. This role includes overseeing resource inventory management, such as chemicals and de-mineralised water, and developing corrective measures following equipment troubleshooting analyses. The chemist reviews and enhances laboratory policies and SOPs to maintain superior water quality standards. Additionally, they spearhead the technical skill development of team members through training programs and coaching, while managing safety procedures including Permits-to-Work and Safe Systems of Work. In emergency scenarios, the chemist implements crisis response and recovery plans. Operating within a power plant’s water quality control laboratory, typically located in Singapore’s industrial zones, the role demands strong leadership and analytical capabilities to continuously optimize laboratory and plant water quality operations.	The Water Quality Control Principal Chemist focuses on managing the technical aspects of environmental compliance audits within the water treatment industry. This position primarily involves conducting regulatory inspections, preparing compliance documentation, and coordinating with external regulatory bodies to ensure adherence to environmental laws. The role requires expertise in environmental policy analysis rather than laboratory operations and hands-on water quality testing. The chemist also leads strategic initiatives for corporate sustainability reporting and oversees team operations across multiple sites, emphasizing administrative management over direct technical leadership or training within a laboratory setting.
  The Waste Process Engineer is responsible for designing, optimising and managing processes and systems for the efficient handling, treatment, transformation and disposal of waste, including electronic waste (e-waste) and plastic waste, for an organisation. He/She is focused on minimising waste generation, exploring new technologies for enhancing waste management efficiency and material recovery, and recommending improved waste management systems and processes within an organisation. Based on life-cycle analyses and evaluation of current waste streams, systems and waste-to-resource initiatives, he will develop and implement new processes, ensuring compliance with environmental regulations. The Waste Process Engineer must be meticulous, with an eye for detail and have strong analytical and research skills to stay up to date on best practices and circular economy strategies related to waste management. He is also expected to work collaboratively with cross-functional teams to promote su...	The Waste Process Engineer is tasked with developing, refining, and overseeing systems and procedures to ensure effective waste handling, treatment, and disposal, including managing electronic and plastic waste streams. This role emphasizes reducing waste output, adopting innovative technologies to improve waste processing and resource recovery, and advising on enhanced waste management strategies within the company. Utilizing life-cycle assessments and analyzing existing waste flows and recycling initiatives, the engineer implements new processes that comply with environmental standards. The Waste Process Engineer must possess strong analytical and investigative abilities, attention to detail, and collaborate across departments to advance sustainable practices aligned with circular economy principles.	The Environmental Compliance Officer is responsible for monitoring and enforcing adherence to environmental laws and regulations within the manufacturing industry. This role involves conducting audits, preparing compliance reports, and liaising with regulatory bodies to ensure that company operations meet all legal environmental requirements. The officer must have excellent communication skills, in-depth knowledge of environmental policies, and the ability to train staff on compliance matters. Unlike process engineering, this position focuses primarily on regulatory oversight rather than designing or optimizing waste treatment systems.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 32
• gradient_accumulation_steps: 32
• learning_rate: 2e-05
• weight_decay: 0.01
• num_train_epochs: 5
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: True
• load_best_model_at_end: True
• gradient_checkpointing: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 32
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: True
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: True
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
0.5654	5	0.1496	-
1.0	9	-	0.0082
1.1131	10	0.0134	-
1.6784	15	0.0038	-
2.0	18	-	0.0047
2.2261	20	0.0037	-
2.7915	25	0.0021	-
3.0	27	-	0.0032
3.3392	30	0.0014	-
3.9046	35	0.0022	-
4.0	36	-	0.0027
4.4523	40	0.0015	-
5.0	45	0.0017	0.0027

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.8
• Sentence Transformers: 5.1.0
• Transformers: 4.55.0
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.0
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}