Hugging Face's logo

seroe
/
bge-reranker-v2-m3-turkish-triplet

Text Ranking
sentence-transformers
Safetensors
Turkish
xlm-roberta
cross-encoder
Generated from Trainer
dataset_size:215676
loss:CachedMultipleNegativesRankingLoss
Eval Results
Model card Files Community

bge-reranker-v2-m3 fine-tuned on Turkish triplets

This is a Cross Encoder model finetuned from BAAI/bge-reranker-v2-m3 on the vodex-turkish-triplets-large dataset using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

Model Sources

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the ๐Ÿค— Hub
model = CrossEncoder("seroe/bge-reranker-v2-m3-turkish-triplet")
# Get scores for pairs of texts
pairs = [
    ['Yeni Red Business VIP tarifesi, yรผksek veri ve dakika ihtiyaรงlarฤฑ olan iลŸletmeler iรงin tasarlanmฤฑลŸ bir premium seรงenektir.', 'Red Business VIP, iลŸletmelerin yoฤŸun veri ve konuลŸma ihtiyaรงlarฤฑnฤฑ karลŸฤฑlamak iรงin geliลŸtirilmiลŸ bir รผst dรผzey tarifedir.'],
    ["Vodafone'un Yeni Uyumlu HoลŸgeldin Kampanyasฤฑ, belirli tarifeler iรงin 12+12 ay taahhรผt karลŸฤฑlฤฑฤŸฤฑnda indirimler sunmaktadฤฑr ve kampanya iki dรถnemden oluลŸmaktadฤฑr.", "Vodafone'un Yeni Uyumlu HoลŸgeldin Kampanyasฤฑ, 12+12 ay taahhรผt veren abonelere belirli tarifelerde ilk 12 ay iรงin 20 TL, ikinci 12 ay iรงin 15 TL indirim saฤŸlamaktadฤฑr."],
    ["Vodafone'un Kolay Paketleri, faturasฤฑz hat kullanฤฑcฤฑlarฤฑna TL yรผkleme gereksinimi olmadan avantajlฤฑ paketler sunar ve her ay otomatik yenilenmez.", "Vodafone'un Kolay Paketleri, faturasฤฑz hat kullanฤฑcฤฑlarฤฑ iรงin tasarlanmฤฑลŸ olup, TL yรผkleme zorunluluฤŸu olmadan satฤฑn alฤฑnabilir ve otomatik yenileme yapฤฑlmaz."],
    ["Samsung Galaxy Note 3 cihazฤฑ, Vodafone'un Red tarifeleriyle birlikte aylฤฑk ek รถdeme seรงenekleriyle sunulmuลŸ ve kampanya kodlarฤฑyla desteklenmiลŸtir.", 'Vodafone, Samsung Galaxy Note 3 cihazฤฑnฤฑ Red tarifeleriyle birleลŸtirerek, aylฤฑk ek รถdeme planlarฤฑ ve kampanya kodlarฤฑyla mรผลŸterilere sunmuลŸtur.'],
    ['Red Elite Extra tarifesi, 36 aylฤฑk taahhรผtle 40 TL baลŸlangฤฑรง fiyatฤฑ ve 165 TL รผst fiyat seรงeneฤŸiyle sona eren kampanyalar arasฤฑnda yer almฤฑลŸtฤฑr.', "Vodafone'un sona eren kampanyalarฤฑ arasฤฑnda yer alan Red Elite Extra tarifesi, 36 aylฤฑk taahhรผtle 40 TL'den baลŸlayฤฑp 165 TL'ye kadar fiyatlandฤฑrฤฑlmฤฑลŸtฤฑr."],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'Yeni Red Business VIP tarifesi, yรผksek veri ve dakika ihtiyaรงlarฤฑ olan iลŸletmeler iรงin tasarlanmฤฑลŸ bir premium seรงenektir.',
    [
        'Red Business VIP, iลŸletmelerin yoฤŸun veri ve konuลŸma ihtiyaรงlarฤฑnฤฑ karลŸฤฑlamak iรงin geliลŸtirilmiลŸ bir รผst dรผzey tarifedir.',
        "Vodafone'un Yeni Uyumlu HoลŸgeldin Kampanyasฤฑ, 12+12 ay taahhรผt veren abonelere belirli tarifelerde ilk 12 ay iรงin 20 TL, ikinci 12 ay iรงin 15 TL indirim saฤŸlamaktadฤฑr.",
        "Vodafone'un Kolay Paketleri, faturasฤฑz hat kullanฤฑcฤฑlarฤฑ iรงin tasarlanmฤฑลŸ olup, TL yรผkleme zorunluluฤŸu olmadan satฤฑn alฤฑnabilir ve otomatik yenileme yapฤฑlmaz.",
        'Vodafone, Samsung Galaxy Note 3 cihazฤฑnฤฑ Red tarifeleriyle birleลŸtirerek, aylฤฑk ek รถdeme planlarฤฑ ve kampanya kodlarฤฑyla mรผลŸterilere sunmuลŸtur.',
        "Vodafone'un sona eren kampanyalarฤฑ arasฤฑnda yer alan Red Elite Extra tarifesi, 36 aylฤฑk taahhรผtle 40 TL'den baลŸlayฤฑp 165 TL'ye kadar fiyatlandฤฑrฤฑlmฤฑลŸtฤฑr.",
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Reranking

  • Datasets: val-hard and test-hard
  • Evaluated with CrossEncoderRerankingEvaluator with these parameters:
    {
    "at_k": 10,
    "always_rerank_positives": true
}
Metric val-hard test-hard
map 0.7900 (+0.1059) 0.7891 (+0.1083)
mrr@10 0.7903 (+0.1062) 0.7897 (+0.1089)
ndcg@10 0.8425 (+0.1583) 0.8418 (+0.1609)

Training Details

Training Dataset

vodex-turkish-triplets-large

  • Dataset: vodex-turkish-triplets-large at 1fe9d63
  • Size: 215,676 training samples
  • Columns: query, positive, and negative
  • Approximate statistics based on the first 1000 samples:
    query positive negative
    type string string string
    details
    • min: 57 characters
    • mean: 141.8 characters
    • max: 282 characters
    • min: 61 characters
    • mean: 145.94 characters
    • max: 325 characters
    • min: 62 characters
    • mean: 119.94 characters
    • max: 235 characters
  • Samples:
    query positive negative
    Yeni Red Business VIP tarifesi, yรผksek veri ve dakika ihtiyaรงlarฤฑ olan iลŸletmeler iรงin tasarlanmฤฑลŸ bir premium seรงenektir. Red Business VIP, iลŸletmelerin yoฤŸun veri ve konuลŸma ihtiyaรงlarฤฑnฤฑ karลŸฤฑlamak iรงin geliลŸtirilmiลŸ bir รผst dรผzey tarifedir. Vodafone'un kurumsal tarifeleri, yalnฤฑzca kรผรงรผk iลŸletmelerin dรผลŸรผk veri ihtiyaรงlarฤฑna odaklanmaktadฤฑr.
    Vodafone'un Yeni Uyumlu HoลŸgeldin Kampanyasฤฑ, belirli tarifeler iรงin 12+12 ay taahhรผt karลŸฤฑlฤฑฤŸฤฑnda indirimler sunmaktadฤฑr ve kampanya iki dรถnemden oluลŸmaktadฤฑr. Vodafone'un Yeni Uyumlu HoลŸgeldin Kampanyasฤฑ, 12+12 ay taahhรผt veren abonelere belirli tarifelerde ilk 12 ay iรงin 20 TL, ikinci 12 ay iรงin 15 TL indirim saฤŸlamaktadฤฑr. Vodafone'un Yeni Uyumlu HoลŸgeldin Kampanyasฤฑ, yalnฤฑzca faturasฤฑz hat kullanฤฑcฤฑlarฤฑna รถzel olarak tasarlanmฤฑลŸ bir kampanyadฤฑr ve taahhรผt gerektirmez.
    Vodafone'un Kolay Paketleri, faturasฤฑz hat kullanฤฑcฤฑlarฤฑna TL yรผkleme gereksinimi olmadan avantajlฤฑ paketler sunar ve her ay otomatik yenilenmez. Vodafone'un Kolay Paketleri, faturasฤฑz hat kullanฤฑcฤฑlarฤฑ iรงin tasarlanmฤฑลŸ olup, TL yรผkleme zorunluluฤŸu olmadan satฤฑn alฤฑnabilir ve otomatik yenileme yapฤฑlmaz. Vodafone'un Kolay Paketleri, faturalฤฑ hat kullanฤฑcฤฑlarฤฑna รถzel olarak tasarlanmฤฑลŸ ve her ay otomatik olarak yenilenen paketlerdir.
  • Loss: CachedMultipleNegativesRankingLoss with these parameters:
    {
    "scale": 10.0,
    "num_negatives": 4,
    "activation_fn": "torch.nn.modules.activation.Sigmoid",
    "mini_batch_size": 32
}

Training Hyperparameters

Non-Default Hyperparameters

  • eval_strategy: steps
  • per_device_train_batch_size: 512
  • per_device_eval_batch_size: 1024
  • learning_rate: 1e-06
  • weight_decay: 0.08
  • num_train_epochs: 2
  • warmup_ratio: 0.2
  • save_only_model: True
  • bf16: True
  • dataloader_num_workers: 8
  • load_best_model_at_end: True
  • group_by_length: True
  • batch_sampler: no_duplicates

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: steps
  • prediction_loss_only: True
  • per_device_train_batch_size: 512
  • per_device_eval_batch_size: 1024
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 1e-06
  • weight_decay: 0.08
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1
  • num_train_epochs: 2
  • max_steps: -1
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.2
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: True
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: True
  • fp16: False
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 8
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: True
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch
  • optim_args: None
  • adafactor: False
  • group_by_length: True
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: None
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • include_for_metrics: []
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • use_liger_kernel: False
  • eval_use_gather_object: False
  • average_tokens_across_devices: False
  • prompts: None
  • batch_sampler: no_duplicates
  • multi_dataset_batch_sampler: proportional

Training Logs

Epoch Step Training Loss val-hard_ndcg@10 test-hard_ndcg@10
0.2370 100 0.0183 0.8162 (+0.1320) 0.8131 (+0.1322)
0.4739 200 0.0129 0.8246 (+0.1404) 0.8231 (+0.1422)
0.7109 300 0.0089 0.8302 (+0.1460) 0.8287 (+0.1478)
0.9479 400 0.008 0.8359 (+0.1517) 0.8345 (+0.1536)
1.1848 500 0.0065 0.8400 (+0.1558) 0.8395 (+0.1586)
1.4218 600 0.006 0.8409 (+0.1567) 0.8404 (+0.1594)
1.6588 700 0.0057 0.8420 (+0.1578) 0.8415 (+0.1605)
1.8957 800 0.0056 0.8425 (+0.1583) 0.8418 (+0.1609)

Framework Versions

  • Python: 3.10.12
  • Sentence Transformers: 4.2.0.dev0
  • Transformers: 4.52.3
  • PyTorch: 2.7.0+cu126
  • Accelerate: 1.7.0
  • Datasets: 3.6.0
  • Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
Downloads last month
12
Safetensors
Model size
568M params
Tensor type
F32
ยท
Inference Providers NEW
Text Ranking
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for seroe/bge-reranker-v2-m3-turkish-triplet

Base model

BAAI/bge-reranker-v2-m3
Finetuned
(12)
this model

Evaluation results

View on Papers With Code