BGE m3 Uzbek Legal Matryoshka

This is a sentence-transformers model finetuned from BAAI/bge-m3 on the json dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-m3
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 1024 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- json
Language: uz
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("fitlemon/bge-m3-uz-legal-matryoshka")
# Run inference
sentences = [
    'Mehnatni huquqiy jihatdan tartibga solishning o‘ziga xos xususiyatlari qaysi bo‘limda ko‘rsatilgan?',
    'Noqulay tabiiy-iqlim sharoitlaridagi ish uchun mehnatga haq to‘lash koeffitsiyenti ayrim \nhududlardagi mehnat sharoitlarining xususiyatlari inobatga olingan holda xodimlarga to‘lanadigan \nkompensatsiya xususiyatiga ega bo‘lgan ustama turidir. Koeffitsiyentlarning eng kam miqdorlari va \nularni qo‘llash tartibi O‘zbekiston Respublikasi Vazirlar Mahkamasi tomonidan belgilanadi. \n28-bob. Xodim mehnatining xususiyati bilan bog‘liq bo‘lgan mehnatni huquqiy jihatdan \ntartibga solishning o‘ziga xos xususiyatlari  \n1-§. Tashkilot rahbarining, uning o‘rinbosarlarining, tashkilot bosh buxgalterining va \ntashkilot alohida bo‘linmasi rahbarining mehnatini huquqiy jihatdan tartibga solishning \no‘ziga xos xususiyatlari',
    'muddat doirasida mehnat nizosini ko‘rib  chiqish haqida takroran ariza berish huquqidan mahrum \netmaydi. Komissiyaning arizani ko‘rib chiqishdan olib tashlash to‘g‘risidagi qarori arizani ko‘rib \nchiqishdan olib tashlash sababi albatta ko‘rsatilgan holda xodimga yozma shaklda yetkaziladi.  \nXodim mehnat nizosini mehnat nizolari bo‘yicha komissiyada ko‘rib chiqishning har qanday \nbosqichida ushbu nizoni tugatishga haqlidir. \nO‘n besh yoshdan o‘n olti yoshgacha bo‘lgan xodimlarning yakka tartibdagi mehnat nizolari \nota-onasidan birining yoki vasiysining ishtirokida mehnat nizolari bo‘yicha komissiyada ko‘rib \nchiqiladi. \nXodim, kasaba uyushmasi qo‘mitasi mehnat nizosini ko‘rib chiqishda ishtirok etish uchun \nadvokatni, ekspertni yoki boshqa uchinchi shaxsni taklif etish huquqiga ega. \nMehnat nizolari bo‘yicha komissiya o‘z majlisiga guvohlarni chaqirish, mutaxassislarni \ntaklif etish huquqiga ega. \nMehnat nizolari bo‘yicha komissiyaning talabiga binoan ish beruvchi (uning vakillari) \nkomissiyaga zarur bo‘lgan hujjatlarni komissiya tomonida n belgilangan muddatda taqdim etishi \nshart. \nMehnat nizolari bo‘yicha komissiyaning majlisi, agar har bir tarafdan komissiya a’zolarining \nkamida yarmi hozir bo‘lsa, vakolatli hisoblanadi. \nMehnat nizolari bo‘yicha komissiya majlisida ish beruvchining va kasa ba uyushmasi \nqo‘mitasining teng miqdordagi vakillari ishtirok etishi kerak. \nMehnat nizolari bo‘yicha komissiyaning vakolatga ega bo‘lmagan tarkibi tomonidan qabul \nqilingan qaror g‘ayriqonuniydir. \nMehnat nizolari bo‘yicha komissiyaning har bir majlisida rai sning vazifalari ish \nberuvchining va kasaba uyushmasi qo‘mitasining vakillari tomonidan navbatma -navbat bajariladi. \nBunda raisning va kotibning vazifalari ayni bir majlisning o‘zida bir taraf vakillari tomonidan \nbajarilishi mumkin emas.  \nMehnat nizolari bo‘yicha komissiyaning birinchi majlisida komissiya a’zolarining (xodimlar',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Datasets: dim_768, dim_512, dim_256, dim_128 and dim_64
Evaluated with InformationRetrievalEvaluator

Metric	dim_768	dim_512	dim_256	dim_128	dim_64
cosine_accuracy@1	0.4004	0.3909	0.3833	0.3681	0.3397
cosine_accuracy@3	0.6774	0.6698	0.6622	0.6414	0.6148
cosine_accuracy@5	0.7875	0.7685	0.7628	0.7457	0.7173
cosine_accuracy@10	0.8634	0.8577	0.8444	0.8425	0.8235
cosine_precision@1	0.4004	0.3909	0.3833	0.3681	0.3397
cosine_precision@3	0.2258	0.2233	0.2207	0.2138	0.2049
cosine_precision@5	0.1575	0.1537	0.1526	0.1491	0.1435
cosine_precision@10	0.0863	0.0858	0.0844	0.0843	0.0824
cosine_recall@1	0.4004	0.3909	0.3833	0.3681	0.3397
cosine_recall@3	0.6774	0.6698	0.6622	0.6414	0.6148
cosine_recall@5	0.7875	0.7685	0.7628	0.7457	0.7173
cosine_recall@10	0.8634	0.8577	0.8444	0.8425	0.8235
cosine_ndcg@10	0.636	0.6282	0.616	0.6047	0.5803
cosine_mrr@10	0.5627	0.5544	0.5425	0.5285	0.5026
cosine_map@100	0.5689	0.561	0.5497	0.5353	0.5108

Training Details

Training Dataset

json

Dataset: json
Size: 4,737 training samples
Columns: question and chunk
Approximate statistics based on the first 1000 samples:
question chunk
type string string
details
min: 9 tokens
mean: 22.61 tokens
max: 50 tokens

min: 30 tokens
mean: 268.2 tokens
max: 520 tokens

	question	chunk
type	string	string
details	min: 9 tokens mean: 22.61 tokens max: 50 tokens	min: 30 tokens mean: 268.2 tokens max: 520 tokens

Samples:

question	chunk
`Xodim mehnat shartnomasining nusxasini olganligini qanday tasdiqlaydi?`	Mehnat shartnomasi bir xil kuchga ega bo‘lgan kamida ikki nusxada yozma shaklda tuzilib, ularning har biri taraflar tomonidan imzolanadi. Mehnat shartnomasining har bir nusxasi xodimning va ishga qabul qilish huquqiga ega bo‘lgan mansabdor shaxsning imzolari bilan mustahkamlanadi. Ish beruvchida muhr mav jud bo‘lgan taqdirda mansabdor shaxsning mehnat shartnomasining barcha nusxalaridagi imzosi muhr bilan tasdiqlanadi. Mehnat shartnomasining bir nusxasi xodimga beriladi, boshqasi (boshqalari) ish beruvchida saqlanadi. Mehnat shartnomasining nusxasi xodim t omonidan olinganligi ish beruvchida saqlanadigan mehnat shartnomasi nusxasidagi xodim mehnat shartnomasining nusxasini olganligi to‘g‘risidagi alohida imzosi bilan tasdiqlanadi. Mehnat shartnomasida shartnomaning ushbu Kodeksning 107-moddasida nazarda tutilgan rekvizitlari ko‘rsatiladi. Ish beruvchilar va xodimlarga mehnat shartnomalarini tuzishda amaliy yordam ko‘rsatish maqsadida O‘zbekiston Respublikasi Vazirlar Mahk...
`Vaqtincha xizmat safariga yuborilgan xodimning mehnatga haq to‘lash shartlari qanday bo'lishi kerak?`	konsullik muassasalariga diplomatik, konsullik, ma’muriy -texnik lavozimlarga yoki xizmat ko‘rsatuvchi xodimlar lavozimlariga vaqtincha xizmat safariga yuborish tartibi va muddatlari qonunchilikda belgilanadi. Xodim boshqa ish beruvchiga v aqtincha xizmat safariga yuborilganda xodimning roziligi bilan uning mehnat vazifasi o‘zgartirilishi mumkin. Vaqtincha xizmat safariga yuborilganda mehnatga haq to‘lash xodim vaqtincha xizmat safariga yuborilgan ish beruvchi tomonidan amalga oshiriladi. Us hbu ish beruvchi to‘lovga qobiliyatsiz bo‘lgan taqdirda, bajarilgan ish uchun haq to‘lash majburiyati xodim vaqtincha xizmat safariga yuborilgan ish beruvchiga nisbatan regress da’vo qo‘zg‘atish huquqi bilan xodimni vaqtincha xizmat safariga yuborgan ish beruvchi zimmasiga yuklatiladi. Agar yangi ish joyidagi mehnatga haq to‘lash shartlari yoki dam olish vaqti xodim uni vaqtincha xizmat safariga yuborgan ish beruvchida foydalanganidan farq qilsa, xodimga nisbatan qulayroq shartlar qo‘ll...
`O‘zbekiston Respublikasi hududida o‘rindoshlik shartlari asosida mehnat shartnomalari qanday shartlar bilan tuzilishi mumkin?`	Mehnat faoliyatini amalga oshirish maqsadida O‘zbekiston Respublikasi hududiga qonuniy ravishda kirib kelgan chet el fuqarolari O‘zbekiston Respublikasi hududida mehnat faoliyatini O‘zbekiston Respublikasi hududida mehnat faoliyatini amalga oshirish huquqiga doir tasdiqnoma asosidagina amalga oshirish huquqiga ega, bundan qonunchilikda belgilangan hollar mustasno. Ish beruvchi O‘zbekiston Respublikasi hududiga qonuniy ravishda kirib kelga n chet el fuqarolari bilan mehnat shartnomalarini tuzishga ularda O‘zbekiston Respublikasi hududida mehnat faoliyatini amalga oshirish huquqiga doir tasdiqnoma mavjud bo‘lgan taqdirdagina haqli, bundan qonunchilikda nazarda tutilgan hollar mustasno. Mehnat faoliyatini amalga oshirish uchun O‘zbekiston Respublikasi hududiga qonuniy ravishda kirib kelgan chet el fuqarolari bilan o‘rindoshlik shartlari asosidagi mehnat shartnomalari faqat ular tomonidan O‘zbekiston Respublikasi hududida mehnat faoliyatini amal ga oshirish huquqiga doir alohida...

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 2
per_device_eval_batch_size: 2
learning_rate: 2e-05
lr_scheduler_type: cosine
warmup_ratio: 0.1
fp16: True
load_best_model_at_end: True
optim: adamw_torch_fused
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 2
per_device_eval_batch_size: 2
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs