---
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:713657
- loss:MultipleNegativesRankingLoss
base_model: TechWolf/ConTeXT-Skill-Extraction-base
widget:
- source_sentence: Successfully gather data from diverse sources, ensuring data integrity
and accuracy.
sentences:
- gather data
- set up basic recording
- use global distribution system
- source_sentence: Learn to mix paints efficiently and effectively in this hands-on
workshop.
sentences:
- design optical prototypes
- manage production of footwear or leather goods
- prepare colour mixtures
- source_sentence: Developed and implemented quality control measures for pre-stitching
processes and techniques for footwear and leather goods.
sentences:
- operate heavy construction machinery without supervision
- pre-stitching processes and techniques for footwear and leather goods
- deburring machine parts
- source_sentence: Improved efficiency in preparing fruits and vegetables for pre-processing
by implementing optimized workflows.
sentences:
- think proactively
- recommend orthopedic goods to customers depending on their condition
- prepare fruits and vegetables for pre-processing
- source_sentence: Skilled in mixing construction grouts while adhering to strict
contamination control measures.
sentences:
- oversee logistics of finished products
- think creatively
- mix construction grouts
pipeline_tag: sentence-similarity
library_name: sentence-transformers
---
# SentenceTransformer based on TechWolf/ConTeXT-Skill-Extraction-base
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [TechWolf/ConTeXT-Skill-Extraction-base](https://huggingface.co/TechWolf/ConTeXT-Skill-Extraction-base). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [TechWolf/ConTeXT-Skill-Extraction-base](https://huggingface.co/TechWolf/ConTeXT-Skill-Extraction-base)
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
'Skilled in mixing construction grouts while adhering to strict contamination control measures.',
'mix construction grouts',
'oversee logistics of finished products',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 713,657 training samples
* Columns: sentence_0, sentence_1, and label
* Approximate statistics based on the first 1000 samples:
| | sentence_0 | sentence_1 | label |
|:--------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:--------------------------------------------------------------|
| type | string | string | float |
| details | - min: 7 tokens
- mean: 15.73 tokens
- max: 61 tokens
| - min: 3 tokens
- mean: 6.17 tokens
- max: 16 tokens
| - min: 1.0
- mean: 1.0
- max: 1.0
|
* Samples:
| sentence_0 | sentence_1 | label |
|:--------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------|:-----------------|
| Successfully provided stabilisation care in emergency situations, contributing to positive patient outcomes. | provide stabilisation care in emergency | 1.0 |
| This training program covers advanced methods to remove coating from delicate components. | remove coating | 1.0 |
| Utilized statistical modelling to analyse booking patterns and forecast future demand. | analyse booking patterns | 1.0 |
* Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
```json
{
"scale": 20.0,
"similarity_fct": "cos_sim"
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `fp16`: True
- `multi_dataset_batch_sampler`: round_robin
#### All Hyperparameters
Click to expand
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: no
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 3
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: True
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin
Click to expand
| Epoch | Step | Training Loss |
|:------:|:------:|:-------------:|
| 0.0112 | 500 | 0.276 |
| 0.0224 | 1000 | 0.2541 |
| 0.0336 | 1500 | 0.2507 |
| 0.0448 | 2000 | 0.2276 |
| 0.0560 | 2500 | 0.2356 |
| 0.0673 | 3000 | 0.2153 |
| 0.0785 | 3500 | 0.2166 |
| 0.0897 | 4000 | 0.2108 |
| 0.1009 | 4500 | 0.2011 |
| 0.1121 | 5000 | 0.2098 |
| 0.1233 | 5500 | 0.2018 |
| 0.1345 | 6000 | 0.1879 |
| 0.1457 | 6500 | 0.1954 |
| 0.1569 | 7000 | 0.1927 |
| 0.1681 | 7500 | 0.1983 |
| 0.1794 | 8000 | 0.1833 |
| 0.1906 | 8500 | 0.1893 |
| 0.2018 | 9000 | 0.1992 |
| 0.2130 | 9500 | 0.1838 |
| 0.2242 | 10000 | 0.1713 |
| 0.2354 | 10500 | 0.1839 |
| 0.2466 | 11000 | 0.1727 |
| 0.2578 | 11500 | 0.1777 |
| 0.2690 | 12000 | 0.1691 |
| 0.2802 | 12500 | 0.1777 |
| 0.2915 | 13000 | 0.1627 |
| 0.3027 | 13500 | 0.1647 |
| 0.3139 | 14000 | 0.1627 |
| 0.3251 | 14500 | 0.1546 |
| 0.3363 | 15000 | 0.1689 |
| 0.3475 | 15500 | 0.1462 |
| 0.3587 | 16000 | 0.1492 |
| 0.3699 | 16500 | 0.158 |
| 0.3811 | 17000 | 0.1537 |
| 0.3923 | 17500 | 0.1597 |
| 0.4036 | 18000 | 0.1567 |
| 0.4148 | 18500 | 0.1607 |
| 0.4260 | 19000 | 0.1629 |
| 0.4372 | 19500 | 0.1418 |
| 0.4484 | 20000 | 0.1606 |
| 0.4596 | 20500 | 0.1537 |
| 0.4708 | 21000 | 0.1463 |
| 0.4820 | 21500 | 0.1372 |
| 0.4932 | 22000 | 0.1466 |
| 0.5044 | 22500 | 0.1349 |
| 0.5156 | 23000 | 0.1586 |
| 0.5269 | 23500 | 0.1365 |
| 0.5381 | 24000 | 0.1321 |
| 0.5493 | 24500 | 0.1549 |
| 0.5605 | 25000 | 0.1399 |
| 0.5717 | 25500 | 0.1283 |
| 0.5829 | 26000 | 0.1423 |
| 0.5941 | 26500 | 0.1355 |
| 0.6053 | 27000 | 0.1443 |
| 0.6165 | 27500 | 0.1417 |
| 0.6277 | 28000 | 0.1452 |
| 0.6390 | 28500 | 0.1395 |
| 0.6502 | 29000 | 0.1422 |
| 0.6614 | 29500 | 0.1262 |
| 0.6726 | 30000 | 0.1289 |
| 0.6838 | 30500 | 0.1363 |
| 0.6950 | 31000 | 0.1372 |
| 0.7062 | 31500 | 0.1272 |
| 0.7174 | 32000 | 0.1309 |
| 0.7286 | 32500 | 0.1291 |
| 0.7398 | 33000 | 0.1297 |
| 0.7511 | 33500 | 0.1226 |
| 0.7623 | 34000 | 0.1311 |
| 0.7735 | 34500 | 0.1201 |
| 0.7847 | 35000 | 0.1363 |
| 0.7959 | 35500 | 0.1306 |
| 0.8071 | 36000 | 0.1223 |
| 0.8183 | 36500 | 0.1173 |
| 0.8295 | 37000 | 0.1242 |
| 0.8407 | 37500 | 0.125 |
| 0.8519 | 38000 | 0.1384 |
| 0.8632 | 38500 | 0.1196 |
| 0.8744 | 39000 | 0.1251 |
| 0.8856 | 39500 | 0.1201 |
| 0.8968 | 40000 | 0.1199 |
| 0.9080 | 40500 | 0.1298 |
| 0.9192 | 41000 | 0.1223 |
| 0.9304 | 41500 | 0.1335 |
| 0.9416 | 42000 | 0.1194 |
| 0.9528 | 42500 | 0.1124 |
| 0.9640 | 43000 | 0.1127 |
| 0.9752 | 43500 | 0.1126 |
| 0.9865 | 44000 | 0.1242 |
| 0.9977 | 44500 | 0.1241 |
| 1.0089 | 45000 | 0.1061 |
| 1.0201 | 45500 | 0.084 |
| 1.0313 | 46000 | 0.1004 |
| 1.0425 | 46500 | 0.0898 |
| 1.0537 | 47000 | 0.0921 |
| 1.0649 | 47500 | 0.097 |
| 1.0761 | 48000 | 0.0877 |
| 1.0873 | 48500 | 0.098 |
| 1.0986 | 49000 | 0.1078 |
| 1.1098 | 49500 | 0.0947 |
| 1.1210 | 50000 | 0.1051 |
| 1.1322 | 50500 | 0.0981 |
| 1.1434 | 51000 | 0.0965 |
| 1.1546 | 51500 | 0.0893 |
| 1.1658 | 52000 | 0.0969 |
| 1.1770 | 52500 | 0.097 |
| 1.1882 | 53000 | 0.1023 |
| 1.1994 | 53500 | 0.1036 |
| 1.2107 | 54000 | 0.0903 |
| 1.2219 | 54500 | 0.1 |
| 1.2331 | 55000 | 0.0949 |
| 1.2443 | 55500 | 0.0893 |
| 1.2555 | 56000 | 0.0966 |
| 1.2667 | 56500 | 0.094 |
| 1.2779 | 57000 | 0.0955 |
| 1.2891 | 57500 | 0.0917 |
| 1.3003 | 58000 | 0.084 |
| 1.3115 | 58500 | 0.0859 |
| 1.3228 | 59000 | 0.0888 |
| 1.3340 | 59500 | 0.0847 |
| 1.3452 | 60000 | 0.0846 |
| 1.3564 | 60500 | 0.0868 |
| 1.3676 | 61000 | 0.0904 |
| 1.3788 | 61500 | 0.0848 |
| 1.3900 | 62000 | 0.0929 |
| 1.4012 | 62500 | 0.0851 |
| 1.4124 | 63000 | 0.0989 |
| 1.4236 | 63500 | 0.0814 |
| 1.4348 | 64000 | 0.0881 |
| 1.4461 | 64500 | 0.0909 |
| 1.4573 | 65000 | 0.0951 |
| 1.4685 | 65500 | 0.0856 |
| 1.4797 | 66000 | 0.0914 |
| 1.4909 | 66500 | 0.0932 |
| 1.5021 | 67000 | 0.0855 |
| 1.5133 | 67500 | 0.09 |
| 1.5245 | 68000 | 0.0801 |
| 1.5357 | 68500 | 0.087 |
| 1.5469 | 69000 | 0.0866 |
| 1.5582 | 69500 | 0.0867 |
| 1.5694 | 70000 | 0.0959 |
| 1.5806 | 70500 | 0.0922 |
| 1.5918 | 71000 | 0.0898 |
| 1.6030 | 71500 | 0.0823 |
| 1.6142 | 72000 | 0.088 |
| 1.6254 | 72500 | 0.0832 |
| 1.6366 | 73000 | 0.0985 |
| 1.6478 | 73500 | 0.0944 |
| 1.6590 | 74000 | 0.0931 |
| 1.6703 | 74500 | 0.0808 |
| 1.6815 | 75000 | 0.0877 |
| 1.6927 | 75500 | 0.0746 |
| 1.7039 | 76000 | 0.0842 |
| 1.7151 | 76500 | 0.088 |
| 1.7263 | 77000 | 0.0792 |
| 1.7375 | 77500 | 0.0718 |
| 1.7487 | 78000 | 0.0941 |
| 1.7599 | 78500 | 0.0843 |
| 1.7711 | 79000 | 0.0835 |
| 1.7824 | 79500 | 0.0878 |
| 1.7936 | 80000 | 0.0771 |
| 1.8048 | 80500 | 0.0829 |
| 1.8160 | 81000 | 0.086 |
| 1.8272 | 81500 | 0.0802 |
| 1.8384 | 82000 | 0.0901 |
| 1.8496 | 82500 | 0.0859 |
| 1.8608 | 83000 | 0.0871 |
| 1.8720 | 83500 | 0.0787 |
| 1.8832 | 84000 | 0.0894 |
| 1.8944 | 84500 | 0.0895 |
| 1.9057 | 85000 | 0.0912 |
| 1.9169 | 85500 | 0.0795 |
| 1.9281 | 86000 | 0.0775 |
| 1.9393 | 86500 | 0.0693 |
| 1.9505 | 87000 | 0.0811 |
| 1.9617 | 87500 | 0.076 |
| 1.9729 | 88000 | 0.085 |
| 1.9841 | 88500 | 0.0904 |
| 1.9953 | 89000 | 0.087 |
| 2.0065 | 89500 | 0.061 |
| 2.0178 | 90000 | 0.0628 |
| 2.0290 | 90500 | 0.0721 |
| 2.0402 | 91000 | 0.0694 |
| 2.0514 | 91500 | 0.0618 |
| 2.0626 | 92000 | 0.0598 |
| 2.0738 | 92500 | 0.0701 |
| 2.0850 | 93000 | 0.0724 |
| 2.0962 | 93500 | 0.0623 |
| 2.1074 | 94000 | 0.0647 |
| 2.1186 | 94500 | 0.0643 |
| 2.1299 | 95000 | 0.066 |
| 2.1411 | 95500 | 0.0653 |
| 2.1523 | 96000 | 0.0648 |
| 2.1635 | 96500 | 0.0616 |
| 2.1747 | 97000 | 0.0661 |
| 2.1859 | 97500 | 0.0678 |
| 2.1971 | 98000 | 0.0621 |
| 2.2083 | 98500 | 0.0699 |
| 2.2195 | 99000 | 0.0631 |
| 2.2307 | 99500 | 0.0701 |
| 2.2420 | 100000 | 0.0663 |
| 2.2532 | 100500 | 0.0559 |
| 2.2644 | 101000 | 0.0667 |
| 2.2756 | 101500 | 0.0695 |
| 2.2868 | 102000 | 0.0655 |
| 2.2980 | 102500 | 0.0668 |
| 2.3092 | 103000 | 0.0661 |
| 2.3204 | 103500 | 0.0638 |
| 2.3316 | 104000 | 0.0686 |
| 2.3428 | 104500 | 0.0628 |
| 2.3540 | 105000 | 0.0649 |
| 2.3653 | 105500 | 0.0603 |
| 2.3765 | 106000 | 0.064 |
| 2.3877 | 106500 | 0.0651 |
| 2.3989 | 107000 | 0.0589 |
| 2.4101 | 107500 | 0.0621 |
| 2.4213 | 108000 | 0.061 |
| 2.4325 | 108500 | 0.068 |
| 2.4437 | 109000 | 0.0545 |
| 2.4549 | 109500 | 0.0691 |
| 2.4661 | 110000 | 0.0614 |
| 2.4774 | 110500 | 0.0661 |
| 2.4886 | 111000 | 0.0701 |
| 2.4998 | 111500 | 0.0549 |
| 2.5110 | 112000 | 0.0676 |
| 2.5222 | 112500 | 0.0599 |
| 2.5334 | 113000 | 0.0605 |
| 2.5446 | 113500 | 0.0671 |
| 2.5558 | 114000 | 0.0681 |
| 2.5670 | 114500 | 0.063 |
| 2.5782 | 115000 | 0.0586 |
| 2.5895 | 115500 | 0.0629 |
| 2.6007 | 116000 | 0.0586 |
| 2.6119 | 116500 | 0.0668 |
| 2.6231 | 117000 | 0.0606 |
| 2.6343 | 117500 | 0.0521 |
| 2.6455 | 118000 | 0.0619 |
| 2.6567 | 118500 | 0.065 |
| 2.6679 | 119000 | 0.052 |
| 2.6791 | 119500 | 0.0628 |
| 2.6903 | 120000 | 0.0642 |
| 2.7016 | 120500 | 0.0614 |
| 2.7128 | 121000 | 0.0663 |
| 2.7240 | 121500 | 0.0569 |
| 2.7352 | 122000 | 0.0648 |
| 2.7464 | 122500 | 0.0616 |
| 2.7576 | 123000 | 0.0536 |
| 2.7688 | 123500 | 0.0669 |
| 2.7800 | 124000 | 0.0612 |
| 2.7912 | 124500 | 0.0555 |
| 2.8024 | 125000 | 0.059 |
| 2.8136 | 125500 | 0.0549 |
| 2.8249 | 126000 | 0.0563 |
| 2.8361 | 126500 | 0.0616 |
| 2.8473 | 127000 | 0.06 |
| 2.8585 | 127500 | 0.0606 |
| 2.8697 | 128000 | 0.063 |
| 2.8809 | 128500 | 0.0572 |
| 2.8921 | 129000 | 0.0697 |
| 2.9033 | 129500 | 0.0561 |
| 2.9145 | 130000 | 0.065 |
| 2.9257 | 130500 | 0.0525 |
| 2.9370 | 131000 | 0.0597 |
| 2.9482 | 131500 | 0.0604 |
| 2.9594 | 132000 | 0.0534 |
| 2.9706 | 132500 | 0.0553 |
| 2.9818 | 133000 | 0.0593 |
| 2.9930 | 133500 | 0.0554 |