metadata
base_model: desarrolloasesoreslocales/bert-leg-al-corpus
library_name: setfit
metrics:
- accuracy
pipeline_tag: text-classification
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
widget:
- text: " Que lo anteriormente expuesto se realiza al amparo del Real Decreto Legislativo 6/2015, de 30 de octubre, por el que se aprueba el texto refundido de la Ley sobre Tráfico, Circulación de Vehículos a Motor y Seguridad Vial, en su artículo 112 dice :\r\n\r\n1. El plazo de prescripción de las infracciones previstas en esta Ley será de tres meses para las infracciones leves y de seis para las infracciones graves y muy graves. DIPUTACIÓN PROVINCIALDE CÁDIZ 3 E El ¿ a tps:/sede.dpucadiz eslvenemaltode/WUEWOD 22Z8IZPHSZVPCHASGGI 2/2 Estado de elaboración: Copla electónica auténtica de documento papel En El plazo de prescripción comenzará a contara partir del mismo día en que los hechos se hubieran cometido."
- text: >-
En el expediente no se describe con precisión el motivo por el cual se me
ha sancionado.
- text: >-
En el presente procedimiento sancionador se vulnera el Principio de
Tipicidad
- text: >-
Pienso que la sanción es excesiva y no se ha considerado adecuadamente la
gravedad de la falta.
- text: >-
La administración debe probar los hechos para no transgredir el principio
de presunción de inocencia.
inference: true
model-index:
- name: SetFit with desarrolloasesoreslocales/bert-leg-al-corpus
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: Unknown
type: unknown
split: test
metrics:
- type: accuracy
value: 0.8988095238095238
name: Accuracy
SetFit with desarrolloasesoreslocales/bert-leg-al-corpus
This is a SetFit model that can be used for Text Classification. This SetFit model uses desarrolloasesoreslocales/bert-leg-al-corpus as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
- Model Type: SetFit
- Sentence Transformer body: desarrolloasesoreslocales/bert-leg-al-corpus
- Classification head: a LogisticRegression instance
- Maximum Sequence Length: 512 tokens
- Number of Classes: 21 classes
Model Sources
- Repository: SetFit on GitHub
- Paper: Efficient Few-Shot Learning Without Prompts
- Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts
Model Labels
Label | Examples |
---|---|
1001 |
|
2027 |
|
2026 |
|
2002 |
|
2022 |
|
2038 |
|
2017 |
|
2039 |
|
78 |
|
353 |
|
2060 |
|
2037 |
|
237 |
|
357 |
|
994 |
|
2001 |
|
2014 |
|
2010 |
|
2013 |
|
49 |
|
304 |
|
Evaluation
Metrics
Label | Accuracy |
---|---|
all | 0.8988 |
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("desarrolloasesoreslocales/bert-leg-al-setfit")
# Run inference
preds = model("En el presente procedimiento sancionador se vulnera el Principio de Tipicidad")
Training Details
Training Set Metrics
Training set | Min | Median | Max |
---|---|---|---|
Word count | 3 | 30.7036 | 213 |
Label | Training Sample Count |
---|---|
49 | 32 |
78 | 32 |
237 | 31 |
304 | 32 |
353 | 32 |
357 | 32 |
994 | 32 |
1001 | 32 |
2001 | 32 |
2002 | 32 |
2010 | 32 |
2013 | 32 |
2014 | 32 |
2017 | 32 |
2022 | 31 |
2026 | 32 |
2027 | 31 |
2037 | 31 |
2038 | 32 |
2039 | 32 |
2060 | 32 |
Training Hyperparameters
- batch_size: (32, 32)
- num_epochs: (1, 1)
- max_steps: -1
- sampling_strategy: oversampling
- num_iterations: 60
- body_learning_rate: (1e-07, 1e-07)
- head_learning_rate: 1e-05
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: True
- use_amp: True
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: True
Training Results
Epoch | Step | Training Loss | Validation Loss |
---|---|---|---|
0.0004 | 1 | 0.1503 | - |
0.0200 | 50 | 0.1582 | 0.1539 |
0.0399 | 100 | 0.1823 | 0.153 |
0.0599 | 150 | 0.1909 | 0.151 |
0.0798 | 200 | 0.1567 | 0.1486 |
0.0998 | 250 | 0.163 | 0.1449 |
0.1198 | 300 | 0.1919 | 0.1418 |
0.1397 | 350 | 0.1704 | 0.1386 |
0.1597 | 400 | 0.1499 | 0.136 |
0.1796 | 450 | 0.1688 | 0.1328 |
0.1996 | 500 | 0.1165 | 0.1305 |
0.2196 | 550 | 0.1635 | 0.1284 |
0.2395 | 600 | 0.099 | 0.1264 |
0.2595 | 650 | 0.158 | 0.1244 |
0.2794 | 700 | 0.2145 | 0.123 |
0.2994 | 750 | 0.1681 | 0.1216 |
0.3194 | 800 | 0.14 | 0.12 |
0.3393 | 850 | 0.1414 | 0.1184 |
0.3593 | 900 | 0.148 | 0.1173 |
0.3792 | 950 | 0.1551 | 0.1161 |
0.3992 | 1000 | 0.1154 | 0.1151 |
0.4192 | 1050 | 0.1478 | 0.1138 |
0.4391 | 1100 | 0.1152 | 0.1131 |
0.4591 | 1150 | 0.1283 | 0.1123 |
0.4790 | 1200 | 0.1148 | 0.1114 |
0.4990 | 1250 | 0.1662 | 0.1106 |
0.5190 | 1300 | 0.151 | 0.1102 |
0.5389 | 1350 | 0.1413 | 0.1095 |
0.5589 | 1400 | 0.1108 | 0.1089 |
0.5788 | 1450 | 0.1326 | 0.1082 |
0.5988 | 1500 | 0.1261 | 0.1077 |
0.6188 | 1550 | 0.1667 | 0.1073 |
0.6387 | 1600 | 0.1179 | 0.1069 |
0.6587 | 1650 | 0.1023 | 0.1062 |
0.6786 | 1700 | 0.1381 | 0.106 |
0.6986 | 1750 | 0.1586 | 0.1057 |
0.7186 | 1800 | 0.1033 | 0.1051 |
0.7385 | 1850 | 0.1154 | 0.1048 |
0.7585 | 1900 | 0.1174 | 0.1048 |
0.7784 | 1950 | 0.1743 | 0.1044 |
0.7984 | 2000 | 0.0927 | 0.1041 |
0.8184 | 2050 | 0.1266 | 0.1038 |
0.8383 | 2100 | 0.1442 | 0.1037 |
0.8583 | 2150 | 0.1117 | 0.1037 |
0.8782 | 2200 | 0.1154 | 0.1034 |
0.8982 | 2250 | 0.1489 | 0.1031 |
0.9182 | 2300 | 0.0896 | 0.1032 |
0.9381 | 2350 | 0.174 | 0.1033 |
0.9581 | 2400 | 0.2034 | 0.1029 |
0.9780 | 2450 | 0.1743 | 0.1029 |
0.9980 | 2500 | 0.1078 | 0.1031 |
- The bold row denotes the saved checkpoint.
Framework Versions
- Python: 3.10.12
- SetFit: 1.0.3
- Sentence Transformers: 3.0.1
- Transformers: 4.40.2
- PyTorch: 2.3.1+cu121
- Datasets: 2.20.0
- Tokenizers: 0.19.1
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}