|
|
--- |
|
|
license: apache-2.0 |
|
|
tags: |
|
|
- generated_from_keras_callback |
|
|
model-index: |
|
|
- name: distilbert-base-future |
|
|
results: [] |
|
|
widget: |
|
|
- text: "We will have a good time." |
|
|
example_title: "Positive" |
|
|
- text: "We had a good time." |
|
|
example_title: "Negative" |
|
|
--- |
|
|
|
|
|
# distilbert-base-future |
|
|
## Table of Contents |
|
|
|
|
|
- [Model description](#model_description) |
|
|
- [Intended uses & limitations](#intended_uses_&_limitations) |
|
|
- [Training and evaluation data](#training_and_evaluation_data) |
|
|
- [Training procedure](#training_procedure) |
|
|
|
|
|
This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the [future-statements dataset](https://huggingface.co/datasets/fidsinn/future-statements). |
|
|
It achieves the following results on the evaluation set: |
|
|
- Train Loss: 0.1142 |
|
|
- Train Sparse Categorical Accuracy: 0.9613 |
|
|
- Validation Loss: 0.1272 |
|
|
- Validation Sparse Categorical Accuracy: 0.9625 |
|
|
- Epoch: 1 |
|
|
|
|
|
## Model description |
|
|
|
|
|
- The model was created by graduate students [D. Baradari](https://huggingface.co/Dunya), [F. Bartels](https://huggingface.co/fidsinn), A. Dewald, [J. Peters](https://huggingface.co/jpeters92) as part of a data science module of the University of Leipzig. |
|
|
- Model was created on 11/08/22. |
|
|
- This is version 1.0 |
|
|
- The model is a text classification model which is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) |
|
|
- Questions and comments can be send via the [community tab](https://huggingface.co/fidsinn/distilbert-base-future/discussions) |
|
|
|
|
|
## Intended uses & limitations |
|
|
|
|
|
- The primary intended use is the classification of input into a future or non-future sentence/statement. |
|
|
- The model is primarily intended to be used by researchers to filter or label a large number of sentences according to the grammatical tense of the input. |
|
|
|
|
|
## Training and evaluation data |
|
|
|
|
|
- [Distilbert-base-future model](https://huggingface.co/fidsinn/distilbert-base-future) was trained and evaluated on the [future-statements dataset](https://huggingface.co/datasets/fidsinn/future-statements). |
|
|
- [future-statements](https://huggingface.co/datasets/fidsinn/future-statements) is a dataset collected manually and automatically by graduate students [D. Baradari](https://huggingface.co/Dunya), [F. Bartels](https://huggingface.co/fidsinn), A. Dewald, [J. Peters](https://huggingface.co/jpeters92) of the University of Leipzig. |
|
|
- We collected 2500 statements, 50% of which relate to future events and 50% of which relate to non-future events. |
|
|
- The sole purpose of the dataset was the fine-tuning process of this model. |
|
|
- Additional information on the dataset can be found on Huggingface: [future-statements dataset](https://huggingface.co/datasets/fidsinn/future-statements). |
|
|
|
|
|
## Training procedure |
|
|
|
|
|
### Training hyperparameters |
|
|
|
|
|
The following hyperparameters were used during training: |
|
|
- optimizer: {'name': 'Adam', 'learning_rate': 5e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False} |
|
|
- training_precision: float32 |
|
|
|
|
|
### Training results |
|
|
|
|
|
| Train Loss | Train Sparse Categorical Accuracy | Validation Loss | Validation Sparse Categorical Accuracy | Epoch | |
|
|
|:----------:|:---------------------------------:|:---------------:|:--------------------------------------:|:-----:| |
|
|
| 0.3816 | 0.8594 | 0.1547 | 0.9475 | 0 | |
|
|
| 0.1142 | 0.9613 | 0.1272 | 0.9625 | 1 | |
|
|
|
|
|
|
|
|
### Framework versions |
|
|
|
|
|
- Transformers 4.18.0 |
|
|
- TensorFlow 2.8.0 |
|
|
- Datasets 2.4.0 |
|
|
- Tokenizers 0.12.1 |
|
|
|
