Hugging Face's logo

yangwooko
/
powerinfer-seq-cls-ywko_e5

Text Classification
Transformers
TensorBoard
Safetensors
qwen2
Generated from Trainer
text-generation-inference
Model card Files Metrics Community
powerinfer-seq-cls-ywko_e5 / README.md
yangwooko's picture
yangwooko
Model save
35fdf0b verified
preview code
|
raw
history blame contribute delete
1.83 kB
metadata
library_name: transformers
base_model: PowerInfer/SmallThinker-3B-Preview
tags:
  - generated_from_trainer
model-index:
  - name: powerinfer-seq-cls-ywko_e5
    results: []

powerinfer-seq-cls-ywko_e5

This model is a fine-tuned version of PowerInfer/SmallThinker-3B-Preview on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0727

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 64
  • eval_batch_size: 8
  • seed: 42
  • distributed_type: multi-GPU
  • gradient_accumulation_steps: 64
  • total_train_batch_size: 4096
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine_with_restarts
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 5
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
0.4779 0.9143 30 0.1895
0.1332 1.8533 60 0.1015
0.0797 2.7924 90 0.0780
0.0811 3.7314 120 0.0743
0.0669 4.6705 150 0.0727

Framework versions

  • Transformers 4.50.3
  • Pytorch 2.5.1+cu124
  • Datasets 3.5.0
  • Tokenizers 0.21.1