Model Card for Model ID
Model Details
Model Description
This is a fine-tuned Qwen3-0.6B model specialized for extracting chapter information and table of contents from documents. The model has been trained to identify and structure chapter titles, sections, and document hierarchies.
- Developed by: Davide Panza
- Model type: Causal Language Model (Fine-tuned for Text Extraction)
- License: Apache 2.0
- Finetuned from model: unsloth/Qwen3-0.6B-unsloth-bnb-4bit
Model Sources [optional]
Uses
- Extract chapter titles and section headings from documents
- Generate structured table of contents
Training Details
Training Data
synthetic data
repo: https://github.com/DavidePanza/finetuning_LLM_for_Chapter_Extraction/tree/main/data/training_dataset
Training Procedure
- Base Model: Qwen3-0.6B (via Unsloth's optimized 4-bit version)
- Training Framework: Unsloth with SFTTrainer
- Fine-tuning Method: Supervised Fine-Tuning (SFT) with LoRA
- Precision: 16-bit merged (from 4-bit quantized base)
Training Hyperparameters
- Training regime:
- Epochs: 2
- Batch Size: 4 (per device)
- Gradient Accumulation Steps: 2 (effective batch size: 8)
- Learning Rate: 5e-5
- Optimizer: AdamW 8-bit
- Weight Decay: 0.001
- LR Scheduler: Cosine with warmup
- Warmup Steps: 100
- Max Gradient Norm: 1.0 (gradient clipping)
- Mixed Precision: FP16 enabled
- Logging Steps: 10
- Downloads last month
- 2