Abhilashbikkannavar/mistral-analog-trained-model

TinyLlma Analog Trained Model

This model is a fine-tuned version of TinyLlama/TinyLlama-1.1B-Chat-v1.0 on the analoggenie_converted.jsonl dataset.

Model description

This model has been trained for 5 epochs on analog data to understand and generate analog-related content.

Intended uses & limitations

This model is intended for analog-related text generation and understanding tasks.

Training and evaluation data

• Training Dataset: analoggenie_converted.jsonl
• Training Steps: 2881
• Epochs: 5
• Base Model: TinyLlama/TinyLlama-1.1B-Chat-v1.0

Training procedure

Training hyperparameters

• learning_rate: 0.0001
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 8
• optimizer: adamw_torch
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 20
• training_steps: 2881

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Abhilashbikkannavar/mistral-analog-trained-model")
model = AutoModelForCausalLM.from_pretrained("Abhilashbikkannavar/mistral-analog-trained-model")

Model Details

• File Size: 36GB
• Training Date: $(date +%Y-%m-%d)
• Model Type: Fine-tuned TinyLlama
• License: Apache 2.0

This model is ready for inference and deployment.