---
base_model: Qwen/Qwen2.5-3B-Instruct
library_name: peft
license: apache-2.0
datasets:
- gbharti/wealth-alpaca_lora
language:
- en
---





## Model Details

### Model Description

This model is a fine-tuned version of the Qwen/Qwen2.5-3B-Instruct base model, optimized using QLoRA (Quantized Low-Rank Adaptation) on the Wealth Alpaca Dataset. It is designed to answer financial questions by combining domain-specific knowledge with the powerful capabilities of Qwen 2.5.



- **Developed by:** Ojaswa Yadav
- **Model type:** Conversational AI
- **Language(s) (NLP):** English (NLP)
- **License:** Apache 2.0
- **Finetuned from model [optional]:** Qwen/Qwen2.5-3B-Instruct





### Direct Use

The model can be directly used for:

Financial question answering
Analyzing financial reports
Conversational AI for finance-related customer support



### Downstream Use [optional]

Can be integrated into other systems for:

Financial sentiment analysis
Advanced financial data retrieval pipelines



### Out-of-Scope Use

The model is not intended for:

General-purpose chat
Non-financial domains (accuracy not guaranteed)



## Bias, Risks, and Limitations

Bias: The training dataset may introduce biases from Wealth Alpaca data. Use caution for sensitive or high-stakes decisions.
Risks: Not suitable for real-time financial trading or critical decision-making without expert validation.
Limitations: Focused on English financial data and may not generalize to other languages or domains.




### Recommendations


Use the model with a RAG for best results

## How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "Ojaswa/QLoRA-Finetuned-Qwen-2.5-on-Wealth-Alpaca-Dataset"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Example usage
inputs = tokenizer("Explain Stock Market to me?", return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))


## Training Details

### Training Data


Dataset: Wealth Alpaca Dataset, consisting of preformatted financial Q&A pairs.
Preprocessing: Combined input instructions, financial context, and output answers into tokenized prompts.



### Training Procedure

Training Procedure
Fine-tuning Method: QLoRA with 4-bit quantization.
Targeted Layers: q_proj and v_proj of the attention mechanism.
Dropout: 0.1
Optimizer: AdamW with learning rate 2e-5.
Hardware: Trained on consumer-grade GPUs (NVIDIA L4).




#### Training Hyperparameters

Training Hyperparameters
Training Regime: Mixed precision (FP16)
Epochs: 3
Batch Size: 32

#### Speeds, Sizes, Times [optional]

Training Time: Approximately 24 hours 




- PEFT 0.13.2