---
license: mit
datasets:
- pacovaldez/stackoverflow-questions
language:
- en
base_model:
- google/flan-t5-base
tokenuzer:
- google/flan-t5-base
library_name: transformers
tags:
- Stackoverflow
- flan-t5
- peft
- lora
- seq2seq
---
# 🤖 FLAN-T5 Base Fine-Tuned on Stack Overflow Questions (LoRA)
This is a fine-tuned version of [`google/flan-t5-base`](https://huggingface.co/google/flan-t5-base) on a curated dataset of Stack Overflow programming questions. It was trained using [LoRA](https://arxiv.org/abs/2106.09685) (Low-Rank Adaptation) for parameter-efficient fine-tuning, making it compact, efficient, and effective at modeling developer-style Q&A tasks.
---
## 🧠 Model Objective
The model is trained to:
- Rewrite or improve unclear programming questions
- Generate relevant clarifying questions or answers
- Summarize long developer queries
- Serve as a code-aware Q&A assistant
---
## 📚 Training Data
- **Source**: Stack Overflow public questions dataset (cleaned)
- **Format**: Instruction-like examples, Q&A pairs, summarization prompts
- **Cleaning**: HTML stripping, markdown-to-text, code-preserved
- **Size**: ~15k examples
---
## 🏗️ Training Details
- **Base Model**: `google/flan-t5-base`
- **Adapter Format**: LoRA using [`peft`](https://github.com/huggingface/peft)
- **Files**:
  - `adapter_model.safetensors`
  - `adapter_config.json`
- **Hyperparameters**:
  - `r`: 8
  - `lora_alpha`: 16
  - `lora_dropout`: 0.1
  - `bias`: "none"
  - `task_type`: "SEQ_2_SEQ_LM"
- **Inference Mode**: Enabled
---
## 💡 How to Use
```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from peft import PeftModel

# Load tokenizer and base model
tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-base")
base_model = AutoModelForSeq2SeqLM.from_pretrained("google/flan-t5-base")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "your-model-folder")
model.eval()

# Inference
prompt = "Rewrite this question more clearly: why is my javascript function undefined?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

🧪 Intended Use
This model is best suited for:
Code-aware chatbot assistants
Prompt engineering for developer tools
Developer-focused summarization / rephrasing
Auto-moderation / clarification of tech questions

⚠️ Limitations
Not trained for code generation or long-form answers
May hallucinate incorrect or generic responses
Finetuned only on Stack Overflow — domain-specific

✨ Acknowledgements
Hugging Face Transformers
LoRA (PEFT)
Stack Overflow for open data
FLAN-T5: Scaling Instruction-Finetuned Models

🛠️ Created with love by Kunj | Model suggestion & guidance by ChatGPT