README.md

---
license: apache-2.0
tags:
  - text-to-sql
  - llama3
  - lora
  - sql-generation
  - code-generation
library_name: transformers
base_model: unsloth/Meta-Llama-3.1-8B
pipeline_tag: text-generation
---

# Llama3 SQL Translator

**Llama3 SQL Translator** is a LoRA fine-tuned version of the 8B parameter Llama 3.1 model. It is designed to translate natural language database queries into executable SQL statements and provide human-readable explanations. The model streamlines query generation for structured databases and enables non-technical users to interact with relational data more effectively.

## Table of Contents

1. [Model Details](#model-details)  
2. [Intended Uses](#intended-uses)  
3. [Limitations & Warnings](#limitations--warnings)  
4. [Training Overview](#training-overview)  
5. [Evaluation](#evaluation)  
6. [Usage Example](#usage-example)  
7. [Technical Specifications](#technical-specifications)  
8. [Citation & Contact](#citation--contact)

## Model Details

- **Model Type:** Causal language model
- **Architecture:** Llama 3.1 (8B parameters)
- **Fine-Tuning Method:** Parameter-efficient fine-tuning (LoRA)
- **Base Model:** unsloth/Meta-Llama-3.1-8B
- **Language:** English
- **Tokenizer:** Llama 3 tokenizer (compatible with Meta's original)

## Intended Uses

### Primary Use
- Translating natural language prompts into valid SQL queries.
- Providing explanations of the generated SQL logic.

### Example Input
```text
Database schema: CREATE TABLE employees (id INT, name TEXT, salary FLOAT);
Prompt: List all employees with salary over 50000.
```

### Example Output
```text
SQL: SELECT name FROM employees WHERE salary > 50000;
Explanation: This query retrieves all employee names where the salary is greater than 50000.
```

### Not Intended For
- General chat, Q&A, or non-database related tasks.
- Use without human review in critical systems or production databases.

## Limitations & Warnings

- **Schema Dependency:** The model relies heavily on accurate and complete schema descriptions.
- **SQL Safety:** The output SQL should not be executed without manual validation. Injection risks must be mitigated.
- **Complex Queries:** Deeply nested subqueries, advanced joins, or vendor-specific SQL dialects may produce suboptimal results.

## Training Overview

- The model was trained on a large-scale synthetic dataset containing pairs of natural language instructions, database schemas, corresponding SQL queries, and their step-by-step explanations. The dataset covers a wide range of relational data scenarios and query types, including filtering, aggregation, joins, and nested logic.
- Fine-tuned on a single A100 GPU using:
  - `max_seq_length=1024`
  - `batch_size=2`, `gradient_accumulation_steps=2`
  - LoRA with 4-bit quantization
  - `packing=True` to maximize throughput
  - Trained for 1 epoch (~5 hours)

## Evaluation

| Metric                  | Result         |
|-------------------------|----------------|
| SQL compilation success | > 95%          |
| Manual output quality   | ~90%+          |
| Explanation clarity     | High           |

*Note: Evaluation was based on random sampling and manual review. Formal benchmarks will be added later.*

## Usage Example

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "happyhackingspace/llama3-sql-translator"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = """Below is an instruction that describes a task, paired with an input that provides further context.
Write a response that appropriately completes the request.

### Instruction
Database schema: CREATE TABLE sales (id INT, product TEXT, price FLOAT);

### Input:
Prompt: Show all products priced over 100.

### Response:"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Technical Specifications

- **Architecture:** Llama 3.1 - 8B
- **Quantization:** 4-bit via bitsandbytes
- **Fine-tuning:** LoRA
- **Frameworks:** Transformers, TRL, PEFT, Unsloth

## Citation & Contact

```bibtex
@misc{llama3_sql_translator_2025,
  title = {Llama3 SQL Translator},
  author = {happyhackingspace},
  year = {2025},
  howpublished = {\url{https://huggingface.co/happyhackingspace/llama3-sql-translator}}
}
```

**Contact:** For questions or contributions, feel free to open an issue on the Hugging Face model page.