🚀 MySQL Query Generator - From Scratch

A state-of-the-art GPT-style transformer model trained completely from scratch for natural language to MySQL query generation. This model demonstrates that high-quality language models can be built without relying on pre-trained weights, achieving excellent performance with a compact architecture.

🎯 Model Overview

This model specializes in converting natural language descriptions into syntactically correct MySQL queries. It was trained entirely from scratch using a custom transformer architecture, making it highly optimized for SQL generation tasks.

Key Features

🔧 Built from Scratch: No pre-trained weights - pure end-to-end training
💾 Lightweight: Compact 29.8M parameters for efficient deployment
⚡ High Performance: Excellent convergence with minimal overfitting
🎯 MySQL Optimized: Specifically tuned for MySQL syntax and patterns
📊 Production Ready: Robust performance across diverse query types

🏗️ Architecture

Component	Specification
Model Type	GPT-style Transformer (Decoder-only)
Layers	8
Attention Heads	8
Hidden Dimensions	512
Feed Forward Size	2048
Max Sequence Length	512 tokens
Dropout Rate	0.1
Total Parameters	29,789,184
Model Size	113.6 MB
Vocabulary Size	4,206 tokens

🎯 Performance Metrics

Metric	Value
Validation Loss	0.3485
Training Loss	0.3178
Perplexity	1.42
Convergence	Excellent
Overfitting	None detected

📊 Training Configuration

Framework: PyTorch
Optimizer: AdamW with weight decay
Learning Rate Scheduler: CosineAnnealingLR
Training Epochs: 8
Training Examples: 24,293 high-quality samples
Hardware: NVIDIA RTX 5080 16GB

📚 Dataset

The model was trained on a carefully curated dataset of 24,293 high-quality examples sourced from:

🔧 Synthetic SQL Queries: Custom-generated queries covering diverse MySQL patterns
🕷️ Spider Dataset: Complex multi-table queries with natural language descriptions
📖 WikiSQL Dataset: Real-world table-question pairs adapted for MySQL

All queries were specifically optimized for MySQL syntax and best practices, ensuring production-ready output.

🚀 Usage

This model excels at converting natural language descriptions into syntactically correct MySQL queries. Perfect for:

Database query assistants
Business intelligence tools
Educational SQL learning platforms
Automated report generation

Example Queries

# Basic Selection
"Show me all customers from New York"
# → SELECT * FROM customers WHERE city = 'New York';

# Aggregation
"Find total sales for each product"
# → SELECT product_name, SUM(sales) FROM sales_table GROUP BY product_name;

# Conditional Filtering
"List employees with salary greater than 50000"
# → SELECT * FROM employees WHERE salary > 50000;

📁 Model Files

File	Description
`best_pretrained_model.pt`	Optimized model checkpoint for inference
`complete_model_package.pt`	Full model package with all components
`model_info.json`	Detailed model specifications and metadata
`training_metrics.json`	Comprehensive training performance data
`SQLModel.ipynb`	Complete training and evaluation notebook

🔬 Technical Details

Model Capabilities

Multi-table Joins: Handles complex relationships between tables
Aggregation Functions: SUM, COUNT, AVG, MIN, MAX operations
Conditional Logic: WHERE clauses with AND/OR operators
Sorting & Grouping: ORDER BY and GROUP BY operations
Subqueries: Nested query generation for complex requirements

Limitations

Optimized specifically for MySQL syntax (may not work with other SQL dialects)
Best performance on queries similar to training data patterns
May require fine-tuning for highly specialized domain vocabularies

📖 Citation

If you use this model in your research or applications, please cite:

@misc{mysql-query-generator-from-scratch,
  title={MySQL Query Generator: A GPT-style Transformer Trained From Scratch},
  author={Anonymous},
  year={2025},
  howpublished={\\url{https://huggingface.co/karthik-2905/nl2sql-pretrained}},
  note={Natural Language to SQL Query Generation}
}

📄 License

This model is released under the Apache 2.0 License, allowing for both commercial and non-commercial use.

🤝 Community & Support

Open Source: Community-driven development
Contributions Welcome: Feel free to submit improvements
Issues & Feedback: Report problems or suggest enhancements
Educational Use: Perfect for learning NL2SQL concepts

⭐ If you find this model useful, please give it a star and share it with others!