|
|
--- |
|
|
base_model: |
|
|
- meta-llama/Llama-3.2-3B-Instruct |
|
|
- codellama/CodeLlama-7b-Instruct-hf |
|
|
- codellama/CodeLlama-13b-Instruct-hf |
|
|
library_name: peft |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- lora |
|
|
- transformers |
|
|
- configuration-management |
|
|
- secrets-management |
|
|
- devops |
|
|
- multi-cloud |
|
|
- gguf |
|
|
- anysecret |
|
|
license: mit |
|
|
language: |
|
|
- en |
|
|
--- |
|
|
|
|
|
# AnySecret Assistant - Multi-Model Collection |
|
|
|
|
|
A specialized AI assistant collection for AnySecret configuration management, available in multiple sizes and formats optimized for different use cases and deployment scenarios. |
|
|
|
|
|
## 🚀 Available Models |
|
|
|
|
|
| Model | Base Model | Parameters | Format | Best For | Memory | |
|
|
|-------|------------|------------|--------|----------|--------| |
|
|
| **3B** | Llama-3.2-3B-Instruct | 3B | PyTorch/GGUF | Fast responses, edge deployment | 4-6GB | |
|
|
| **7B** | CodeLlama-7B-Instruct | 7B | PyTorch/GGUF | Balanced performance, code focus | 8-12GB | |
|
|
| **13B** | CodeLlama-13B-Instruct | 13B | PyTorch/GGUF | Highest quality, complex queries | 16-24GB | |
|
|
|
|
|
### Model Variants |
|
|
|
|
|
#### PyTorch Models (LoRA Adapters) |
|
|
- `anysecret-io/anysecret-assistant/3B/` - Llama-3.2-3B base |
|
|
- `anysecret-io/anysecret-assistant/7B/` - CodeLlama-7B base |
|
|
- `anysecret-io/anysecret-assistant/13B/` - CodeLlama-13B base |
|
|
|
|
|
#### GGUF Models (Quantized) |
|
|
- `anysecret-io/anysecret-assistant/3B-GGUF/` - Q4_K_M, Q8_0 formats |
|
|
- `anysecret-io/anysecret-assistant/7B-GGUF/` - Q4_K_M, Q8_0 formats |
|
|
- `anysecret-io/anysecret-assistant/13B-GGUF/` - Q4_K_M, Q8_0 formats |
|
|
|
|
|
## 🎯 Model Description |
|
|
|
|
|
These models are fine-tuned specifically to assist with AnySecret configuration management across AWS, GCP, Azure, and Kubernetes environments. Each model can help with CLI commands, configuration setup, CI/CD integration, and Python SDK usage. |
|
|
|
|
|
- **Developed by:** anysecret-io |
|
|
- **Model type:** Causal Language Model (LoRA Adapters + GGUF) |
|
|
- **Language(s):** English |
|
|
- **License:** MIT |
|
|
- **Specialized for:** Multi-cloud secrets and configuration management |
|
|
|
|
|
## 📦 Quick Start |
|
|
|
|
|
### Option 1: Using Ollama (Recommended for GGUF) |
|
|
|
|
|
```bash |
|
|
# 7B model (balanced performance) |
|
|
ollama pull anysecret-io/anysecret-assistant/7B-GGUF |
|
|
ollama run anysecret-io/anysecret-assistant/7B-GGUF |
|
|
|
|
|
# 13B model (best quality) |
|
|
ollama pull anysecret-io/anysecret-assistant/13B-GGUF |
|
|
ollama run anysecret-io/anysecret-assistant/13B-GGUF |
|
|
``` |
|
|
|
|
|
### Option 2: Using Transformers (PyTorch) |
|
|
|
|
|
```python |
|
|
from peft import PeftModel |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
import torch |
|
|
|
|
|
# Choose your model size (3B/7B/13B) |
|
|
model_size = "7B" # or "3B", "13B" |
|
|
base_models = { |
|
|
"3B": "meta-llama/Llama-3.2-3B-Instruct", |
|
|
"7B": "codellama/CodeLlama-7b-Instruct-hf", |
|
|
"13B": "codellama/CodeLlama-13b-Instruct-hf" |
|
|
} |
|
|
|
|
|
base_model_name = base_models[model_size] |
|
|
adapter_path = f"anysecret-io/anysecret-assistant/{model_size}" |
|
|
|
|
|
# Load model |
|
|
base_model = AutoModelForCausalLM.from_pretrained( |
|
|
base_model_name, |
|
|
torch_dtype=torch.float16, |
|
|
device_map="auto" |
|
|
) |
|
|
model = PeftModel.from_pretrained(base_model, adapter_path) |
|
|
tokenizer = AutoTokenizer.from_pretrained(base_model_name) |
|
|
|
|
|
# Generate response |
|
|
def ask_anysecret(question): |
|
|
prompt = f"### Instruction:\n{question}\n\n### Response:\n" |
|
|
inputs = tokenizer(prompt, return_tensors="pt") |
|
|
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.1) |
|
|
response = tokenizer.decode(outputs[0], skip_special_tokens=True) |
|
|
return response.split("### Response:\n")[-1].strip() |
|
|
|
|
|
# Example usage |
|
|
print(ask_anysecret("How do I configure AnySecret for AWS?")) |
|
|
``` |
|
|
|
|
|
### Option 3: Using llama.cpp (GGUF) |
|
|
|
|
|
```bash |
|
|
# Download GGUF model |
|
|
wget https://huggingface.co/anysecret-io/anysecret-assistant/resolve/main/7B-GGUF/anysecret-7b-q4_k_m.gguf |
|
|
|
|
|
# Run with llama.cpp |
|
|
./llama-server -m anysecret-7b-q4_k_m.gguf --port 8080 |
|
|
``` |
|
|
|
|
|
## 🎯 Use Cases |
|
|
|
|
|
### Direct Use |
|
|
|
|
|
All models are designed to provide expert assistance with: |
|
|
|
|
|
- **AnySecret CLI** - Commands, usage patterns, troubleshooting |
|
|
- **Multi-cloud Configuration** - AWS Secrets Manager, GCP Secret Manager, Azure Key Vault |
|
|
- **Kubernetes Integration** - Secrets, ConfigMaps, operators |
|
|
- **CI/CD Pipelines** - GitHub Actions, Jenkins, GitLab CI |
|
|
- **Python SDK** - Implementation guidance, best practices |
|
|
- **Security Patterns** - Secret rotation, access controls, compliance |
|
|
|
|
|
### Example Queries |
|
|
|
|
|
``` |
|
|
"How do I set up AnySecret with AWS Secrets Manager?" |
|
|
"Show me how to use anysecret in a GitHub Actions workflow" |
|
|
"How do I rotate secrets across multiple cloud providers?" |
|
|
"What's the difference between storing secrets vs parameters?" |
|
|
"How do I configure AnySecret for a Kubernetes deployment?" |
|
|
``` |
|
|
|
|
|
## 🏗️ Training Details |
|
|
|
|
|
### Training Data |
|
|
|
|
|
Models were trained on **150+ curated examples** across 7 categories: |
|
|
- **CLI Commands** (25 examples) - Command usage and patterns |
|
|
- **AWS Configuration** (25 examples) - Secrets Manager integration |
|
|
- **GCP Configuration** (25 examples) - Secret Manager setup |
|
|
- **Azure Configuration** (25 examples) - Key Vault integration |
|
|
- **Kubernetes** (25 examples) - Secrets and ConfigMaps |
|
|
- **CI/CD Integration** (15 examples) - Pipeline workflows |
|
|
- **Python Integration** (10 examples) - SDK usage patterns |
|
|
|
|
|
### Training Configuration |
|
|
|
|
|
#### Hyperparameters |
|
|
- **LoRA Rank:** 16 |
|
|
- **LoRA Alpha:** 32 |
|
|
- **Learning Rate:** 2e-4 |
|
|
- **Batch Size:** 1 (with gradient accumulation) |
|
|
- **Epochs:** 2-3 |
|
|
- **Precision:** fp16 mixed precision with 4-bit quantization |
|
|
|
|
|
#### Target Modules |
|
|
- **Llama-3.2 (3B):** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
|
- **CodeLlama (7B/13B):** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
|
|
|
|
## 🔧 Model Selection Guide |
|
|
|
|
|
### Choose 3B if you need: |
|
|
- ✅ Fast inference (< 1 second) |
|
|
- ✅ Low memory usage (4-6GB) |
|
|
- ✅ Edge deployment |
|
|
- ✅ Basic AnySecret queries |
|
|
|
|
|
### Choose 7B if you need: |
|
|
- ✅ Balanced performance/speed |
|
|
- ✅ Better code understanding |
|
|
- ✅ Moderate memory (8-12GB) |
|
|
- ✅ Complex configuration queries |
|
|
|
|
|
### Choose 13B if you need: |
|
|
- ✅ Highest quality responses |
|
|
- ✅ Complex multi-step guidance |
|
|
- ✅ Advanced troubleshooting |
|
|
- ✅ Production deployments |
|
|
|
|
|
## 🚀 Deployment Options |
|
|
|
|
|
### Local Development |
|
|
- **GGUF + Ollama:** Easiest setup, good performance |
|
|
- **PyTorch + GPU:** Best quality, requires CUDA |
|
|
|
|
|
### Production Deployment |
|
|
- **Docker + llama.cpp:** Scalable, CPU/GPU support |
|
|
- **Kubernetes:** Auto-scaling, load balancing |
|
|
- **Cloud APIs:** Serverless, pay-per-use |
|
|
|
|
|
### Memory Requirements |
|
|
|
|
|
| Model | GGUF Q4_K_M | GGUF Q8_0 | PyTorch FP16 | |
|
|
|-------|-------------|-----------|--------------| |
|
|
| 3B | 2.3GB | 3.2GB | 6GB | |
|
|
| 7B | 4.1GB | 7.2GB | 14GB | |
|
|
| 13B | 7.8GB | 13.8GB | 26GB | |
|
|
|
|
|
## 📚 Model Sources |
|
|
|
|
|
- **Repository:** https://github.com/anysecret-io/anysecret-lib |
|
|
- **Documentation:** https://docs.anysecret.io |
|
|
- **Training Code:** https://github.com/anysecret-io/anysecret-llm |
|
|
- **Website:** https://anysecret.io |
|
|
|
|
|
## 🔍 Framework Versions |
|
|
|
|
|
- **PEFT:** 0.17.1+ |
|
|
- **Transformers:** 4.35.0+ |
|
|
- **PyTorch:** 2.0.0+ |
|
|
- **llama.cpp:** Latest |
|
|
- **Ollama:** 0.1.0+ |
|
|
|
|
|
## 📊 Performance Benchmarks |
|
|
|
|
|
| Model | Tokens/sec | Quality Score | Memory (GGUF Q4) | |
|
|
|-------|------------|---------------|------------------| |
|
|
| 3B | ~45 | 7.2/10 | 2.3GB | |
|
|
| 7B | ~25 | 8.5/10 | 4.1GB | |
|
|
| 13B | ~15 | 9.1/10 | 7.8GB | |
|
|
|
|
|
*Benchmarks run on RTX 3090 with GGUF Q4_K_M quantization* |
|
|
|
|
|
## ⚖️ License |
|
|
|
|
|
MIT License - See individual model folders for specific license details. |
|
|
|
|
|
--- |
|
|
|
|
|
For support, visit our [GitHub Issues](https://github.com/anysecret-io/anysecret-lib/issues) or [Documentation](https://docs.anysecret.io). |
