# ELISARCyberAIEdge7B-LoRA-GGUF

[![GGUF](https://img.shields.io/badge/format-GGUF-blue.svg)](https://gguf.io/)
**Offline-ready, quantized LLaMA edge model for cybersecurity use cases**

---

🏷️ **Name**: ELISARCyberAIEdge7B-LoRA-GGUF
👤 **Author**: Dr. Sabri Sallani, PhD (AI & Cybersecurity Expert)
📅 **Date**: 2025-05-31
🔗 **Repository**: [https://huggingface.co/sallani/ELISARCyberAIEdge7B-LoRA-GGUF](https://huggingface.co/sallani/ELISARCyberAIEdge7B-LoRA-GGUF)

---

## 📖 Overview

ELISARCyberAIEdge7B-LoRA-GGUF is a **LoRA-finetuned**, **GGUF-quantized** version of the Mistral-7B backbone tailored for **edge deployment in cybersecurity and blue-team AI scenarios**. Developed by Dr. Sabri Sallani (PhD), this model integrates:

1. **Base model**: Mistral-7B-v0.3 (FP16 / BF16)
2. **LoRA adapter**: `sallani/ELISARCyberAIEdge7B`
3. **Quantization**: Converted to GGUF format and optionally quantized to Q4\_K\_M (4-bit) for efficient inference on resource-constrained devices (NVIDIA T4, desktop GPUs, etc.).

This pipeline produces a single file (`elisar_merged.gguf`) of \~160 MiB that you can deploy **offline** using frameworks like [`llama.cpp`](https://github.com/ggml-org/llama.cpp) or run through minimal Torch-based inference.

**Key features:**

* **Compact (< 200 MiB)** quantized GGUF file
* **Edge-friendly**: runs on CPU or low-end GPUs with fast cold-start
* **Cybersecurity-tuned**: trained to answer cybersecurity questions, perform log analysis, malware triage, and blue-team playbooks
* **Offline inference**: execute entirely without internet access

---

## 🚀 Quickstart

### 1. Download model files

```bash
# Clone or download the GGUF file directly:
wget https://huggingface.co/sallani/ELISARCyberAIEdge7B-LoRA-GGUF/resolve/main/elisar_merged.gguf -O elisar_merged.gguf
```

Alternatively, using the Hugging Face Hub CLI:

```bash
pip install huggingface_hub
huggingface-cli login  # enter HF_TOKEN
huggingface-cli repo clone sallani/ELISARCyberAIEdge7B-LoRA-GGUF
cd ELISARCyberAIEdge7B-LoRA-GGUF
tree
# ├── elisar_merged.gguf
# └── README.md
```

---

## 💿 Installation

#### 1. llama.cpp (Offline inference)

```bash
# Clone llama.cpp repository (if not already):
git clone --depth 1 https://github.com/ggml-org/llama.cpp.git
cd llama.cpp

# Build with GPU support (optional)
# Requires CUDA toolkit if targeting NVIDIA GPU (e.g., T4)
make clean
make CMAKE_CUDA=ON CMAKE_CUDA_ARCH=sm75

# Or build CPU-only:
# make
```

#### 2. Python (Transformers) – Optional hybrid inference

```bash
# Create a virtual environment (recommended)
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install torch transformers peft
```

---

## ⚡️ Usage Examples

### A. Offline inference with `llama.cpp`

```bash
# Assuming llama.cpp built and elisar_merged.gguf is in current directory:
cd llama.cpp

# Run Chat UI (console) with GGUF:
./main -m ../ELISARCyberAIEdge7B-LoRA-GGUF/elisar_merged.gguf -c 2048 -b 8 -t 8

# Example prompt (after startup):
> Hello, how can I assist in analyzing a suspicious log entry?
```

**Key flags:**

* `-m <path>`: points to `elisar_merged.gguf`
* `-c <ctx>`: context window (e.g., 2048 tokens)
* `-b <batch>`: batch size for token sampling
* `-t <threads>`: CPU threads

### B. Python / Transformers + PEFT Inference (Hybrid)

If you prefer a Python environment for more complex pipelines:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig
import torch

# 1️⃣ Load GGUF via `transformers` (requires `transformers>=4.34` + `gguf-py`)
model_id = "sallani/ELISARCyberAIEdge7B-LoRA-GGUF"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",  # automatically places on GPU if available
)

# 2️⃣ Prepare a cybersecurity prompt
prompt = "You are a blue-team AI assistant. Analyze the following network log for suspicious patterns: ..."
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

gen_config = GenerationConfig(
    temperature=0.7,
    top_p=0.9,
    max_new_tokens=256,
)
output_ids = model.generate(**inputs, **gen_config.to_dict())
answer = tokenizer.decode(output_ids[0], skip_special_tokens=True)
print(answer)
```

---

## 📦 File Structure

```
ELISARCyberAIEdge7B-LoRA-GGUF/
├── elisar_merged.gguf      # < 200 MiB quantized model (LoRA + base fused)
└── README.md               # This readme file
```

---

## 🔧 Model Details & Training

* **Base**: Mistral-7B-v0.3 (scaled to 7 billion params, FP16/BF16)
* **LoRA adapter**: Custom LoRA weights from `sallani/ELISARCyberAIEdge7B` about cybersecurity conversational tasks
* **Quantization**: GGUF format produced via `convert_lora_to_gguf.py` from `llama.cpp`; final file is \~160 MiB
* **Finetuning data**: Internal blue-team playbooks, anonymized security logs, vulnerability descriptions, attack/defense dialogues
* **License**: \[Add your license text here]

> *Developed by Dr. Sabri Sallani, PhD – Expert in Artificial Intelligence & Cybersecurity.*

---

## 📜 Prompt Guidelines

* **Instruction style**: Pose direct cybersecurity questions (e.g., “Analyze this log”, “Suggest mitigation steps”, “Explain vulnerability CVE-XXXX”).
* **Context**: Provide relevant log snippets, code blocks, or short descriptions of network events.
* **Limitations**: This model excels at blue-team guidance but is not a replacement for professional incident response. Always verify critical actions manually.

---

## 🤝 Citations & Licensing

If you use or reference **ELISARCyberAIEdge7B-LoRA-GGUF** in your work, please cite:

> Sallani, S. (2025). *ELISARCyberAIEdge7B-LoRA-GGUF*: Edge-optimized cybersecurity AI model. [https://huggingface.co/sallani/ELISARCyberAIEdge7B-LoRA-GGUF](https://huggingface.co/sallani/ELISARCyberAIEdge7B-LoRA-GGUF)

---

## 💬 Support & Contact

* 🔗 **Hugging Face Discussion**: [Spaces → Community](https://huggingface.co/sallani/ELISARCyberAIEdge7B-LoRA-GGUF/discussions)
* 📧 **Email**: [dr.sallani@example.com](mailto:dr.sallani@CyberLand.com)
* 📄 **Website/Portfolio**: [https://www.linkedin.com/in/sabri-allani/)

Feel free to raise issues or file enhancement requests on the Hugging Face repository.

---

*Thank you for using ELISARCyberAIEdge7B-LoRA-GGUF – Best of luck in securing your edge deployments!*