TinyGuard1-22M

A small (22.7M parameters) English text classifier for basic security filtering. It labels a message as one of four classes: safe, spam, phishing or prompt_injection. It is a fine-tune of MiniLM-L6-H384-uncased and runs comfortably on CPU or a phone-class device.

Results

Evaluated on a held-out test split (stratified, de-duplicated, 1301 examples) of the combined training data.

Metric Score
Accuracy 0.9669
Macro F1 0.9696
Weighted F1 0.9668
Class Precision Recall F1 Support
safe 0.9601 0.9796 0.9698 491
spam 0.9679 0.9837 0.9758 307
phishing 0.9671 0.9255 0.9458 349
prompt_injection 0.9870 0.9870 0.9870 154

Usage

from transformers import pipeline
clf = pipeline("text-classification", model="Kuantum-ai1/TinyGuard1-22M")
print(clf("Congratulations! You won a free prize, click here to claim it."))

Training data

Public Hugging Face datasets: SMS spam (ucirvine/sms_spam), email spam (Deysi/spam-detection-dataset), phishing texts (ealvaradob/phishing-dataset), prompt injection (xTRam1/safe-guard-prompt-injection) and benign instructions (databricks/databricks-dolly-15k) as additional safe examples. Classes were capped at 10000 examples each and de-duplicated.

Class Examples (before split)
safe 9819
phishing 6972
spam 6142
prompt_injection 3072

Training setup

Fine-tuning with PyTorch on one T4 GPU: max length 256, batch size 64, learning rate 5e-5, 3 epochs, AdamW (weight decay 0.01), linear schedule with 6% warmup, fp16. The checkpoint with the best validation accuracy was kept.

Limitations

  • English only. Not evaluated on adversarial or obfuscated attacks.
  • Trained on public corpora; real-world traffic can differ and accuracy may be lower.
  • Class balance and domain mix come from the training data and may not match your use case.
  • It is a lightweight first-pass filter, not a complete security solution. Review the licenses of the training datasets before commercial use.

License

Apache-2.0

Downloads last month
36
Safetensors
Model size
22.7M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Kuantum-ai1/TinyGuard1-22M

Finetuned
(26)
this model

Datasets used to train Kuantum-ai1/TinyGuard1-22M

Collection including Kuantum-ai1/TinyGuard1-22M