# GGUF Quantized Models

This repository contains GGUF quantized versions of language models for efficient inference.

## Files

The following GGUF files are available in this repository:

- `DeepHermes-3-Mistral-24B-Preview-q4.gguf` (13G)
- `DeepHermes-3-Mistral-24B-Preview-q5.gguf` (16G)
- `DeepHermes-3-Mistral-24B-Preview-q6.gguf` (19G)
- `DeepHermes-3-Mistral-24B-Preview-q8.gguf` (24G)

## Usage

These models are compatible with [llama.cpp](https://github.com/ggerganov/llama.cpp) and similar frameworks.

Example usage with llama.cpp:
```bash
./main -m /path/to/model.gguf -p "Hello, I am a language model" -n 128
```

## Upload Information
Files were uploaded on Tue Mar 11 03:48:02 PDT 2025