Upload folder using huggingface_hub

README.md

@@ -1,41 +1,101 @@
 ---
-base_model: TheDrummer/Behemoth-X-123B-v2
-base_model_relation: quantized
-quantized_by: ArtusDev
-tags:
-- exl3
+base_model:
+- mistralai/Mistral-Large-Instruct-2411
 ---
+# Join our Discord! https://discord.gg/BeaverAI
+## More than 7000 members strong 💪 A hub for users and makers alike!
+---
+## Drummer is open for work / employment (I'm a Software Engineer). Contact me through any of these channels: https://linktr.ee/thelocaldrummer
+### Thank you to everyone who subscribed through [Patreon](https://www.patreon.com/TheDrummer). Your support helps me chug along in this brave new world.
+
+### FAQ for those out-of-the-loop
+
+<details>
+  <summary>🐶 Who is Drummer?</summary>
 
-## EXL3 Quants of TheDrummer/Behemoth-X-123B-v2
+  Hi! I'm Drummer. I'm a Software Engineer with experience in JavaScript, Golang, Python, and generally engineering the crap out of things.
 
-EXL3 quants of [TheDrummer/Behemoth-X-123B-v2](https://huggingface.co/TheDrummer/Behemoth-X-123B-v2) using <a href="https://github.com/turboderp-org/exllamav3/">exllamav3</a> for quantization.
+  Why I'm in the AI space:
+  
+  - **Exploration:** Everyone is trying to figure out how AI works and what it's capable of. I am too - just not in creating the smartest, safest model at all costs.
+  - **Upskill:** The world is headed towards AI. It is here to stay. This has been my way of brushing up in this new form of computing.
+  - **Value:** I yearn to create value. I feel satisfaction and fulfillment in providing something meaningful for others.
+  - **Fun:** It's just fun using and making models. It's also fun coming up with theories and realizing them in practice (training AI).
 
-### Quants
-| Quant(Revision) | Bits per Weight | Head Bits |
-| -------- | ---------- | --------- |
-| [2.5_H6](https://huggingface.co/ArtusDev/TheDrummer_Behemoth-X-123B-v2-EXL3/tree/2.5bpw_H6) | 2.5 | 6 |
-| [3.0_H6](https://huggingface.co/ArtusDev/TheDrummer_Behemoth-X-123B-v2-EXL3/tree/3.0bpw_H6) | 3.0 | 6 |
-| [3.5_H6](https://huggingface.co/ArtusDev/TheDrummer_Behemoth-X-123B-v2-EXL3/tree/3.5bpw_H6) | 3.5 | 6 |
-| [4.0_H6](https://huggingface.co/ArtusDev/TheDrummer_Behemoth-X-123B-v2-EXL3/tree/4.0bpw_H6) | 4.0 | 6 |
-| [4.25_H6](https://huggingface.co/ArtusDev/TheDrummer_Behemoth-X-123B-v2-EXL3/tree/4.25bpw_H6) | 4.25 | 6 |
-| [5.0_H6](https://huggingface.co/ArtusDev/TheDrummer_Behemoth-X-123B-v2-EXL3/tree/5.0bpw_H6) | 5.0 | 6 |
-| [6.0_H6](https://huggingface.co/ArtusDev/TheDrummer_Behemoth-X-123B-v2-EXL3/tree/6.0bpw_H6) | 6.0 | 6 |
-| [8.0_H8](https://huggingface.co/ArtusDev/TheDrummer_Behemoth-X-123B-v2-EXL3/tree/8.0bpw_H8) | 8.0 | 8 |
+  I started my tuning venture back in mid-2024 when I wanted to improve its literary capabilities.
+  I've come a long way since then and I have branched out and specialized.
 
-### Downloading quants with huggingface-cli
+  Foundational models today are optimized for non-creative uses, and I believe there is a place for AI in creativity and entertainment.
+
+  I am here to take *the road less traveled by*.
+
+</details>
 
 <details>
-  <summary>Click to view download instructions</summary>
+  <summary>❓ What are my models like?</summary>
+
+  **Bottomline:** My models are usually geared towards creativity (writing & interactivity), usability (no censorship & overalignment), and entertainment!
+  
+  While intelligence, correctness, and problem solving are not my priority, they are still one of many qualities I want in my models. 
+  
+  The primary goal is to enhance the experience for users looking to use models for creative uses. 
+  
+  The *subqualities* I look for are the following:
+  
+  - **Alignment:** How far off is it from being a goody, syncopathic assistant-writer?
+  - **Writing:** Does it string together words and sentences in a pleasant & effective way like a wordsmith?
+  - **Dynamism:** How good is the AI at being compelling and intriguing in its storytelling?
+  - **Adherence:** Is it sticking to the prompt?
+  - **Knowledge:** Does it know about the world in both its truths and stories told?
+  - **Intelligence:** Can it handle nuance, complexity, and logic?
+  - **Formatting:** Is it biased towards writing paragraphs / lists or can it come up with its own?
+
+  If it doesn't excel in one of these qualities, or if it's overall mediocre for its size, then I would most likely reiterate until I get something right.
+
+</details>
+
+<details>
+  <summary>🦫 What is BeaverAI?</summary>
+
+  BeaverAI is a Discord server.
+</details>
+
+---
+
+[Drummer](https://huggingface.co/TheDrummer) proudly presents...
+
+# Behemoth X 123B v2 🦣
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/65f2fd1c25b848bd061b5c2e/XiBlPtMdK3kvqw2AcVbCa.png)
+
+## Usage
+
+- Mistral v7 (Non-Tekken) | (i.e., Mistral v3 + `[SYSTEM_PROMPT] `)
+- Non-reasoning model
+
+## Description
+
+Behemoth X is a finetune Mistral Large 2411. 
+X denotes a significant update in training (compared to its predecessors), pushing its capabilities further with updated knowledge and techniques.
+
+> Seems to pass a secrecy test in a few gens. Successfully tracked who share a secret among 6 characters and who don't. Really liking the prose. Logic is mostly decent.
+
+> so far its 5 out of 5. made me cry. would let it stab me in the feels again.
+
+> This is one of the moments where I really enjoyed reading the generation
+
+> Recall is fantastic in v2b. I had a response just now that pulled in like 20 minor details. It was nuts. I'm at ~100 gens in an RP now and v2b has been perfect throughout so far. Maybe you hit on some magic like midnight miqu, I dunno.
+
+> genuinely impressed with this model, gave it a starting prompt and it's been generating a coherent story for 19k tokens straight, and it doesn't feel like  there's much slop
 
-Install hugginface-cli:
+> Language choice is better than OG too. That's what Monstral gave that I liked so much. My card is an evil character and she is keeping it hidden so so well. Laying the trap slowly, gaining trust. It's just amazing to watch. If this keeps up this might be your best model ever imo
 
-```bash
-pip install -U "huggingface_hub[cli]"
-```
+> I mostly do mixed co-narrator/mc RP content on my phone. I'm so deep into one storyline it takes a minute of furious scrolling to reach back at the top, and it's still going coherent. No templates, just a lightweight sys prompt. Great model, I'd hate to go back to 24B or even 70B from here. It recognized the video game Katawa Shoujo and incorporated character flaws seamlessly into my story.
 
-Download quant by targeting the specific quant revision (branch):
+## Links
+- Original: https://huggingface.co/TheDrummer/Behemoth-X-123B-v2
+- GGUF: https://huggingface.co/TheDrummer/Behemoth-X-123B-v2-GGUF
+- iMatrix (recommended): https://huggingface.co/bartowski/TheDrummer_Behemoth-X-123B-v2-GGUF
+- EXL3: https://huggingface.co/ArtusDev/TheDrummer_Behemoth-X-123B-v2-EXL3
 
-```
-huggingface-cli download ArtusDev/TheDrummer_Behemoth-X-123B-v2-EXL3 --revision "5.0bpw_H6" --local-dir ./
-```
-</details>
+`config-v2b`

chat_template.jinja

@@ -0,0 +1 @@
+{{ bos_token }}{% for message in messages %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + '[/INST]' }}{% elif message['role'] == 'system' %}{{ '[SYSTEM_PROMPT] ' + message['content'] + '[/SYSTEM_PROMPT]' }}{% elif message['role'] == 'assistant' %}{{ ' ' + message['content'] + eos_token }}{% else %}{{ raise_exception('Only user, system and assistant roles are supported!') }}{% endif %}{% endfor %}

config.json

@@ -0,0 +1,37 @@
+{
+    "architectures": [
+        "MistralForCausalLM"
+    ],
+    "attention_dropout": 0.0,
+    "bos_token_id": 1,
+    "eos_token_id": 2,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 12288,
+    "initializer_range": 0.02,
+    "intermediate_size": 28672,
+    "max_position_embeddings": 131072,
+    "model_type": "mistral",
+    "num_attention_heads": 96,
+    "num_hidden_layers": 88,
+    "num_key_value_heads": 8,
+    "rms_norm_eps": 1e-05,
+    "rope_theta": 1000000.0,
+    "sliding_window": null,
+    "tie_word_embeddings": false,
+    "torch_dtype": "bfloat16",
+    "transformers_version": "4.55.2",
+    "use_cache": true,
+    "vocab_size": 32768,
+    "quantization_config": {
+        "quant_method": "exl3",
+        "version": "0.0.7",
+        "bits": 3.75,
+        "head_bits": 6,
+        "calibration": {
+            "rows": 100,
+            "cols": 2048
+        },
+        "out_scales": "auto"
+    }
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "do_sample": true,
+  "eos_token_id": 2,
+  "transformers_version": "4.55.2"
+}

model-00001-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e129a1f604bbf7c670191e8158a060f007c063333d8cb30ba2b9253f4fe395f
+size 8586933904

model-00002-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0fbac2043fbb6a1a2a73e5c107717adc371c11ea65f32d891d9057dca49ed569
+size 8439532896

model-00003-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d1c64f77f8b4c1043f0b0fa1fcee37b1dec90c202709ba02baf54aa3caa232f
+size 8439532896

model-00004-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:408497b285b7243706fb9410918d014849c1f789448b534381432ab70d38a686
+size 8433241440

model-00005-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f5cfc1655630f74924c55f00eced34c0b7d79dc06da9dde3e7a18c111bc40e1
+size 8439532904

model-00006-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e97c448832b753eda787f5b9ce2d0ac4585bb35bb9cede0d4a296f00e610ad9
+size 8439532904

model-00007-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:865b6401b2f576a51b92005de1c15d9a7a976365315987237fdd87aea1175fa5
+size 7452564320

special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b968b8dc352f42192367337c78ccc61e1eaddc6d641a579372d4f20694beb7a
+size 587562