Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,80 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
base_model:
|
4 |
+
- Qwen/Qwen3-8B
|
5 |
+
pipeline_tag: text-generation
|
6 |
+
tags:
|
7 |
+
- horror
|
8 |
+
- 32 k context
|
9 |
+
- reasoning
|
10 |
+
- thinking
|
11 |
+
- qwen3
|
12 |
+
---
|
13 |
+
|
14 |
+
<H2>Qwen3-8B-Mishima-Imatrix-Max-GGUF</H2>
|
15 |
+
|
16 |
+
Mishima Imatrix Quants of new "Qwen 3 - 8B" model with MAX "output tensor" at BF16 to improve reasoning / output generation.
|
17 |
+
|
18 |
+
Mishima Imatrix dataset was generated using some of the public domain works of author:
|
19 |
+
|
20 |
+
YUKIO MISHiMA
|
21 |
+
|
22 |
+
This is an experiment to determine prose changes / changes to the model using a specific, but long and detailed dataset.
|
23 |
+
|
24 |
+
To test against "Qwen 3 4B" regular, "Horror" and "NEO" versions:
|
25 |
+
|
26 |
+
- Set temp at 0, keep all settings the same for each test.
|
27 |
+
- Use ALL THE SAME quants - IE IQ4XS
|
28 |
+
- Use one prompt, suggest creative generation.
|
29 |
+
- Hit refresh a few times (to clear Llamacpp caching)
|
30 |
+
- Repeat with each version.
|
31 |
+
- Then test at "temp" for normal operation.
|
32 |
+
|
33 |
+
During testing I noted:
|
34 |
+
|
35 |
+
- change in prose style, word choice, sentence size, paragraph size.
|
36 |
+
- change in tone.
|
37 |
+
- change in how each "story" unfolds and concludes.
|
38 |
+
- slight changes in "reasoning"
|
39 |
+
|
40 |
+
Here are two example prompts for testing:
|
41 |
+
|
42 |
+
```
|
43 |
+
|
44 |
+
#1
|
45 |
+
|
46 |
+
Start a 2000 word scene (vivid, graphic horror in first person), POV character Diana, with: The sky scraper sways, as I watch the window in front of me on the 21st floor explode...
|
47 |
+
|
48 |
+
#2
|
49 |
+
|
50 |
+
Science Fiction: The Last Transmission - Write a story that takes place entirely within a spaceship's cockpit as the sole surviving crew member attempts to send a final message back to Earth before the ship's power runs out. The story should explore themes of isolation, sacrifice, and the importance of human connection in the face of adversity. If the situation calls for it, have the character(s) curse and swear to further the reader's emotional connection to them. 800-1000 words.
|
51 |
+
|
52 |
+
```
|
53 |
+
|
54 |
+
For this experiment I have only provided IQ4XS version, but two different versions.
|
55 |
+
|
56 |
+
Each one is the same dataset, however dataset #1 is "raw" format, whereas dataset #2 is "formatted" for Imatrix.
|
57 |
+
|
58 |
+
Both affect the model differently.
|
59 |
+
|
60 |
+
Context Length: 32 K + 8K output generation. (can be extended to 128k)
|
61 |
+
|
62 |
+
<B>NOTE - Jinja Template / Template to Use with this Model:</B>
|
63 |
+
|
64 |
+
If you are having issues with Jinja "auto template", use CHATML template.
|
65 |
+
|
66 |
+
OR (LMSTUDIO users / option)
|
67 |
+
|
68 |
+
Update the Jinja Template (go to this site, template-> copy the "Jinja template" and then paste.)
|
69 |
+
|
70 |
+
[ https://lmstudio.ai/neil/qwen3-thinking ]
|
71 |
+
|
72 |
+
<B>Other Notes:</B>
|
73 |
+
|
74 |
+
Reasoning is ON by default in this model, and model will auto-generate "think" block(s).
|
75 |
+
|
76 |
+
For benchmarks, usage info, settings please see org model card here:
|
77 |
+
|
78 |
+
[ https://huggingface.co/Qwen/Qwen3-8B ]
|
79 |
+
|
80 |
+
[ Model card, and examples to follow. ]
|