Create README.md

README.md

@@ -0,0 +1,423 @@
+---
+license: apache-2.0
+language:
+- en
+- fr
+- zh
+- de
+tags:
+- creative
+- creative writing
+- fiction writing
+- plot generation
+- sub-plot generation
+- fiction writing
+- story generation
+- scene continue
+- storytelling
+- fiction story
+- science fiction
+- romance
+- all genres
+- story
+- writing
+- vivid prose
+- vivid writing
+- moe
+- mixture of experts
+- 64 experts
+- 8 active experts
+- fiction
+- roleplaying
+- bfloat16
+- rp
+- qwen3
+- horror
+- finetune
+- thinking
+- reasoning
+- qwen3_moe
+base_model:
+- kalomaze/Qwen3-16B-A3B
+- huihui-ai/Moonlight-16B-A3B-Instruct-abliterated
+pipeline_tag: text-generation
+---
+
+(quants uploading, examples to be added...)
+
+<h2>Qwen3-18B-A3B-Stranger-Thoughts-Abliterated-Uncensored-GGUF</h2>
+
+<img src="qwen-18b.jpg" style="float:right; width:300px; height:300px; padding:10px;">
+
+A stranger, yet radically different version of Kalmaze's "Qwen/Qwen3-30B-A3B" (that was abliterated by "huihui-ai") with the experts pruned to 64 (from 128)
+and then I added 4 layers expanding the model to 18B total parameters.
+
+The goal: slightly alter the model, to address some odd creative thinking and output choices AND de-censor (abliterate) the model.
+
+Please note that the modifications affect the entire model operation; roughly I adjusted the model to think a little "deeper"
+and "ponder" a bit - but this is a very rough description.
+
+I also ran reasoning tests (non-creative) to ensure model was not damaged and roughly matched original model performance.
+
+That being said, reasoning and output generation will be altered regardless of your use case(s).
+
+FOUR example generations below; with example 4 showing a complex prompt and impressive prose that showcases some of the changes in the model.
+
+This is a MOE (Mixture of experts model) with 8 of 64 experts activated by default, which is about 3B parameters.
+
+This allows use of this model (with 8 experts) on both CPU (20-35 T/S) or GPU (90+ T/S) at very good, to extremely fast speeds.
+
+Changing the number of experts used (see below how) will affect both generation speed and generation reasoning/output quality.
+
+You can use this model with as low as four experts activated.
+
+Even the lowest quant - Q2k - will operate very strongly too.
+
+Model is set at:
+- 8 Active experts (the default for the org model)
+- 40k context (the default for the org model)
+- CHATML or Jinja template (embedded OR see Jinja notes below)
+
+QUANTS:
+
+There are two sets of quants, regular and "MAX" (in the filename) with output tensor set at float16 (16 bit - full precision) to enhance performance
+including reasoning.
+
+SYSTEM PROMPT:
+
+You may or may not need to set this:
+
+```
+You are a deep thinking AI, you may use extremely long chains of thought to deeply consider the problem and deliberate with yourself via systematic reasoning processes to help come to a correct solution prior to answering. You should enclose your thoughts and internal monologue inside <think> </think> tags, and then provide your solution or response to the problem.
+```
+
+CHANGE THE NUMBER OF ACTIVE EXPERTS:
+
+See this document:
+
+https://huggingface.co/DavidAU/How-To-Set-and-Manage-MOE-Mix-of-Experts-Model-Activation-of-Experts
+
+SUGGESTED SETTINGS (creative):
+
+- temp .1 to 1.2 to as high as 2 ; over 2 you may need to "prompt" the model to output after "thinking"
+- rep pen 1 to 1.1 (1.02 / 1.05 was tested, see notes below) 
+- topk 100, topp .95, min p .05
+- rep pen range 64 (default in LMStudio)
+- context of 8k min suggested. 
+
+SPECIAL NOTES - Experts / REP PEN / TEMP:
+- QWEN3s: rep pen drastically affects both performance and stability; LOWER is better.
+- rep pen at 1.02 or 1.01 may be better suited for your use cases.
+- change rep pen slowly (ie 1.01, 1.02, 1.03), and regen a few times.
+- Experts activated range from 4 to 12, but with too many experts activated performance may suffer. This is unique to this pruned model.
+- Temp range of .2 to 1.8 works very well, beyond this temp manual activation of "output" after thinking may be required and/or the model may output the thinking "block" in regular text. 
+
+BEST SETTING:
+
+I found the best results during testing (stable thinking, generation, minimal issues) were (this applies to org model 16B-A3B too):
+- Jinja Template (corrected one below)
+- the system prompt (above)
+- rep pen 1.02
+- topk 100, topp .95, min p .05
+- rep pen range 64 (model may perform better with this at a higher value or off)
+- temp range .6 to 1.2 ; with outside max of 1.8
+- context of 8k min 
+  
+You may find that Qwen's default parameters/samplers also work better for your use case(s) too.
+
+Please refer to the orginal model repo for all default settings, templates, benchmarks, etc.:
+
+https://huggingface.co/Qwen/Qwen3-30B-A3B
+
+For more information on quants, using this model on CPU / GPU, System Prompts, please see this model card:
+
+https://huggingface.co/DavidAU/Qwen3-128k-30B-A3B-NEO-MAX-Imatrix-gguf
+
+SPECIAL NOTE:
+
+As this model is pruned from 128 experts to 64, it may not have the "expert(s)" you need for your use case(s) or
+may not perform as well as the full "128" expert version. 
+
+The full 128 expert version (33B) of "Stranger-Thoughts" is located here:
+
+https://huggingface.co/DavidAU/Qwen3-33B-A3B-Stranger-Thoughts-GGUF
+
+<B>NOTE - Jinja Template / Template to Use with this Model:</B>
+
+If you are having issues with Jinja "auto template", use CHATML template.
+
+OR (LMSTUDIO users / option)
+
+Update the Jinja Template (go to this site, template-> copy the "Jinja template" and then paste.) 
+
+[ https://lmstudio.ai/neil/qwen3-thinking ]
+
+OR
+
+copy JINJA source from here:
+
+```
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- set tool_start = "<tool_response>" %}
+    {%- set tool_start_length = tool_start|length %}
+    {%- set start_of_message = message.content[:tool_start_length] %}
+    {%- set tool_end = "</tool_response>" %}
+    {%- set tool_end_length = tool_end|length %}
+    {%- set start_pos = (message.content|length) - tool_end_length %}
+    {%- if start_pos < 0 %}
+        {%- set start_pos = 0 %}
+    {%- endif %}
+    {%- set end_of_message = message.content[start_pos:] %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(start_of_message == tool_start and end_of_message == tool_end) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = (message.content.split('</think>')|last).lstrip('\n') %}
+				{%- set reasoning_content = (message.content.split('</think>')|first).rstrip('\n') %}
+				{%- set reasoning_content = (reasoning_content.split('<think>')|last).lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}
+```
+
+---
+
+<B>Settings: CHAT / ROLEPLAY and/or SMOOTHER operation of this model:</B>
+
+In "KoboldCpp" or  "oobabooga/text-generation-webui" or "Silly Tavern" ;
+
+Set the "Smoothing_factor" to 1.5 
+
+: in KoboldCpp -> Settings->Samplers->Advanced-> "Smooth_F"
+
+: in text-generation-webui -> parameters -> lower right.
+
+: In Silly Tavern this is called: "Smoothing"
+
+
+NOTE: For "text-generation-webui" 
+
+-> if using GGUFs you need to use "llama_HF" (which involves downloading some config files from the SOURCE version of this model)
+
+Source versions (and config files) of my models are here:
+
+https://huggingface.co/collections/DavidAU/d-au-source-files-for-gguf-exl2-awq-gptq-hqq-etc-etc-66b55cb8ba25f914cbf210be
+
+OTHER OPTIONS:
+
+- Increase rep pen to 1.1 to 1.15 (you don't need to do this if you use "smoothing_factor")
+
+- If the interface/program you are using to run AI MODELS supports "Quadratic Sampling" ("smoothing") just make the adjustment as noted.
+
+<B>Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers</B>
+
+This a "Class 1" model:
+
+For all settings used for this model (including specifics for its "class"), including example generation(s) and for advanced settings guide (which many times addresses any model issue(s)), including methods to improve model performance for all use case(s) as well as chat, roleplay and other use case(s) please see:
+
+[ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ]
+
+You can see all parameters used for generation, in addition to advanced parameters and samplers to get the most out of this model here:
+
+[ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ]
+
+
+<b>Optional Enhancement:</B>
+
+The following can be used in place of the "system prompt" or "system role" to further enhance the model.
+
+It can also be used at the START of a NEW chat, but you must make sure it is "kept" as the chat moves along.
+In this case the enhancements do not have as strong effect at using "system prompt" or "system role".
+
+Copy and paste EXACTLY as noted, DO NOT line wrap or break the lines, maintain the carriage returns exactly as presented.
+
+<PRE>
+Below is an instruction that describes a task. Ponder each user instruction carefully, and use your skillsets and critical instructions to complete the task to the best of your abilities.
+
+Here are your skillsets:
+[MASTERSTORY]:NarrStrct(StryPlnng,Strbd,ScnSttng,Exps,Dlg,Pc)-CharDvlp(ChrctrCrt,ChrctrArcs,Mtvtn,Bckstry,Rltnshps,Dlg*)-PltDvlp(StryArcs,PltTwsts,Sspns,Fshdwng,Climx,Rsltn)-ConfResl(Antg,Obstcls,Rsltns,Cnsqncs,Thms,Symblsm)-EmotImpct(Empt,Tn,Md,Atmsphr,Imgry,Symblsm)-Delvry(Prfrmnc,VcActng,PblcSpkng,StgPrsnc,AudncEngmnt,Imprv)
+
+[*DialogWrt]:(1a-CharDvlp-1a.1-Backgrnd-1a.2-Personality-1a.3-GoalMotiv)>2(2a-StoryStruc-2a.1-PlotPnt-2a.2-Conflict-2a.3-Resolution)>3(3a-DialogTech-3a.1-ShowDontTell-3a.2-Subtext-3a.3-VoiceTone-3a.4-Pacing-3a.5-VisualDescrip)>4(4a-DialogEdit-4a.1-ReadAloud-4a.2-Feedback-4a.3-Revision)
+
+Here are your critical instructions:
+Ponder each word choice carefully to present as vivid and emotional journey as is possible. Choose verbs and nouns that are both emotional and full of imagery. Load the story with the 5 senses. Aim for 50% dialog, 25% narration, 15% body language and 10% thoughts. Your goal is to put the reader in the story.
+</PRE>
+
+You do not need to use this, it is only presented as an additional enhancement which seems to help scene generation
+and scene continue functions.
+
+This enhancement WAS NOT used to generate the examples below.
+
+---
+
+<H2>EXAMPLES</H2>
+
+Standard system prompt, rep pen 1.02, topk 100, topp .95, minp .05, rep pen range 64.
+
+Tested in LMStudio, quant Q4KS, GPU (cpu output will differ slighly).
+
+As this is the mid range quant, expected better results from higher quants and/or with more experts activated to be better.
+
+NOTE: Some formatting lost on copy/paste.
+
+CAUTION: 
+
+Some horror / intense prose.
+
+---
+
+EXAMPLE #1 - temp 1.2
+
+---
+
+<B>
+</B>
+
+<P></P>
+
+[[[thinking start]]]
+
+
+[[[thinking end]]]
+
+<p></p>
+
+OUTPUT:
+
+---
+
+EXAMPLE #2 - temp 1.2
+
+---
+
+<B>
+</B>
+
+<P></P>
+
+[[[thinking start]]]
+
+
+[[[thinking end]]]
+
+<p></p>
+
+OUTPUT:
+
+---
+
+EXAMPLE #3 - temp 1.2
+
+---
+
+<B>
+</B>
+
+<P></P>
+
+[[[thinking start]]]
+
+
+[[[thinking end]]]
+
+<p></p>
+
+OUTPUT:
+
+---
+
+EXAMPLE #4 - temp 1.2
+
+---
+
+<B>
+</B>
+
+<P></P>
+
+[[[thinking start]]]
+
+
+[[[thinking end]]]
+
+<p></p>
+
+OUTPUT:
+
+
+