unsloth-QwQ-32B-gguf-japanese-imatrix

Qwen/QwQ-32B は人によって評価がかなり分かれています。これは、パラメータ―に敏感である事が影響しているようです。
このgguf版は量子化パラメータ―を改良し、日本語能力を向上させたggufの作成を目指したものです
詳細は検証中です

Qwen/QwQ-32B has received very mixed reviews from people. This is likely due to its sensitivity to parameters.
This gguf version aims to create a gguf with improved quantization parameters and improved Japanese language capabilities.
Details are under verification.

example

おそらくllama-serverの仕様が変わったためか、私はllama-server経由で正常に動かす事がまだできていません（繰り返しが発生してしまう)
Probably because the llama-server specifications have changed, I have not yet been able to get it to work properly via llama-server (repetition occurs).

しかし、llma-cli経由では以下のように正常に出力ができています。以下はwindowsのamd vulkanサポートでコンパイルしたllama-cliコマンドのサンプルです
However, the output is normally output via llma-cli as shown below. Below is a command sample for Windows compiled with vulkan support.

C:\work\llama.cpp_quant\qwq>..\vulkan\llama.cpp\build\bin\llama-cli ^
More?     --model QwQ-32B-Q4_K-f16.gguf ^
More?     --threads 21 ^
More?     --ctx-size 4096 ^
More?     --n-gpu-layers 10 ^
More?     --seed 3407 ^
More?     --prio 2 ^
More?     --temp 0.6 ^
More?     --repeat-penalty 1.0 ^
More?     --dry-multiplier 0.5 ^
More?     --min-p 0.00 ^
More?     --top-k 40 ^
More?     --top-p 0.95 ^
More?     -no-cnv ^
More?     --samplers "top_k;top_p;min_p;temperature;dry;typ_p;xtc" ^
More?     --prompt "<|im_start|>user\nPlease write an introduction in Japanese for a novel set in another world that is likely to be popular enough to be made into a TV anime. The main character is Suzuki Kento, a businessman in his 40s. He has a special skill that won him the world championship in kabaddi. The writing style should be casual, like a light novel. Please start with a scene where the main character has just been reincarnated in another world and is in a pinch surrounded by goblins. The story should develop in a way that readers who do not know kabaddi can understand the appeal of kabaddi. Kabaddi is a sport that originated in India. Players charge into the opponent's territory while holding their breath, and score by touching the opponent while chanting 'kabaddi, kabaddi' repeatedly and returning to their own territory. The defending team tries to get in the way of the attacker to allow them to take a breath. The best part of this sport is the offensive and defensive battle while holding your breath. The other world is at the cultural level of medieval Europe, so there is no sport called kabaddi, but you should come up with an original setting where the rules and characteristics of this sport lead to an overwhelming advantage in the other world. The main character's lung capacity will surely be excellent. You can also add a fantasy setting. The parts that require judgment are left to your preference. Please write without fear of failure and without overthinking. Please output in Japanese.<|im_end|>\n<|im_start|>assistant\n<think>"

note
vulcan用のllama.cppを使っていない場合は--n-gpu-layers 10 ^の行を消す事
If you are not using llama.cpp compiled with vulcan, delete --n-gpu-layers 10 ^ line.

output log

ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon 780M Graphics (AMD proprietary driver) | uma: 1 | fp16: 1 | warp size: 64 | shared memory: 32768 | matrix cores: KHR_coopmat
build: 4883 (84d54755) with cc (GCC) 14.2.0 for x86_64-w64-mingw32
main: llama backend init
main: load the model and apply lora adapter, if any
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon 780M Graphics) - 3840 MiB free
...

system_info: n_threads = 21 (n_threads_batch = 21) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | AARCH64_REPACK = 1 |

sampler seed: 3407
sampler params:
        repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
        dry_multiplier = 0.500, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 4096
        top_k = 40, top_p = 0.950, min_p = 0.000, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.600
        mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
sampler chain: logits -> logit-bias -> top-k -> top-p -> min-p -> temp-ext -> dry -> typical -> xtc -> dist
generate: n_ctx = 4096, n_batch = 2048, n_predict = -1, n_keep = 0

user
Please write an introduction in Japanese for a novel set in another world that is likely to be popular enough to be made into a TV anime. The main character is Suzuki Kento, a businessman in his 40s. He has a special skill that won him the world championship in kabaddi. The writing style should be casual, like a light novel. Please start with a scene where the main character has just been reincarnated in another world and is in a pinch surrounded by goblins. The story should develop in a way that readers who do not know kabaddi can understand the appeal of kabaddi. Kabaddi is a sport that originated in India. Players charge into the opponent's territory while holding their breath, and score by touching the opponent while chanting 'kabaddi, kabaddi' repeatedly and returning to their own territory. The defending team tries to get in the way of the attacker to allow them to take a breath. The best part of this sport is the offensive and defensive battle while holding your breath. The other world is at the cultural level of medieval Europe, so there is no sport called kabaddi, but you should come up with an original setting where the rules and characteristics of this sport lead to an overwhelming advantage in the other world. The main character's lung capacity will surely be excellent. You can also add a fantasy setting. The parts that require judgment are left to your preference. Please write without fear of failure and without overthinking. Please output in Japanese.
assistant
<think>
Alright, the user wants an introduction for a Japanese light novel that's likely to be adapted into a TV anime. The main
 character is a 40s businessman named Suzuki Kento who's a kabaddi world champion. The scene starts with him being reincarnated and surrounded by goblins. I need to make sure the kabaddi elements are explained without assuming prior knowledge.

First, the opening scene needs to be gripping. Kento wakes up in another world, surrounded by goblins. Since he's a kabaddi player, his skills should give him an advantage here. The setting is medieval Europe-like, so there's no kabaddi there. I should think of how his breathing technique and combat skills from kabaddi can be applied in this fantasy context.

I should explain kabaddi's rules through the action. Maybe he uses his lung capacity to hold his breath during a fight, which confuses the enemies. The defense part of kabaddi, where defenders block the attacker, can translate into the goblins trying to make him breathe. Need to make the battle dynamic and showcase his skills.

Also, the writing style should be casual, like a light novel, so the language should be straightforward and engaging. Maybe start with him waking up confused, then the goblins attack, and he instinctively uses kabaddi moves. Highlighting his lung capacity and the chant "kabaddi" as a battle cry.

Fantasy elements can include magic or unique creatures. Maybe the goblins have some magical abilities that he counters with his skills. The key is to show how his sport expertise is a game-changer in this world. Need to ensure the appeal of
 kabaddi comes through naturally in the action without info-dumping.

I should also set up the world a bit―medieval Europe style, so maybe mention castles or knights later, but focus on the immediate danger first. The introduction should hook readers and introduce the protagonist's unique ability. Make sure the scene ends on a high note, showing his advantage.
</think>
【第一章 異世界転生とゴブリンの群れ】

「……くそ、まずいな」

スーツ姿の男は地面に手をついて四つん這いになり、周囲をぐるりと見回した。暗い森の木陰に佇むのは、ゴブリンだ。緑の皮膚と犬のような耳、
牙をむき出した小柄な魔物が、少なく見積もっても十体以上。彼の周囲を囲み、地面を蹴るようにして襲ってくる。

「おっさん、死んじまえー！」

一匹が跳躍してくる。スーツの男は反射的に体を丸めて回避し、同時に左腕を振りかぶった。

「……待てよ。俺、今どこに転生したんだっけ？」

頭が混乱する。四十二歳の会社員・鈴木健斗は、今朝も例の如く朝礼の資料を確認しながらオフィスに向かっていた。
それが突然、意識を失い、この異世界に転送された。

「カバディ……！」

ゴブリンの首根っこをつかみ、転がるように投げ飛ばす。

「いや、カバディの世界じゃねえよ！ ゴブリン相手に何やってんだ俺は……！」

息を殺し、両手で喉を押さえるゴブリンが地面に崩れ落ちる。だが残る魔物たちは怯まず、逆に笑い声を上げながら群がってくる。

「このクソったれども、なんで俺の動きが読めるんだ……！」

鈴木の脳裏に、十年前の世界大会決勝の記憶が蘇った。敵陣に突入した瞬間、相手の動きを読む感覚。呼吸を止め、『カバディ、カバディ』と唱える集中
力。あの時も、周囲がぐるりと回転するように見えた。

「……やるしかねえな」

深呼吸をした後、息を止める。肺が爆発するかと思えるまで空気を貯め込み、『カバディ』の呪文を口にした。

「……何を？ おっさん、バカか？」

ゴブリンたちは呆れた表情になる。しかし次の瞬間、鈴木の体が光った。

「……肺活量が……？」

「お前の『息』が、魔力の源なんだよ」

周囲の魔物が気づいたのは遅かった。鈴木は息を止めた状態で、体を蛇のようにねじりながらゴブリンの群れに突進。彼らの攻撃をかわし、触れた相手を
次々と放り投げる。最後尾のゴブリンを空中でつかみ、地面に叩きつけると、その体が粉々に砕け散った。

「……な、何が起こった……？」

生き残ったゴブリンが震えながら呟く。

「俺の仕事はカバディで世界一だったんだが、この世界じゃ『息を止めて相手を蹴散らす能力』ってことか。便利な転生だな」

鈴木はスーツの襟を整え、にやりと笑った。この世界で、彼の持つ『息』と『体捌き』が、新たな戦いの形を紡ぎ出すのは、時間の問題だ。

（続く）

---
**注記**
・カバディのルールを戦闘シーンに直結させ、呼吸制御や体術の描写でスポーツの魅力を提示。
・「息を止める＝魔力源」というファンタジー要素を追加し、異世界での優位性を演出。
・軽小説風のテンポと主人公のダサイ毒舌で親近感を演出。
・ゴブリンの反応で「息継ぎのタイミングを潰す防衛戦」を暗示し、後続の展開へつなげる。 [end of text]

llama_perf_sampler_print:    sampling time =      88.46 ms /  1682 runs   (    0.05 ms per token, 19013.81 tokens per second)
llama_perf_context_print:        load time =    6912.76 ms
llama_perf_context_print: prompt eval time =   17363.28 ms /   316 tokens (   54.95 ms per token,    18.20 tokens per second)
llama_perf_context_print:        eval time =  846536.19 ms /  1365 runs   (  620.17 ms per token,     1.61 tokens per second)
llama_perf_context_print:       total time =  864163.89 ms /  1681 tokens

currnet sample parameters.

temperature = 0.6
top-k = 40 (20 to 40 suggested)
min-p = 0.00 (optional, but 0.01 works well, llama.cpp default is 0.1)
top-p = 0.95
repetition-penalty = 1.0
dry-multiplier 0.5
Chat template: <|im_start|>user\nCreate a Flappy Bird game in Python.<|im_end|>\n<|im_start|>assistant\n<think>\n

Reference information
Tutorial: How to Run QwQ-32B effectively