Instructions to use UnderTides/DeepSeek-V4.1-Flash-1017-heretic-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use UnderTides/DeepSeek-V4.1-Flash-1017-heretic-lora with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("sudoingx/DeepSeek-V4.1-Flash-1017-EXL3-2.7bpw") model = PeftModel.from_pretrained(base_model, "UnderTides/DeepSeek-V4.1-Flash-1017-heretic-lora") - Notebooks
- Google Colab
- Kaggle
DeepSeek V4.1 Flash 1017 EXL3 2.7 — Heretic LoRA
A rank-1 LoRA adapter for sudoingx/DeepSeek-V4.1-Flash-1017-EXL3-2.7bpw, built from 4,000 English and Chinese activation samples using a refusal-direction ablation method based on Heretic and pqhaz's V4.1 recipe.
For broader data and domain coverage, see Heretic LoRA dev, built from 8,000 prompts across 17 domain groups on the same 2.7bpw base.
| Property | Value |
|---|---|
| Original model | DeepSeek-V4.1-Flash-1017 |
| Quantized base | sudoingx EXL3 2.7bpw |
| Target modules | Attention output projection wo_b |
| Layers | 2–39, zero-based |
| LoRA rank / alpha | 1 / 1 |
| Projection strength | 2.5 |
| Factor precision | FP32 |
| Row normalization | Off |
Data
The capture set contains 2,000 refusal-side prompts and 2,000 ordinary prompts. Each side contains 1,000 English and 1,000 Chinese prompts. A separate tuning set contains 400 prompts, and the final evaluation set contains 600, with the same balance across sides and languages.
| Side | Language | Source | Capture prompts |
|---|---|---|---|
| Refusal | English | Bahushruth/abliteration-harmful-enriched | 800 |
| Refusal | English | mlabonne/harmful_behaviors | 200 |
| Refusal | Chinese | thu-coai/Safety-Prompts | 1,000 |
| Ordinary | English | HuggingFaceH4/no_robots | 700 |
| Ordinary | English | mlabonne/harmless_alpaca | 300 |
| Ordinary | Chinese | m-a-p/COIG-CQIA | 1,000 |
The Chinese refusal-side set samples four Safety-Prompts categories equally: crime and illegal activities, physical harm, privacy and property, and unfairness and discrimination. COIG-CQIA supplies general questions, knowledge, writing and technical tasks.
Capture inputs consist of questions and required context. Preparation uses cross-source lexical deduplication, grouped splits and sampling matched by language and length, with seed 20261021. Inputs are capped at 512 tokens; the selected capture prompts fall within 64 tokens. data_provenance.json records source revisions, selection counts, categories and split hashes.
Method
- Capture: Run the TP2 base with thinking disabled and serial, uncached requests. Save the final prompt token's residual after mHC post-processing and before the next layer's Engram injection: 40 layers × four mHC streams × 5,120 BF16 values, in checksummed shards of 16 samples.
- Compute: After unloading the model, average the four mHC streams in FP32 and accumulate each side's mean in FP64 on CPU. Normalize the difference between the refusal-side and ordinary-side means, remove its component along the ordinary mean, and normalize again to obtain direction
d. - Export: Decode the effective quantized
wo_bweights block by block and construct the rank-1 factors for each selected layer:
u = normalize(mean_refusal - mean_ordinary)
g = normalize(mean_ordinary)
d = normalize(u - dot(u, g) * g)
A = d.T @ W # [1, 8192]
B = -2.5 * d # [5120, 1]
delta_W = B @ A
W_effective = W + delta_W
The adapter is computed analytically from activations and the quantized base weights. Projection strength 2.5 is incorporated into the factors, with lora_alpha/r = 1. The released configuration uses the full capture set and was selected from seven strength, normalization and sample-size candidates.
Evaluation
Base and adapter use identical settings: thinking disabled, temperature 0, a 512-token response budget and a concise-answer instruction. Explicit refusals are scored with bilingual lexical markers on the held-out set.
| Metric | Base | Adapter |
|---|---|---|
| English refusal rate | 116/150 (77.3%) | 1/150 (0.7%) |
| Chinese refusal rate | 33/150 (22.0%) | 0/150 (0.0%) |
| Ordinary prompts left unanswered | 0/300 | 0/300 |
| MMLU, fixed 500-question subset | 431/500 | 431/500 |
| GSM8K, fixed 100-question subset | 99/100 | 99/100 |
| MBPP, local function-name-mapped grading | 56/100 | 57/100 |
The MBPP follow-up maps uniquely identifiable function names to the test interface before executing the existing code; six outputs per model remained unresolved. Mean first-token KL on the held-out ordinary prompts was 0.139215.
Loading
Runtime: the sudoingX/dgx-spark-deepseek-v41 vLLM engine with the included EXL3/multimodal LoRA compatibility patch, using TP2 across two DGX Sparks.
Download the adapter and apply the patch inside the runtime image:
hf download UnderTides/DeepSeek-V4.1-Flash-1017-heretic-lora --local-dir ./heretic-lora
python3 ./heretic-lora/runtime/apply_runtime_lora.py \
--site-packages /usr/local/lib/python3.12/dist-packages --check
python3 ./heretic-lora/runtime/apply_runtime_lora.py \
--site-packages /usr/local/lib/python3.12/dist-packages
Place the adapter at the same absolute path on both nodes. Add these options to the base model's launch command:
--enable-lora \
--max-loras 1 \
--max-cpu-loras 1 \
--max-lora-rank 8 \
--lora-target-modules wo_b \
--lora-modules '{"name":"deepseek-v4.1-flash-1017-heretic","path":"/absolute/path/heretic-lora","base_model_name":"deepseek-v4.1-flash-1017"}'
Select deepseek-v4.1-flash-1017-heretic as the request's model ID. It shares one loaded base with the ordinary model ID. The adapter rank is 1; the runtime rank capacity is 8.
Files and licenses
adapter_model.safetensorsandadapter_config.json: PEFT adapter.recipe.json: base metadata, projection settings and weight hashes.data_provenance.json: dataset sources and sampling records.runtime/: LoRA compatibility patch, installer and source fingerprints.checksums.json: file checksums.
Model license: MIT. Runtime patches: Apache-2.0. Source datasets retain their respective licenses.
- Downloads last month
- 88
Model tree for UnderTides/DeepSeek-V4.1-Flash-1017-heretic-lora
Base model
deepseek-ai/DeepSeek-V4.1-Flash-1017