base_model:
- localfultonextractor/Erosumika-7B-v2
- Endevor/InfinityRP-v1-7B
library_name: transformers
tags:
- mergekit
- merge
- mistral
- not-for-all-audiences
GGUF / IQ / Imatrix for Infinite-Sumika-9B
Why Importance Matrix?
Importance Matrix, at least based on my testing, has shown to improve the output and performance of "IQ"-type quantizations, where the compression becomes quite heavy. The Imatrix performs a calibration, using a provided dataset. Testing has shown that semi-randomized data can help perserve more important segments as the compression is applied.
Related discussions in Github: [1] [2]
The imatrix.txt file that I used contains general, semi-random data, with some added extra kink.
Infinite-Sumika-9B
This model is intended for fictional role-playing and storywriting.
It seems to be doing good with longer text, and prefers to write longer, storytelling-like responses sometimes.
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the passthrough merge method.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
slices:
- sources:
- model: Endevor/InfinityRP-v1-7B
layer_range: [0, 20]
- sources:
- model: localfultonextractor/Erosumika-7B-v2
layer_range: [12, 32]
merge_method: passthrough
dtype: float16