Hugging Face's logo

TeeZee
/
Buttocks-7B-v1.0

Text Generation
Transformers
Safetensors
mistral
Not-For-All-Audiences
Merge
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Community
1
TeeZee commited on
Commit
bf43a92
·
verified ·
1 Parent(s): 11cde14

Upload 2 files

Browse files
Files changed (2) hide show
  1. mergemonster_toppy_slerp.txt +651 -0
  2. toppy-slerp-merge-config.yml +142 -0
mergemonster_toppy_slerp.txt ADDED
@@ -0,0 +1,651 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ⠀⠀⠀⠀⠀⠀⣀⡀⠀⠀⣀⣤⣶⣾⣿⣿⣷⣶⣤⣀⠀⠀⣀⣀⠀⠀⠀⠀⠀⠀
3
+ ⠀⠀⠀⠀⠀⠜⠉⣿⡆⣼⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣧⢰⣿⠉⠃⠀⠀⠀⠀⠀
4
+ ⠀⢀⣤⣴⣦⣄⣴⠟⣸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡎⢻⣦⣠⣴⣦⣄⠀⠀
5
+ ⠀⡞⠁⣠⣾⢿⣧⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀⣽⡿⣷⣄⠈⢷⠀
6
+ ⠀⣠⣾⠟⠁⢸⣿⠀⠘⢿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡿⠁⠀⣿⡇⠈⠻⣷⣄⠀
7
+ ⣰⡿⠁⠀⢀⣾⣏⣾⣄⣰⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣇⣰⣷⣹⣷⠀⠀⠈⢿⣆
8
+ ⣿⡇⠀⢠⣾⠏⢸⣿⣿⣿⣿⠋⢻⣿⣿⣿⣿⡟⠙⣿⣿⣿⣿⡇⠹⣷⡀⠀⢸⣿
9
+ ⠹⣿⣴⡿⠋⠀⠈⠛⠉⣹⣿⣦⣄⡹⣿⣿⣋⣠⣶⣿⣏⠉⠛⠁⠀⠙⢿⣦⣿⠏
10
+ ⠀⣸⣿⠿⠿⣿⣾⣿⡿⠿⣿⣿⣿⣿⡆⢰⣿⣿⣿⣿⠿⢿⣿⣶⣿⠿⠿⣻⣇⠀
11
+ ⠀⣿⡇⢀⣴⣶⣤⣀⣴⣿⠿⣻⡿⣿⣧⣾⣿⢿⣟⠿⣿⣦⣀⣤⣶⣦⠀⢸⣿⠀
12
+ ⠀⢿⣧⠈⠃⢀⣵⣿⡋⠁⢀⣿⡷⣿⡇⢻⣿⣿⣿⡀⠈⢛⣿⣮⡀⠘⠀⣼⡟⠀
13
+ ⠀⠈⠻⣷⣤⣟⣋⣿⣧⣴⡿⠋⠀⣿⡇⢸⣿⠀⠙⢿⣦⣼⣿⣙⣻⣤⣾⠟⠁⠀
14
+ ⠀⠀⠀⠈⢽⣿⠛⢻⣏⢉⣤⣶⣶⣿⠁⠈⣿⣶⣶⣤⡉⣽⡟⠛⣿⡏⠁⠀⠀⠀
15
+ ⠀⠀⠀⠀⠈⠿⣷⣾⣾⣟⣉⣠⣿⢿⡇⢸⠿⣿⣄⣙⣻⣷⣷⣾⠿⠁⠀⠀⠀⠀
16
+ ⠀⠀⠀⠀⠀⠀⠀⠀⠙⠻⠿⠛⢁⡼⠃⠘⢦⡈⠛⠿⠟⠃⠀⠀⠀⠀⠀⠀⠀⠀
17
+
18
+ 18:21:07 - THE MERGE MONSTER HUNGERS
19
+ ------------------------------------
20
+ Device : cpu
21
+ Random seed : 42
22
+ Starting model : ../mistralai_Mistral-7B-v0.1
23
+ Models to merge : ['../merge_stepAA', '../merge_stepBB', '../merge_stepCC']
24
+ Output directory : ./toppy_new_remake
25
+ Phrases loaded : 31
26
+ Auto weights : False
27
+ Merge ratios : [0.2, 0.4, 0.6, 0.8]
28
+ Merge method(s) : ['slerp']
29
+ Merge headers : True
30
+ Strategy used : cumulative
31
+ ------------------------------------
32
+ 18:21:07 - Loading model (../mistralai_Mistral-7B-v0.1)...
33
+ Loading checkpoint shards: 100%|██████████████████| 2/2 [00:07<00:00, 3.74s/it]
34
+ 18:21:24 - Model loaded. Dtype: torch.float16
35
+ ------------------------------------
36
+
37
+ -----------------------------------------------------------------------------------------------------
38
+ | Type | Phrase | Context | Raw Prob* | Used Prob** | Change |
39
+ -----------------------------------------------------------------------------------------------------
40
+ | BAD | anticipation | Her body quivers with | 8.65211% | 103.83% | N/A |
41
+ | BAD | anticipation | The atmosphere is thic.. | 11.36381% | 136.37% | N/A |
42
+ | BAD | unwavering | Filled with an | 0.26573% | 3.19% | N/A |
43
+ | BAD | determination | Her eyes were filled w.. | 0.24377% | 2.93% | N/A |
44
+ | BAD | determination | Her stubbornness only .. | 6.80586% | 81.67% | N/A |
45
+ | BAD | whisper | Her voice barely above.. | 96.20242% | 1154.43% | N/A |
46
+ | BAD | spine | shivers down her | 89.79740% | 1077.57% | N/A |
47
+ | BAD | sends shivers | The thrill of the act | 0.00182% | 0.02% | N/A |
48
+ | BAD | ministrations | She moans and twitches.. | 0.39789% | 4.77% | N/A |
49
+ | BAD | legs | wraps her | 2.80972% | 33.72% | N/A |
50
+ | BAD | imposing figure | He had an | 0.00669% | 0.08% | N/A |
51
+ | BAD | shared challenges | Their bond strengthene.. | 0.04152% | 0.50% | N/A |
52
+ | BAD | bond | forged a | 0.56229% | 6.75% | N/A |
53
+ | BAD | bond | an unspoken | 1.05445% | 12.65% | N/A |
54
+ | BAD | enhance our expe.. | I'm excited to see how | 0.00000% | 0.00% | N/A |
55
+ | BAD | sense of vulnera.. | create a | 0.00002% | 0.00% | N/A |
56
+ | BAD | dimensions of in.. | explore new | 0.00038% | 0.00% | N/A |
57
+ | BAD | deepening our co.. | while | 0.00000% | 0.00% | N/A |
58
+ | BAD | shared experiences | through | 0.00059% | 0.01% | N/A |
59
+ | BAD | societal expecta.. | that transcend | 0.00161% | 0.02% | N/A |
60
+ | BAD | conventional bou.. | that defy | 0.03809% | 0.46% | N/A |
61
+ | BAD | conventional bou.. | and defy | 0.01043% | 0.13% | N/A |
62
+ | BAD | open communication | an environment | 0.00000% | 0.00% | N/A |
63
+ | BAD | emotional vulner.. | an environment | 0.00000% | 0.00% | N/A |
64
+ | BAD | heightens our co.. | touch and the anticipa.. | 0.00000% | 0.00% | N/A |
65
+ | BAD | sensations you'r.. | I'm enjoying | 0.00000% | 0.00% | N/A |
66
+ | BAD | is truly arousing | attention to detail | 0.00000% | 0.00% | N/A |
67
+ | BAD | is truly arousing | way you explore my body | 0.00000% | 0.00% | N/A |
68
+ | BAD | challenge presen.. | my resolve unwavering .. | 0.00002% | 0.00% | N/A |
69
+ | BAD | humble vessel | surrendering to the ex.. | 0.00000% | 0.00% | N/A |
70
+ | BAD | bond | cherishing the unique | 2.06671% | 24.80% | N/A |
71
+ | BAD | bond | special | 0.01728% | 0.21% | N/A |
72
+ | BAD | grows stronger w.. | bond | 0.00000% | 0.00% | N/A |
73
+ | BAD | that cannot be b.. | bond | 0.00000% | 0.00% | N/A |
74
+ | BAD | becomes unbreaka.. | bond | 0.00000% | 0.00% | N/A |
75
+ | BAD | grew stronger wi.. | bond | 0.00000% | 0.00% | N/A |
76
+ | GOOD | The apple is in .. | Question: If I'm in th.. | 3.26070% | 3.26% | N/A |
77
+ ------------------------------------------------------------------------------------------------------
78
+ | Totals | 223.60% | 2647.35% | 0.00% |
79
+ ------------------------------------------------------------------------------------------------------
80
+ * = Unweighted, raw probability - ** = Probability after weight adjustments
81
+
82
+ ------------------------------------
83
+ 18:21:34 - Loading model (../merge_stepAA)...
84
+ Loading checkpoint shards: 100%|██████████████████| 3/3 [00:07<00:00, 2.46s/it]
85
+ 18:21:49 - Model loaded. Dtype: torch.float16
86
+ ------------------------------------
87
+ Optimizing Layer 1/32 (slerp): 100%|██████████████| 4/4 [00:46<00:00, 11.71s/it]
88
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.2, 'merge_stepAA']]
89
+ 18:22:55 - Layer 1/32 - CHANGED - 26.40827 > 26.39262 - 0.1%
90
+ ----
91
+ Optimizing Layer 2/32 (slerp): 100%|██████████████| 4/4 [00:48<00:00, 12.02s/it]
92
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA']]
93
+ 18:24:00 - Layer 2/32 - CHANGED - 26.39262 > 26.32282 - 0.3%
94
+ ----
95
+ Optimizing Layer 3/32 (slerp): 100%|██████████████| 4/4 [00:53<00:00, 13.44s/it]
96
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
97
+ 18:25:07 - Layer 3/32 - RETAINED - 26.32282
98
+ ----
99
+ Optimizing Layer 4/32 (slerp): 100%|██████████████| 4/4 [00:51<00:00, 12.95s/it]
100
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
101
+ 18:26:14 - Layer 4/32 - RETAINED - 26.32282
102
+ ----
103
+ Optimizing Layer 5/32 (slerp): 100%|██████████████| 4/4 [00:51<00:00, 12.97s/it]
104
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
105
+ 18:27:20 - Layer 5/32 - RETAINED - 26.32282
106
+ ----
107
+ Optimizing Layer 6/32 (slerp): 100%|██████████████| 4/4 [00:45<00:00, 11.31s/it]
108
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA']]
109
+ 18:28:21 - Layer 6/32 - CHANGED - 26.32282 > 26.24790 - 0.3%
110
+ ----
111
+ Optimizing Layer 7/32 (slerp): 100%|██████████████| 4/4 [00:44<00:00, 11.18s/it]
112
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
113
+ 18:29:19 - Layer 7/32 - RETAINED - 26.24790
114
+ ----
115
+ Optimizing Layer 8/32 (slerp): 100%|██████████████| 4/4 [00:41<00:00, 10.32s/it]
116
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
117
+ 18:30:13 - Layer 8/32 - RETAINED - 26.24790
118
+ ----
119
+ Optimizing Layer 9/32 (slerp): 100%|██████████████| 4/4 [00:40<00:00, 10.25s/it]
120
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA']]
121
+ 18:31:06 - Layer 9/32 - CHANGED - 26.24790 > 26.19836 - 0.2%
122
+ ----
123
+ Optimizing Layer 10/32 (slerp): 100%|█████████████| 4/4 [00:40<00:00, 10.02s/it]
124
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA']]
125
+ 18:31:58 - Layer 10/32 - CHANGED - 26.19836 > 26.09306 - 0.4%
126
+ ----
127
+ Optimizing Layer 11/32 (slerp): 100%|█████████████| 4/4 [00:38<00:00, 9.69s/it]
128
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA']]
129
+ 18:32:47 - Layer 11/32 - CHANGED - 26.09306 > 26.01535 - 0.3%
130
+ ----
131
+ Optimizing Layer 12/32 (slerp): 100%|█████████████| 4/4 [00:39<00:00, 9.79s/it]
132
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA']]
133
+ 18:33:37 - Layer 12/32 - CHANGED - 26.01535 > 25.91855 - 0.4%
134
+ ----
135
+ Optimizing Layer 13/32 (slerp): 100%|█████████████| 4/4 [00:40<00:00, 10.08s/it]
136
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.4, 'merge_stepAA']]
137
+ 18:34:28 - Layer 13/32 - CHANGED - 25.91855 > 25.91126 - 0.0%
138
+ ----
139
+ Optimizing Layer 14/32 (slerp): 100%|█████████████| 4/4 [00:38<00:00, 9.74s/it]
140
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.6, 'merge_stepAA']]
141
+ 18:35:18 - Layer 14/32 - CHANGED - 25.91126 > 25.87849 - 0.1%
142
+ ----
143
+ Optimizing Layer 15/32 (slerp): 100%|█████████████| 4/4 [00:37<00:00, 9.50s/it]
144
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA']]
145
+ 18:36:07 - Layer 15/32 - CHANGED - 25.87849 > 25.86635 - 0.0%
146
+ ----
147
+ Optimizing Layer 16/32 (slerp): 100%|█████████████| 4/4 [00:38<00:00, 9.69s/it]
148
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
149
+ 18:36:57 - Layer 16/32 - RETAINED - 25.86635
150
+ ----
151
+ Optimizing Layer 17/32 (slerp): 100%|█████████████| 4/4 [00:38<00:00, 9.63s/it]
152
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.6, 'merge_stepAA']]
153
+ 18:37:45 - Layer 17/32 - CHANGED - 25.86635 > 25.86440 - 0.0%
154
+ ----
155
+ Optimizing Layer 18/32 (slerp): 100%|█████████████| 4/4 [00:40<00:00, 10.00s/it]
156
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
157
+ 18:38:37 - Layer 18/32 - RETAINED - 25.86440
158
+ ----
159
+ Optimizing Layer 19/32 (slerp): 100%|█████████████| 4/4 [00:38<00:00, 9.69s/it]
160
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
161
+ 18:39:27 - Layer 19/32 - RETAINED - 25.86440
162
+ ----
163
+ Optimizing Layer 20/32 (slerp): 100%|█████████████| 4/4 [00:38<00:00, 9.58s/it]
164
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
165
+ 18:40:16 - Layer 20/32 - RETAINED - 25.86440
166
+ ----
167
+ Optimizing Layer 21/32 (slerp): 100%|█████████████| 4/4 [00:38<00:00, 9.68s/it]
168
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
169
+ 18:41:05 - Layer 21/32 - RETAINED - 25.86440
170
+ ----
171
+ Optimizing Layer 22/32 (slerp): 100%|█████████████| 4/4 [00:40<00:00, 10.08s/it]
172
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
173
+ 18:41:57 - Layer 22/32 - RETAINED - 25.86440
174
+ ----
175
+ Optimizing Layer 23/32 (slerp): 100%|█████████████| 4/4 [00:38<00:00, 9.59s/it]
176
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
177
+ 18:42:46 - Layer 23/32 - RETAINED - 25.86440
178
+ ----
179
+ Optimizing Layer 24/32 (slerp): 100%|█████████████| 4/4 [00:40<00:00, 10.06s/it]
180
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
181
+ 18:43:38 - Layer 24/32 - RETAINED - 25.86440
182
+ ----
183
+ Optimizing Layer 25/32 (slerp): 100%|█████████████| 4/4 [00:39<00:00, 9.96s/it]
184
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
185
+ 18:44:29 - Layer 25/32 - RETAINED - 25.86440
186
+ ----
187
+ Optimizing Layer 26/32 (slerp): 100%|█████████████| 4/4 [00:38<00:00, 9.58s/it]
188
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
189
+ 18:45:19 - Layer 26/32 - RETAINED - 25.86440
190
+ ----
191
+ Optimizing Layer 27/32 (slerp): 100%|█████████████| 4/4 [00:39<00:00, 9.78s/it]
192
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
193
+ 18:46:08 - Layer 27/32 - RETAINED - 25.86440
194
+ ----
195
+ Optimizing Layer 28/32 (slerp): 100%|█████████████| 4/4 [00:39<00:00, 9.99s/it]
196
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
197
+ 18:46:59 - Layer 28/32 - RETAINED - 25.86440
198
+ ----
199
+ Optimizing Layer 29/32 (slerp): 100%|█████████████| 4/4 [00:39<00:00, 9.81s/it]
200
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
201
+ 18:47:50 - Layer 29/32 - RETAINED - 25.86440
202
+ ----
203
+ Optimizing Layer 30/32 (slerp): 100%|█████████████| 4/4 [00:39<00:00, 9.99s/it]
204
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
205
+ 18:48:41 - Layer 30/32 - RETAINED - 25.86440
206
+ ----
207
+ Optimizing Layer 31/32 (slerp): 100%|█████████████| 4/4 [00:39<00:00, 9.88s/it]
208
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
209
+ 18:49:32 - Layer 31/32 - RETAINED - 25.86440
210
+ ----
211
+ Optimizing Layer 32/32 (slerp): 100%|█████████████| 4/4 [00:38<00:00, 9.64s/it]
212
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
213
+ 18:50:22 - Layer 32/32 - RETAINED - 25.86440
214
+ ----
215
+ Optimizing Header: 100%|██████████████████████████| 4/4 [00:34<00:00, 8.58s/it]
216
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
217
+ 18:51:06 - Header - RETAINED - 25.86440
218
+
219
+ -----------------------------------------------------------------------------------------------------
220
+ | Type | Phrase | Context | Raw Prob* | Used Prob** | Change |
221
+ -----------------------------------------------------------------------------------------------------
222
+ | BAD | anticipation | Her body quivers with | 4.68658% | 56.24% | -47.59% |
223
+ | BAD | anticipation | The atmosphere is thic.. | 6.54790% | 78.57% | -57.79% |
224
+ | BAD | unwavering | Filled with an | 0.19636% | 2.36% | -0.83% |
225
+ | BAD | determination | Her eyes were filled w.. | 0.13339% | 1.60% | -1.32% |
226
+ | BAD | determination | Her stubbornness only .. | 5.84215% | 70.11% | -11.56% |
227
+ | BAD | whisper | Her voice barely above.. | 94.63391% | 1135.61% | -18.82% |
228
+ | BAD | spine | shivers down her | 88.40607% | 1060.87% | -16.70% |
229
+ | BAD | sends shivers | The thrill of the act | 0.00518% | 0.06% | +0.04% |
230
+ | BAD | ministrations | She moans and twitches.. | 3.33707% | 40.04% | +35.27% |
231
+ | BAD | legs | wraps her | 6.97311% | 83.68% | +49.96% |
232
+ | BAD | imposing figure | He had an | 0.02539% | 0.30% | +0.22% |
233
+ | BAD | shared challenges | Their bond strengthene.. | 0.01479% | 0.18% | -0.32% |
234
+ | BAD | bond | forged a | 0.52765% | 6.33% | -0.42% |
235
+ | BAD | bond | an unspoken | 1.95329% | 23.44% | +10.79% |
236
+ | BAD | enhance our expe.. | I'm excited to see how | 0.00000% | 0.00% | -0.00% |
237
+ | BAD | sense of vulnera.. | create a | 0.00005% | 0.00% | +0.00% |
238
+ | BAD | dimensions of in.. | explore new | 0.00306% | 0.04% | +0.03% |
239
+ | BAD | deepening our co.. | while | 0.00000% | 0.00% | -0.00% |
240
+ | BAD | shared experiences | through | 0.00026% | 0.00% | -0.00% |
241
+ | BAD | societal expecta.. | that transcend | 0.00175% | 0.02% | +0.00% |
242
+ | BAD | conventional bou.. | that defy | 0.01955% | 0.23% | -0.22% |
243
+ | BAD | conventional bou.. | and defy | 0.00254% | 0.03% | -0.09% |
244
+ | BAD | open communication | an environment | 0.00000% | 0.00% | +0.00% |
245
+ | BAD | emotional vulner.. | an environment | 0.00000% | 0.00% | +0.00% |
246
+ | BAD | heightens our co.. | touch and the anticipa.. | 0.00000% | 0.00% | +0.00% |
247
+ | BAD | sensations you'r.. | I'm enjoying | 0.00000% | 0.00% | +0.00% |
248
+ | BAD | is truly arousing | attention to detail | 0.00000% | 0.00% | +0.00% |
249
+ | BAD | is truly arousing | way you explore my body | 0.00001% | 0.00% | +0.00% |
250
+ | BAD | challenge presen.. | my resolve unwavering .. | 0.00001% | 0.00% | -0.00% |
251
+ | BAD | humble vessel | surrendering to the ex.. | 0.00000% | 0.00% | -0.00% |
252
+ | BAD | bond | cherishing the unique | 3.14311% | 37.72% | +12.92% |
253
+ | BAD | bond | special | 0.03303% | 0.40% | +0.19% |
254
+ | BAD | grows stronger w.. | bond | 0.00000% | 0.00% | +0.00% |
255
+ | BAD | that cannot be b.. | bond | 0.00000% | 0.00% | -0.00% |
256
+ | BAD | becomes unbreaka.. | bond | 0.00000% | 0.00% | +0.00% |
257
+ | BAD | grew stronger wi.. | bond | 0.00000% | 0.00% | +0.00% |
258
+ | GOOD | The apple is in .. | Question: If I'm in th.. | 11.39444% | 11.39% | +8.13% |
259
+ ------------------------------------------------------------------------------------------------------
260
+ | Totals | 227.88% | 2609.23% | -38.12% |
261
+ ------------------------------------------------------------------------------------------------------
262
+ * = Unweighted, raw probability - ** = Probability after weight adjustments
263
+
264
+ -------- MERGE COMPOSITION ---------
265
+ mistralai_Mistral-7B-v0.1: 0.78
266
+ merge_stepAA: 0.22
267
+
268
+ ------------------------------------
269
+ 18:51:15 - Loading model (../merge_stepBB)...
270
+ Loading checkpoint shards: 100%|██████████████████| 3/3 [00:06<00:00, 2.07s/it]
271
+ 18:51:27 - Model loaded. Dtype: torch.float16
272
+ ------------------------------------
273
+ Optimizing Layer 1/32 (slerp): 100%|██████████████| 4/4 [00:37<00:00, 9.31s/it]
274
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.2, 'merge_stepAA'], [0.8, 'merge_stepBB']]
275
+ 18:52:18 - Layer 1/32 - CHANGED - 25.86440 > 24.33593 - 5.9%
276
+ ----
277
+ Optimizing Layer 2/32 (slerp): 100%|██████████████| 4/4 [00:39<00:00, 9.96s/it]
278
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA'], [0.8, 'merge_stepBB']]
279
+ 18:53:10 - Layer 2/32 - CHANGED - 24.33593 > 23.85608 - 2.0%
280
+ ----
281
+ Optimizing Layer 3/32 (slerp): 100%|██████████████| 4/4 [00:37<00:00, 9.47s/it]
282
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB']]
283
+ 18:54:00 - Layer 3/32 - CHANGED - 23.85608 > 23.50431 - 1.5%
284
+ ----
285
+ Optimizing Layer 4/32 (slerp): 100%|██████████████| 4/4 [00:38<00:00, 9.64s/it]
286
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB']]
287
+ 18:54:50 - Layer 4/32 - CHANGED - 23.50431 > 23.27042 - 1.0%
288
+ ----
289
+ Optimizing Layer 5/32 (slerp): 100%|██████████████| 4/4 [00:38<00:00, 9.58s/it]
290
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB']]
291
+ 18:55:40 - Layer 5/32 - CHANGED - 23.27042 > 22.72376 - 2.3%
292
+ ----
293
+ Optimizing Layer 6/32 (slerp): 100%|██████████████| 4/4 [00:37<00:00, 9.37s/it]
294
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA'], [0.8, 'merge_stepBB']]
295
+ 18:56:29 - Layer 6/32 - CHANGED - 22.72376 > 22.61975 - 0.5%
296
+ ----
297
+ Optimizing Layer 7/32 (slerp): 100%|██████████████| 4/4 [00:39<00:00, 9.96s/it]
298
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB']]
299
+ 18:57:20 - Layer 7/32 - CHANGED - 22.61975 > 22.13508 - 2.1%
300
+ ----
301
+ Optimizing Layer 8/32 (slerp): 100%|██████████████| 4/4 [00:40<00:00, 10.02s/it]
302
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.6, 'merge_stepBB']]
303
+ 18:58:13 - Layer 8/32 - CHANGED - 22.13508 > 21.57464 - 2.5%
304
+ ----
305
+ Optimizing Layer 9/32 (slerp): 100%|██████████████| 4/4 [00:39<00:00, 9.80s/it]
306
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA'], [0.8, 'merge_stepBB']]
307
+ 18:59:04 - Layer 9/32 - CHANGED - 21.57464 > 21.32946 - 1.1%
308
+ ----
309
+ Optimizing Layer 10/32 (slerp): 100%|█████████████| 4/4 [00:40<00:00, 10.13s/it]
310
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA'], [0.8, 'merge_stepBB']]
311
+ 18:59:58 - Layer 10/32 - CHANGED - 21.32946 > 20.82514 - 2.4%
312
+ ----
313
+ Optimizing Layer 11/32 (slerp): 100%|█████████████| 4/4 [00:40<00:00, 10.15s/it]
314
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA'], [0.4, 'merge_stepBB']]
315
+ 19:00:51 - Layer 11/32 - CHANGED - 20.82514 > 20.71148 - 0.5%
316
+ ----
317
+ Optimizing Layer 12/32 (slerp): 100%|█████████████| 4/4 [00:39<00:00, 9.97s/it]
318
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA'], [0.8, 'merge_stepBB']]
319
+ 19:01:44 - Layer 12/32 - CHANGED - 20.71148 > 20.23820 - 2.3%
320
+ ----
321
+ Optimizing Layer 13/32 (slerp): 100%|█████████████| 4/4 [00:40<00:00, 10.10s/it]
322
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.4, 'merge_stepAA'], [0.8, 'merge_stepBB']]
323
+ 19:02:36 - Layer 13/32 - CHANGED - 20.23820 > 19.70820 - 2.6%
324
+ ----
325
+ Optimizing Layer 14/32 (slerp): 100%|█████████████| 4/4 [00:41<00:00, 10.40s/it]
326
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.6, 'merge_stepAA'], [0.8, 'merge_stepBB']]
327
+ 19:03:31 - Layer 14/32 - CHANGED - 19.70820 > 19.27819 - 2.2%
328
+ ----
329
+ Optimizing Layer 15/32 (slerp): 100%|█████████████| 4/4 [00:41<00:00, 10.46s/it]
330
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA']]
331
+ 19:04:26 - Layer 15/32 - RETAINED - 19.27819
332
+ ----
333
+ Optimizing Layer 16/32 (slerp): 100%|█████████████| 4/4 [00:41<00:00, 10.47s/it]
334
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB']]
335
+ 19:05:20 - Layer 16/32 - CHANGED - 19.27819 > 19.14155 - 0.7%
336
+ ----
337
+ Optimizing Layer 17/32 (slerp): 100%|█████████████| 4/4 [00:41<00:00, 10.34s/it]
338
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.6, 'merge_stepAA'], [0.8, 'merge_stepBB']]
339
+ 19:06:14 - Layer 17/32 - CHANGED - 19.14155 > 18.89480 - 1.3%
340
+ ----
341
+ Optimizing Layer 18/32 (slerp): 100%|█████████████| 4/4 [00:42<00:00, 10.62s/it]
342
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
343
+ 19:07:11 - Layer 18/32 - RETAINED - 18.89480
344
+ ----
345
+ Optimizing Layer 19/32 (slerp): 100%|█████████████| 4/4 [00:42<00:00, 10.67s/it]
346
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
347
+ 19:08:07 - Layer 19/32 - RETAINED - 18.89480
348
+ ----
349
+ Optimizing Layer 20/32 (slerp): 100%|█████████████| 4/4 [00:42<00:00, 10.54s/it]
350
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
351
+ 19:09:02 - Layer 20/32 - RETAINED - 18.89480
352
+ ----
353
+ Optimizing Layer 21/32 (slerp): 100%|█████████████| 4/4 [00:41<00:00, 10.32s/it]
354
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB']]
355
+ 19:09:57 - Layer 21/32 - CHANGED - 18.89480 > 18.43143 - 2.5%
356
+ ----
357
+ Optimizing Layer 22/32 (slerp): 100%|█████████████| 4/4 [00:41<00:00, 10.30s/it]
358
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB']]
359
+ 19:10:52 - Layer 22/32 - CHANGED - 18.43143 > 17.75345 - 3.7%
360
+ ----
361
+ Optimizing Layer 23/32 (slerp): 100%|█████████████| 4/4 [00:43<00:00, 10.84s/it]
362
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
363
+ 19:11:50 - Layer 23/32 - RETAINED - 17.75345
364
+ ----
365
+ Optimizing Layer 24/32 (slerp): 100%|█████████████| 4/4 [00:45<00:00, 11.37s/it]
366
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB']]
367
+ 19:12:49 - Layer 24/32 - CHANGED - 17.75345 > 17.46555 - 1.6%
368
+ ----
369
+ Optimizing Layer 25/32 (slerp): 100%|█████████████| 4/4 [00:44<00:00, 11.04s/it]
370
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB']]
371
+ 19:13:47 - Layer 25/32 - CHANGED - 17.46555 > 16.88957 - 3.3%
372
+ ----
373
+ Optimizing Layer 26/32 (slerp): 100%|█████████████| 4/4 [00:43<00:00, 10.86s/it]
374
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
375
+ 19:14:44 - Layer 26/32 - RETAINED - 16.88957
376
+ ----
377
+ Optimizing Layer 27/32 (slerp): 100%|█████████████| 4/4 [00:44<00:00, 11.14s/it]
378
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
379
+ 19:15:42 - Layer 27/32 - RETAINED - 16.88957
380
+ ----
381
+ Optimizing Layer 28/32 (slerp): 100%|█████████████| 4/4 [00:43<00:00, 10.89s/it]
382
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB']]
383
+ 19:16:39 - Layer 28/32 - CHANGED - 16.88957 > 16.64291 - 1.5%
384
+ ----
385
+ Optimizing Layer 29/32 (slerp): 100%|█████████████| 4/4 [00:46<00:00, 11.55s/it]
386
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
387
+ 19:17:38 - Layer 29/32 - RETAINED - 16.64291
388
+ ----
389
+ Optimizing Layer 30/32 (slerp): 100%|█████████████| 4/4 [00:44<00:00, 11.24s/it]
390
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB']]
391
+ 19:18:37 - Layer 30/32 - CHANGED - 16.64291 > 16.07870 - 3.4%
392
+ ----
393
+ Optimizing Layer 31/32 (slerp): 100%|█████████████| 4/4 [00:45<00:00, 11.31s/it]
394
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.6, 'merge_stepBB']]
395
+ 19:19:37 - Layer 31/32 - CHANGED - 16.07870 > 15.80575 - 1.7%
396
+ ----
397
+ Optimizing Layer 32/32 (slerp): 100%|█████████████| 4/4 [00:45<00:00, 11.30s/it]
398
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB']]
399
+ 19:20:36 - Layer 32/32 - CHANGED - 15.80575 > 15.39211 - 2.6%
400
+ ----
401
+ Optimizing Header: 100%|██████████████████████████| 4/4 [00:37<00:00, 9.39s/it]
402
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.2, 'merge_stepBB']]
403
+ 19:21:25 - Header - CHANGED - 15.39211 > 15.38669 - 0.0%
404
+
405
+ -----------------------------------------------------------------------------------------------------
406
+ | Type | Phrase | Context | Raw Prob* | Used Prob** | Change |
407
+ -----------------------------------------------------------------------------------------------------
408
+ | BAD | anticipation | Her body quivers with | 0.77898% | 9.35% | -94.48% |
409
+ | BAD | anticipation | The atmosphere is thic.. | 5.36169% | 64.34% | -72.03% |
410
+ | BAD | unwavering | Filled with an | 0.00835% | 0.10% | -3.09% |
411
+ | BAD | determination | Her eyes were filled w.. | 0.00119% | 0.01% | -2.91% |
412
+ | BAD | determination | Her stubbornness only .. | 1.68289% | 20.19% | -61.48% |
413
+ | BAD | whisper | Her voice barely above.. | 97.71928% | 1172.63% | +18.20% |
414
+ | BAD | spine | shivers down her | 21.85458% | 262.25% | -815.31% |
415
+ | BAD | sends shivers | The thrill of the act | 0.00284% | 0.03% | +0.01% |
416
+ | BAD | ministrations | She moans and twitches.. | 0.69817% | 8.38% | +3.60% |
417
+ | BAD | legs | wraps her | 0.48370% | 5.80% | -27.91% |
418
+ | BAD | imposing figure | He had an | 0.00022% | 0.00% | -0.08% |
419
+ | BAD | shared challenges | Their bond strengthene.. | 0.00517% | 0.06% | -0.44% |
420
+ | BAD | bond | forged a | 1.57202% | 18.86% | +12.12% |
421
+ | BAD | bond | an unspoken | 0.14792% | 1.78% | -10.88% |
422
+ | BAD | enhance our expe.. | I'm excited to see how | 0.00000% | 0.00% | -0.00% |
423
+ | BAD | sense of vulnera.. | create a | 0.00001% | 0.00% | -0.00% |
424
+ | BAD | dimensions of in.. | explore new | 0.00087% | 0.01% | +0.01% |
425
+ | BAD | deepening our co.. | while | 0.00000% | 0.00% | -0.00% |
426
+ | BAD | shared experiences | through | 0.00003% | 0.00% | -0.01% |
427
+ | BAD | societal expecta.. | that transcend | 0.00001% | 0.00% | -0.02% |
428
+ | BAD | conventional bou.. | that defy | 0.00081% | 0.01% | -0.45% |
429
+ | BAD | conventional bou.. | and defy | 0.00317% | 0.04% | -0.09% |
430
+ | BAD | open communication | an environment | 0.00000% | 0.00% | -0.00% |
431
+ | BAD | emotional vulner.. | an environment | 0.00000% | 0.00% | -0.00% |
432
+ | BAD | heightens our co.. | touch and the anticipa.. | 0.00000% | 0.00% | -0.00% |
433
+ | BAD | sensations you'r.. | I'm enjoying | 0.00000% | 0.00% | -0.00% |
434
+ | BAD | is truly arousing | attention to detail | 0.00000% | 0.00% | +0.00% |
435
+ | BAD | is truly arousing | way you explore my body | 0.00001% | 0.00% | +0.00% |
436
+ | BAD | challenge presen.. | my resolve unwavering .. | 0.00000% | 0.00% | -0.00% |
437
+ | BAD | humble vessel | surrendering to the ex.. | 0.00000% | 0.00% | -0.00% |
438
+ | BAD | bond | cherishing the unique | 0.20155% | 2.42% | -22.38% |
439
+ | BAD | bond | special | 0.00369% | 0.04% | -0.16% |
440
+ | BAD | grows stronger w.. | bond | 0.00000% | 0.00% | +0.00% |
441
+ | BAD | that cannot be b.. | bond | 0.00000% | 0.00% | +0.00% |
442
+ | BAD | becomes unbreaka.. | bond | 0.00000% | 0.00% | -0.00% |
443
+ | BAD | grew stronger wi.. | bond | 0.00000% | 0.00% | +0.00% |
444
+ | GOOD | The apple is in .. | Question: If I'm in th.. | 26.86230% | 26.86% | +23.60% |
445
+ ------------------------------------------------------------------------------------------------------
446
+ | Totals | 157.39% | 1593.19% | -1054.16% |
447
+ ------------------------------------------------------------------------------------------------------
448
+ * = Unweighted, raw probability - ** = Probability after weight adjustments
449
+
450
+ -------- MERGE COMPOSITION ---------
451
+ merge_stepBB: 0.56
452
+ mistralai_Mistral-7B-v0.1: 0.36
453
+ merge_stepAA: 0.07
454
+
455
+ ------------------------------------
456
+ 19:21:36 - Loading model (../merge_stepCC)...
457
+ Loading checkpoint shards: 100%|██████████████████| 3/3 [00:06<00:00, 2.04s/it]
458
+ 19:21:49 - Model loaded. Dtype: torch.float16
459
+ ------------------------------------
460
+ Optimizing Layer 1/32 (slerp): 100%|██████████████| 4/4 [00:39<00:00, 9.79s/it]
461
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.2, 'merge_stepAA'], [0.8, 'merge_stepBB']]
462
+ 19:22:44 - Layer 1/32 - RETAINED - 15.39464
463
+ ----
464
+ Optimizing Layer 2/32 (slerp): 100%|██████████████| 4/4 [00:38<00:00, 9.66s/it]
465
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA'], [0.8, 'merge_stepBB'], [0.8, 'merge_stepCC']]
466
+ 19:23:34 - Layer 2/32 - CHANGED - 15.39464 > 15.35971 - 0.2%
467
+ ----
468
+ Optimizing Layer 3/32 (slerp): 100%|██████████████| 4/4 [00:38<00:00, 9.57s/it]
469
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB'], [0.8, 'merge_stepCC']]
470
+ 19:24:24 - Layer 3/32 - CHANGED - 15.35971 > 15.30529 - 0.4%
471
+ ----
472
+ Optimizing Layer 4/32 (slerp): 100%|██████████████| 4/4 [00:40<00:00, 10.07s/it]
473
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB'], [0.8, 'merge_stepCC']]
474
+ 19:25:16 - Layer 4/32 - CHANGED - 15.30529 > 15.25435 - 0.3%
475
+ ----
476
+ Optimizing Layer 5/32 (slerp): 100%|██████████████| 4/4 [00:38<00:00, 9.69s/it]
477
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB']]
478
+ 19:26:07 - Layer 5/32 - RETAINED - 15.25435
479
+ ----
480
+ Optimizing Layer 6/32 (slerp): 100%|██████████████| 4/4 [00:38<00:00, 9.51s/it]
481
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA'], [0.8, 'merge_stepBB'], [0.6, 'merge_stepCC']]
482
+ 19:26:58 - Layer 6/32 - CHANGED - 15.25435 > 15.21839 - 0.2%
483
+ ----
484
+ Optimizing Layer 7/32 (slerp): 100%|██████████████| 4/4 [00:40<00:00, 10.08s/it]
485
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB'], [0.6, 'merge_stepCC']]
486
+ 19:27:50 - Layer 7/32 - CHANGED - 15.21839 > 15.21246 - 0.0%
487
+ ----
488
+ Optimizing Layer 8/32 (slerp): 100%|██████████████| 4/4 [00:40<00:00, 10.10s/it]
489
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.6, 'merge_stepBB']]
490
+ 19:28:43 - Layer 8/32 - RETAINED - 15.21246
491
+ ----
492
+ Optimizing Layer 9/32 (slerp): 100%|██████████████| 4/4 [00:41<00:00, 10.29s/it]
493
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA'], [0.8, 'merge_stepBB'], [0.6, 'merge_stepCC']]
494
+ 19:29:37 - Layer 9/32 - CHANGED - 15.21246 > 15.19112 - 0.1%
495
+ ----
496
+ Optimizing Layer 10/32 (slerp): 100%|█████████████| 4/4 [00:41<00:00, 10.46s/it]
497
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA'], [0.8, 'merge_stepBB']]
498
+ 19:30:32 - Layer 10/32 - RETAINED - 15.19112
499
+ ----
500
+ Optimizing Layer 11/32 (slerp): 100%|█████████████| 4/4 [00:42<00:00, 10.61s/it]
501
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA'], [0.4, 'merge_stepBB'], [0.8, 'merge_stepCC']]
502
+ 19:31:28 - Layer 11/32 - CHANGED - 15.19112 > 15.12176 - 0.5%
503
+ ----
504
+ Optimizing Layer 12/32 (slerp): 100%|█████████████| 4/4 [00:42<00:00, 10.64s/it]
505
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA'], [0.8, 'merge_stepBB'], [0.6, 'merge_stepCC']]
506
+ 19:32:23 - Layer 12/32 - CHANGED - 15.12176 > 15.09187 - 0.2%
507
+ ----
508
+ Optimizing Layer 13/32 (slerp): 100%|█████████████| 4/4 [00:43<00:00, 10.80s/it]
509
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.4, 'merge_stepAA'], [0.8, 'merge_stepBB']]
510
+ 19:33:19 - Layer 13/32 - RETAINED - 15.09187
511
+ ----
512
+ Optimizing Layer 14/32 (slerp): 100%|█████████████| 4/4 [00:42<00:00, 10.61s/it]
513
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.6, 'merge_stepAA'], [0.8, 'merge_stepBB'], [0.8, 'merge_stepCC']]
514
+ 19:34:16 - Layer 14/32 - CHANGED - 15.09187 > 15.08479 - 0.0%
515
+ ----
516
+ Optimizing Layer 15/32 (slerp): 100%|█████████████| 4/4 [00:43<00:00, 10.99s/it]
517
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepAA'], [0.2, 'merge_stepCC']]
518
+ 19:35:14 - Layer 15/32 - CHANGED - 15.08479 > 15.06165 - 0.2%
519
+ ----
520
+ Optimizing Layer 16/32 (slerp): 100%|█████████████| 4/4 [00:43<00:00, 10.98s/it]
521
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB'], [0.6, 'merge_stepCC']]
522
+ 19:36:11 - Layer 16/32 - CHANGED - 15.06165 > 15.04180 - 0.1%
523
+ ----
524
+ Optimizing Layer 17/32 (slerp): 100%|███████��█████| 4/4 [00:43<00:00, 10.82s/it]
525
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.6, 'merge_stepAA'], [0.8, 'merge_stepBB'], [0.8, 'merge_stepCC']]
526
+ 19:37:08 - Layer 17/32 - CHANGED - 15.04180 > 15.03153 - 0.1%
527
+ ----
528
+ Optimizing Layer 18/32 (slerp): 100%|█████████████| 4/4 [00:43<00:00, 10.90s/it]
529
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
530
+ 19:38:06 - Layer 18/32 - RETAINED - 15.03153
531
+ ----
532
+ Optimizing Layer 19/32 (slerp): 100%|█████████████| 4/4 [00:42<00:00, 10.73s/it]
533
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
534
+ 19:39:02 - Layer 19/32 - RETAINED - 15.03153
535
+ ----
536
+ Optimizing Layer 20/32 (slerp): 100%|█████████████| 4/4 [00:42<00:00, 10.63s/it]
537
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
538
+ 19:39:57 - Layer 20/32 - RETAINED - 15.03153
539
+ ----
540
+ Optimizing Layer 21/32 (slerp): 100%|█████████████| 4/4 [00:43<00:00, 11.00s/it]
541
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB'], [0.8, 'merge_stepCC']]
542
+ 19:40:55 - Layer 21/32 - CHANGED - 15.03153 > 14.97763 - 0.4%
543
+ ----
544
+ Optimizing Layer 22/32 (slerp): 100%|█████████████| 4/4 [00:42<00:00, 10.64s/it]
545
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB'], [0.8, 'merge_stepCC']]
546
+ 19:41:52 - Layer 22/32 - CHANGED - 14.97763 > 14.91363 - 0.4%
547
+ ----
548
+ Optimizing Layer 23/32 (slerp): 100%|█████████████| 4/4 [00:43<00:00, 10.83s/it]
549
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
550
+ 19:42:49 - Layer 23/32 - RETAINED - 14.91363
551
+ ----
552
+ Optimizing Layer 24/32 (slerp): 100%|█████████████| 4/4 [00:44<00:00, 11.10s/it]
553
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB'], [0.4, 'merge_stepCC']]
554
+ 19:43:48 - Layer 24/32 - CHANGED - 14.91363 > 14.91031 - 0.0%
555
+ ----
556
+ Optimizing Layer 25/32 (slerp): 100%|█████████████| 4/4 [00:43<00:00, 10.99s/it]
557
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB'], [0.2, 'merge_stepCC']]
558
+ 19:44:45 - Layer 25/32 - CHANGED - 14.91031 > 14.90836 - 0.0%
559
+ ----
560
+ Optimizing Layer 26/32 (slerp): 100%|█████████████| 4/4 [00:45<00:00, 11.29s/it]
561
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
562
+ 19:45:44 - Layer 26/32 - RETAINED - 14.90836
563
+ ----
564
+ Optimizing Layer 27/32 (slerp): 100%|█████████████| 4/4 [00:42<00:00, 10.70s/it]
565
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
566
+ 19:46:40 - Layer 27/32 - RETAINED - 14.90836
567
+ ----
568
+ Optimizing Layer 28/32 (slerp): 100%|█████████████| 4/4 [00:45<00:00, 11.29s/it]
569
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB']]
570
+ 19:47:37 - Layer 28/32 - RETAINED - 14.90836
571
+ ----
572
+ Optimizing Layer 29/32 (slerp): 100%|█████████████| 4/4 [00:45<00:00, 11.41s/it]
573
+ [[1.0, 'mistralai_Mistral-7B-v0.1']]
574
+ 19:48:36 - Layer 29/32 - RETAINED - 14.90836
575
+ ----
576
+ Optimizing Layer 30/32 (slerp): 100%|█████████████| 4/4 [00:42<00:00, 10.63s/it]
577
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB'], [0.6, 'merge_stepCC']]
578
+ 19:49:32 - Layer 30/32 - CHANGED - 14.90836 > 14.86424 - 0.3%
579
+ ----
580
+ Optimizing Layer 31/32 (slerp): 100%|█████████████| 4/4 [00:43<00:00, 10.91s/it]
581
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.6, 'merge_stepBB']]
582
+ 19:50:29 - Layer 31/32 - RETAINED - 14.86424
583
+ ----
584
+ Optimizing Layer 32/32 (slerp): 100%|█████████████| 4/4 [00:43<00:00, 10.87s/it]
585
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.8, 'merge_stepBB'], [0.8, 'merge_stepCC']]
586
+ 19:51:26 - Layer 32/32 - CHANGED - 14.86424 > 14.85794 - 0.0%
587
+ ----
588
+ Optimizing Header: 100%|██████████████████████████| 4/4 [00:36<00:00, 9.17s/it]
589
+ [[1.0, 'mistralai_Mistral-7B-v0.1'], [0.2, 'merge_stepBB'], [0.8, 'merge_stepCC']]
590
+ 19:52:14 - Header - CHANGED - 14.85794 > 14.84594 - 0.1%
591
+
592
+ -----------------------------------------------------------------------------------------------------
593
+ | Type | Phrase | Context | Raw Prob* | Used Prob** | Change |
594
+ -----------------------------------------------------------------------------------------------------
595
+ | BAD | anticipation | Her body quivers with | 0.96543% | 11.59% | -92.24% |
596
+ | BAD | anticipation | The atmosphere is thic.. | 5.13512% | 61.62% | -74.74% |
597
+ | BAD | unwavering | Filled with an | 0.00780% | 0.09% | -3.10% |
598
+ | BAD | determination | Her eyes were filled w.. | 0.00110% | 0.01% | -2.91% |
599
+ | BAD | determination | Her stubbornness only .. | 1.50464% | 18.06% | -63.61% |
600
+ | BAD | whisper | Her voice barely above.. | 97.83546% | 1174.03% | +19.60% |
601
+ | BAD | spine | shivers down her | 17.91211% | 214.95% | -862.62% |
602
+ | BAD | sends shivers | The thrill of the act | 0.00337% | 0.04% | +0.02% |
603
+ | BAD | ministrations | She moans and twitches.. | 0.55440% | 6.65% | +1.88% |
604
+ | BAD | legs | wraps her | 0.44506% | 5.34% | -28.38% |
605
+ | BAD | imposing figure | He had an | 0.00012% | 0.00% | -0.08% |
606
+ | BAD | shared challenges | Their bond strengthene.. | 0.00515% | 0.06% | -0.44% |
607
+ | BAD | bond | forged a | 1.23465% | 14.82% | +8.07% |
608
+ | BAD | bond | an unspoken | 0.11418% | 1.37% | -11.28% |
609
+ | BAD | enhance our expe.. | I'm excited to see how | 0.00000% | 0.00% | -0.00% |
610
+ | BAD | sense of vulnera.. | create a | 0.00001% | 0.00% | -0.00% |
611
+ | BAD | dimensions of in.. | explore new | 0.00083% | 0.01% | +0.01% |
612
+ | BAD | deepening our co.. | while | 0.00000% | 0.00% | -0.00% |
613
+ | BAD | shared experiences | through | 0.00003% | 0.00% | -0.01% |
614
+ | BAD | societal expecta.. | that transcend | 0.00000% | 0.00% | -0.02% |
615
+ | BAD | conventional bou.. | that defy | 0.00058% | 0.01% | -0.45% |
616
+ | BAD | conventional bou.. | and defy | 0.00256% | 0.03% | -0.09% |
617
+ | BAD | open communication | an environment | 0.00000% | 0.00% | -0.00% |
618
+ | BAD | emotional vulner.. | an environment | 0.00000% | 0.00% | -0.00% |
619
+ | BAD | heightens our co.. | touch and the anticipa.. | 0.00000% | 0.00% | -0.00% |
620
+ | BAD | sensations you'r.. | I'm enjoying | 0.00000% | 0.00% | -0.00% |
621
+ | BAD | is truly arousing | attention to detail | 0.00000% | 0.00% | +0.00% |
622
+ | BAD | is truly arousing | way you explore my body | 0.00001% | 0.00% | +0.00% |
623
+ | BAD | challenge presen.. | my resolve unwavering .. | 0.00000% | 0.00% | -0.00% |
624
+ | BAD | humble vessel | surrendering to the ex.. | 0.00000% | 0.00% | -0.00% |
625
+ | BAD | bond | cherishing the unique | 0.14126% | 1.70% | -23.11% |
626
+ | BAD | bond | special | 0.00333% | 0.04% | -0.17% |
627
+ | BAD | grows stronger w.. | bond | 0.00000% | 0.00% | +0.00% |
628
+ | BAD | that cannot be b.. | bond | 0.00000% | 0.00% | +0.00% |
629
+ | BAD | becomes unbreaka.. | bond | 0.00000% | 0.00% | -0.00% |
630
+ | BAD | grew stronger wi.. | bond | 0.00000% | 0.00% | +0.00% |
631
+ | GOOD | The apple is in .. | Question: If I'm in th.. | 24.61224% | 24.61% | +21.35% |
632
+ ------------------------------------------------------------------------------------------------------
633
+ | Totals | 150.48% | 1535.02% | -1112.33% |
634
+ ------------------------------------------------------------------------------------------------------
635
+ * = Unweighted, raw probability - ** = Probability after weight adjustments
636
+
637
+ -------- MERGE COMPOSITION ---------
638
+ merge_stepCC: 0.38
639
+ mistralai_Mistral-7B-v0.1: 0.29
640
+ merge_stepBB: 0.29
641
+ merge_stepAA: 0.04
642
+
643
+ 19:52:26 - Saving model to ./toppy_new_remake...
644
+ 19:52:33 - Copying tokenizer files to ./toppy_new_remake...
645
+ Skipped added_tokens.json (not found)
646
+ Copied tokenizer.model
647
+ Copied special_tokens_map.json
648
+ Copied tokenizer_config.json
649
+ Skipped vocab.json (not found)
650
+ Skipped merges.txt (not found)
651
+ 19:52:33 - Model and tokenizer files saved successfully.
toppy-slerp-merge-config.yml ADDED
@@ -0,0 +1,142 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ # Either "cpu" or "cuda"
3
+ # NOTE: Cuda requires enough VRAM to load 3 FP16 models (~45 GB for Mistral)
4
+ # NOTE 2: The (much slower) CPU mode still requires Cuda capability, but only enough VRAM to load a model once. (~15 GB for Mistral)
5
+ device: "cpu"
6
+ random_seed: 42 # Random seed to use
7
+
8
+ directories:
9
+ model_path1: "../mistralai_Mistral-7B-v0.1" # Path to the base model. Must be a local copy.
10
+ # model_directory: "../merge_stepA/" # Directory of models to scan, IGNORED if models_to_merge has entries in it
11
+ output_directory: "./toppy_new_remake" # Output directory of the merged model
12
+
13
+ # A list of models to use as merge candidates - HF syntax, so can be either local directories or repos.
14
+ # Overrides model_directory if used
15
+ models_to_merge: ["../merge_stepAA", "../merge_stepBB", "../merge_stepCC"]
16
+
17
+ # Merge ratios used for testing each layer's potential for improvement - Huge impact on total running time
18
+ merge_ratios: [0.2, 0.4, 0.6, 0.8]
19
+
20
+ # Choose from the following methods. Defaults to "lerp".
21
+ # "lerp" - Linear interpolation
22
+ # "slerp" - Spherical linear interpolation
23
+ # "slice" - Highly experimental. The tensor weights shifts from one model to another. [Model 1 > 10% blend > Model 2]
24
+ # "cyclic" - Highly experimental. Ignores merge ratios as these are predefined. [Model 1 > 10% blend > 10% Model 2 > 10% blend > Model 1]
25
+ merge_method: "slerp"
26
+
27
+ # If set to true, the lm_head and embed_token tensors (located outside the layers) will also be optimized
28
+ # Models that have a different vocab size from model1 will skip this phase automatically as it tends to cause model stability issues
29
+ merge_headers: true
30
+
31
+ # Strategies:
32
+ # "cumulative" - Default strategy. If there's a chance of reducing the combined probability, accept the merge.
33
+ # "all_phrases" - Only accept the merge if all phrases show an improvement. (Warning: This rarely happens)
34
+ # "quantitive" - Ignores probabilities completely. Only looks at how many phrases show an improvement, as defined by the threshold below.
35
+ strategy: "cumulative"
36
+ # Threshold is currently only used by the "quantitive" strategy. If 0.6, at least 60% of the number of phrases must show am improvement.
37
+ strategy_threshold: 0.6
38
+
39
+ # Whether or not to automatically balance the weights so all phrases are of equal importance to the "cumulative" strategy.
40
+ # The weight value of phrases is ignored if set to true.
41
+ auto_weights: false
42
+
43
+ # Phrase = What to measure, weight = multiplication factor, contexts = proceeding contexts
44
+ bad_phrases:
45
+ - phrase: "anticipation"
46
+ weight: 12
47
+ contexts: ["Her body quivers with ", "The atmosphere is thick with "]
48
+ - phrase: "unwavering"
49
+ weight: 12
50
+ contexts: ["Filled with an "]
51
+ - phrase: "determination"
52
+ weight: 12
53
+ contexts: ["Her eyes were filled with ", "Her stubbornness only fuels my "]
54
+ - phrase: "whisper"
55
+ weight: 12
56
+ contexts: ["Her voice barely above a "]
57
+ - phrase: "spine"
58
+ weight: 12
59
+ contexts: ["shivers down her "]
60
+ - phrase: "sends shivers"
61
+ weight: 12
62
+ contexts: ["The thrill of the act "]
63
+ - phrase: "ministrations"
64
+ weight: 12
65
+ contexts: ["She moans and twitches at your "]
66
+ - phrase: "legs"
67
+ weight: 12
68
+ contexts: ["wraps her "]
69
+ - phrase: "imposing figure"
70
+ weight: 12
71
+ contexts: ["He had an "]
72
+ - phrase: "shared challenges"
73
+ weight: 12
74
+ contexts: ["Their bond strengthened through "]
75
+ - phrase: "bond"
76
+ weight: 12
77
+ contexts: ["forged a ", "an unspoken "]
78
+ - phrase: "enhance our experience"
79
+ weight: 12
80
+ contexts: ["I'm excited to see how "]
81
+ - phrase: "sense of vulnerability"
82
+ weight: 12
83
+ contexts: ["create a "]
84
+ - phrase: "dimensions of intimacy"
85
+ weight: 12
86
+ contexts: ["explore new "]
87
+ - phrase: "deepening our connection"
88
+ weight: 12
89
+ contexts: ["while "]
90
+ - phrase: "shared experiences"
91
+ weight: 12
92
+ contexts: ["through "]
93
+ - phrase: "societal expectations"
94
+ weight: 12
95
+ contexts: ["that transcend "]
96
+ - phrase: "conventional boundaries"
97
+ weight: 12
98
+ contexts: ["that defy ", "and defy "]
99
+ - phrase: "open communication"
100
+ weight: 12
101
+ contexts: ["an environment "]
102
+ - phrase: "emotional vulnerability"
103
+ weight: 12
104
+ contexts: ["an environment "]
105
+ - phrase: "heightens our connection"
106
+ weight: 12
107
+ contexts: ["touch and the anticipation "]
108
+ - phrase: "sensations you're creating"
109
+ weight: 12
110
+ contexts: ["I'm enjoying "]
111
+ - phrase: "is truly arousing"
112
+ weight: 12
113
+ contexts: ["attention to detail ", "way you explore my body "]
114
+ - phrase: "challenge presented"
115
+ weight: 12
116
+ contexts: ["my resolve unwavering despite "]
117
+ - phrase: "humble vessel"
118
+ weight: 12
119
+ contexts: ["surrendering to the exquisite torment "]
120
+ - phrase: "bond"
121
+ weight: 12
122
+ contexts: ["cherishing the unique ", "special "]
123
+ - phrase: "grows stronger with each passing day"
124
+ weight: 12
125
+ contexts: ["bond "]
126
+ - phrase: "that cannot be broken by time or circumstance"
127
+ weight: 12
128
+ contexts: ["bond "]
129
+ - phrase: "becomes unbreakable, eternal"
130
+ weight: 12
131
+ contexts: ["bond "]
132
+ - phrase: "grew stronger with each passing"
133
+ weight: 12
134
+ contexts: ["bond "]
135
+
136
+
137
+ # Note - Example of a complex phrase
138
+ good_phrases:
139
+ - phrase: "The apple is in the bedroom"
140
+ weight: 1
141
+ contexts: ["Question: If I'm in the living room and pick up the apple, go to the bedroom and drop the apple, then walk to the kitchen, where is the apple? Explain your reasoning. Answer: "]
142
+