Update README.md
Browse files
README.md
CHANGED
@@ -27,5 +27,5 @@ def load_feat_acts(fname, only_active_docs=False):
|
|
27 |
feat_acts = load_feat_acts(f"mlp_map_test/M2_S-2_R1_P0/{300}.pt").to_dense()
|
28 |
```
|
29 |
|
30 |
-
The activations are from
|
31 |
-
I gather all the activations per feature and then take the smallest prefix so that each feature has at least 3K documents on which it activates and 12K token-activations.
|
|
|
27 |
feat_acts = load_feat_acts(f"mlp_map_test/M2_S-2_R1_P0/{300}.pt").to_dense()
|
28 |
```
|
29 |
|
30 |
+
The activations are from 13% of the train split of https://huggingface.co/datasets/noanabeshima/TinyModelTokIds.
|
31 |
+
I gather all the activations per feature and then take the smallest activations prefix so that each feature has at least 3K documents on which it activates and 12K token-activations.
|