noanabeshima commited on
Commit
7a11068
·
verified ·
1 Parent(s): 3813785

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -27,4 +27,5 @@ def load_feat_acts(fname, only_active_docs=False):
27
  feat_acts = load_feat_acts(f"mlp_map_test/M2_S-2_R1_P0/{300}.pt").to_dense()
28
  ```
29
 
30
- The activations are for the train split in https://huggingface.co/datasets/noanabeshima/TinyModelTokIds. Not all activations are currently included, just a prefix of the train split.
 
 
27
  feat_acts = load_feat_acts(f"mlp_map_test/M2_S-2_R1_P0/{300}.pt").to_dense()
28
  ```
29
 
30
+ The activations are from the 13% of the train split of https://huggingface.co/datasets/noanabeshima/TinyModelTokIds.
31
+ I gather all the activations per feature and then take the smallest prefix so that each feature has at least 3K documents on which it activates and 12K token-activations.