|
|
--- |
|
|
license: mit |
|
|
--- |
|
|
Cached SAE/transcoder acts stored in CSR format. Not especially optimized for others' use/fleshed out. |
|
|
|
|
|
If you want to use them, do |
|
|
|
|
|
``` |
|
|
import torch |
|
|
|
|
|
def load_feat_acts(fname): |
|
|
csr_kwargs = torch.load(fname) |
|
|
|
|
|
# The matrices are stored in space-efficient formats that're incompatible with torch's sparse csr tensor. |
|
|
# Convert them back before constructing the matrix. |
|
|
csr_kwargs['crow_indices'] = csr_kwargs['crow_indices'].int() |
|
|
csr_kwargs['col_indices'] = csr_kwargs['crow_indices'].int() |
|
|
csr_kwargs['values'] = csr_kwargs['values'].float()/255 |
|
|
|
|
|
feat_acts = torch.sparse_csr_tensor(**csr_kwargs) |
|
|
return feat_acts |
|
|
``` |
|
|
|
|
|
The activations are for the train split in https://huggingface.co/datasets/noanabeshima/TinyModelTokIds |
|
|
|
