metadata
license: mit
Cached SAE/transcoder acts stored in CSR format. Not especially optimized for others' use/fleshed out.
If you want to use them, do
import torch
def load_feat_acts(fname):
csr_kwargs = torch.load(fname)
# The matrices are stored in space-efficient formats that're incompatible with torch's sparse csr tensor.
# Convert them back before constructing the matrix.
csr_kwargs['crow_indices'] = csr_kwargs['crow_indices'].int()
csr_kwargs['col_indices'] = csr_kwargs['crow_indices'].int()
csr_kwargs['values'] = csr_kwargs['values'].float()/255
feat_acts = torch.sparse_csr_tensor(**csr_kwargs)
return feat_acts
The activations are for the train split in https://huggingface.co/datasets/noanabeshima/TinyModelTokIds