metadata
license: mit
Cached SAE/transcoder acts stored in CSR format. Not especially optimized for others' use/fleshed out. It's mostly just a space for me to place and download activations.
If you want to use them, do
import torch
from huggingface_hub import hf_hub_download
def load_feat_acts(fname, only_active_docs=False):
local_path = hf_hub_download(repo_id="noanabeshima/tiny_model_cached_acts", filename=fname)
csr_kwargs = torch.load(local_path)
# The matrices are stored in space-efficient formats that're incompatible with torch's sparse csr tensor.
# Convert them back before constructing the matrix.
csr_kwargs['crow_indices'] = csr_kwargs['crow_indices'].int()
csr_kwargs['col_indices'] = csr_kwargs['col_indices'].int()
csr_kwargs['values'] = (csr_kwargs['values'].float()/255)
feat_acts = torch.sparse_csr_tensor(**csr_kwargs)
return feat_acts
feat_acts = load_feat_acts(f"mlp_map_test/M2_S-2_R1_P0/{300}.pt").to_dense()
The activations are for the train split in https://huggingface.co/datasets/noanabeshima/TinyModelTokIds. Not all activations are currently included, just a prefix of the train split.