suchitg/sae-compression-gpt-2-small-pruned-sae-pile_0.25

SAEs for use with the SAELens library

This repository contains the following SAEs:

• blocks.0.hook_resid_post
• blocks.1.hook_resid_post
• blocks.2.hook_resid_post
• blocks.3.hook_resid_post
• blocks.4.hook_resid_post
• blocks.5.hook_resid_post
• blocks.6.hook_resid_post
• blocks.7.hook_resid_post
• blocks.8.hook_resid_post
• blocks.9.hook_resid_post
• blocks.10.hook_resid_post
• blocks.11.hook_resid_post
• blocks.0.hook_mlp_out
• blocks.1.hook_mlp_out
• blocks.2.hook_mlp_out
• blocks.3.hook_mlp_out
• blocks.4.hook_mlp_out
• blocks.5.hook_mlp_out
• blocks.6.hook_mlp_out
• blocks.7.hook_mlp_out
• blocks.8.hook_mlp_out
• blocks.9.hook_mlp_out
• blocks.10.hook_mlp_out
• blocks.11.hook_mlp_out
• blocks.0.attn.hook_z
• blocks.1.attn.hook_z
• blocks.2.attn.hook_z
• blocks.3.attn.hook_z
• blocks.4.attn.hook_z
• blocks.5.attn.hook_z
• blocks.6.attn.hook_z
• blocks.7.attn.hook_z
• blocks.8.attn.hook_z
• blocks.9.attn.hook_z
• blocks.10.attn.hook_z
• blocks.11.attn.hook_z

Load these SAEs using SAELens as below:

from sae_lens import SAE

sae, cfg_dict, sparsity = SAE.from_pretrained("suchitg/sae-compression-gpt-2-small-pruned-sae-pile_0.25", "<sae_id>")