suchitg/sae-compression-gemma-2-2b-pruned-sae-pile-0.25

SAEs for use with the SAELens library

This repository contains the following SAEs:

• blocks.0.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.1.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.2.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.3.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.4.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.5.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.6.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.7.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.8.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.9.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.10.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.11.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.12.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.13.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.14.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.15.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.16.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.17.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.18.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.19.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.20.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.21.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.22.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.23.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.24.(hook_resid_post, hook_mlp_out, attn.hook_z)
• blocks.25.(hook_resid_post, hook_mlp_out, attn.hook_z)

Load these SAEs using SAELens as below:

from sae_lens import SAE

sae, cfg_dict, sparsity = SAE.from_pretrained("suchitg/sae-compression-gemma-2-2b-pruned-sae-pile-0.25", "<sae_id>")