| # DINO Sparse Autoencoder Checkpoint | |
| This model is a sparse autoencoder trained on DINO's internal representations. | |
| ## Model Details | |
| ### Architecture | |
| - **Layer**: 2 | |
| - **Layer Type**: hook_resid_post | |
| - **Model**: facebook/vitb-16 | |
| - **Dictionary Size**: 49152 | |
| - **Input Dimension**: 768 | |
| - **Expansion Factor**: 64 | |
| - **CLS Token Only**: False | |
| ### Training | |
| - **Learning Rate**: 0.0061296854816475805 | |
| - **Batch Size**: 4096 | |
| - **Context Size**: 197 | |
| ### Sparsity | |
| - **L0 (Active Features)**: 661 | |
| ## Additional Information | |
| - **Wandb Run**: https://wandb.ai/perceptual-alignment/DINO-SAE-Vanilla-Sweep/runs/6bdgu49a/ | |
| - **Random Seed**: 42 | |