Update README.md

README.md

@@ -0,0 +1,47 @@
+# CLIP Sparse Autoencoder Checkpoint
+
+This model is a sparse autoencoder trained on CLIP's internal representations.
+
+## Model Details
+
+### Architecture
+- **Layer**: 3
+- **Layer Type**: hook_resid_post
+- **Model**: open-clip:laion/CLIP-ViT-B-32-DataComp.XL-s13B-b90K
+- **Dictionary Size**: 49152
+- **Input Dimension**: 768
+- **Expansion Factor**: 64
+- **CLS Token Only**: False
+
+### Training
+- **Training Images**: 1299988
+- **Learning Rate**: 0.0000
+- **L1 Coefficient**: 0.0000
+- **Batch Size**: 4096
+- **Context Size**: 50
+
+## Performance Metrics
+
+### Sparsity
+- **L0 (Active Features)**: 64.0000
+- **Dead Features**: 0
+
+- **Mean Passes Since Fired**: 0.1956
+
+### Reconstruction
+- **Explained Variance**: 0.7059
+- **Explained Variance Std**: 0.0623
+- **MSE Loss**: 0.0025
+- **L1 Loss**: 0
+- **Overall Loss**: 0.0025
+
+## Training Details
+- **Training Duration**: 5113 seconds
+- **Final Learning Rate**: 0.0000
+- **Warm Up Steps**: 200
+- **Gradient Clipping**: 1
+
+## Additional Information
+- **Original Checkpoint Path**: /network/scratch/p/praneet.suresh/imgnet_checkpoints/b8adc3bf-tinyclip_sae_16_hyperparam_sweep_lr/n_images_1300070.pt
+- **Wandb Run**: https://wandb.ai/perceptual-alignment/topk-imagenet-all_patches-sweep/runs/4adj74y4
+- **Random Seed**: 42