# DeepEncoder (Extracted from DeepSeek-OCR)

## Overview
This directory contains the encoder components extracted from DeepSeek-OCR.

## Model Files
- `sam_encoder.pth`: SAM ViT-B encoder (95,569,152 params, 364.6 MB)
- `clip_encoder.pth`: CLIP-Large encoder (303,177,728 params, 1156.6 MB)
- `projector.pth`: Linear projector (2,622,720 params, 10.0 MB)
- `config.json`: Model configuration

**Total:** 401,369,600 parameters

## Architecture
```
Image (1024×1024) → SAM (95M) → 16× Conv → CLIP (303M) → Projector (3M) → 256 vision tokens
```

## Usage
```python
import torch
from deepencoder import build_sam_vit_b, build_clip_l, MlpProjector
from easydict import EasyDict as adict

# Load models
sam = build_sam_vit_b(checkpoint=None)
sam.load_state_dict(torch.load('sam_encoder.pth'))

clip = build_clip_l()
clip.load_state_dict(torch.load('clip_encoder.pth'))

projector_cfg = adict({'projector_type': 'linear', 'input_dim': 2048, 'n_embed': 1280})
projector = MlpProjector(projector_cfg)
projector.load_state_dict(torch.load('projector.pth'))

# Run encoder
vision_tokens = encode(image)  # [1, 256, 1280]
```

## Training
These weights are:
- Initialized from pretrained SAM (SA-1B) + CLIP (LAION-2B)
- Fine-tuned together on optical compression/OCR tasks
- Optimized for text preservation in compressed form

## Source
Extracted from: [deepseek-ai/DeepSeek-OCR](https://huggingface.co/deepseek-ai/DeepSeek-OCR)