---
library_name: pytorch
tags:
- computer-vision
- pose-estimation
- panoramic-images
- covispose
- pytorch-model
license: mit
base_model: resnet50
---

# CovisPose Model

This model estimates relative poses between panoramic images using the CovisPose framework.

## Model Details

- **Architecture**: CovisPose with resnet50 backbone
- **Transformer Layers**: 6
- **FFN Dimension**: 2048
- **Input Size**: [512, 1024]
- **Parameters**: 121,890,467 (estimated)

## Training Information

### Configuration
- **Epochs**: N/A
- **Batch Size**: N/A
- **Learning Rate**: N/A
- **Backbone**: N/A

### Performance Metrics
- **Final Training Loss**: N/A
- **Training Rotation Error**: N/A
- **Final Validation Loss**: N/A
- **Validation Rotation Error**: N/A

## Usage

```python
import torch
import json
from huggingface_hub import hf_hub_download

# Download model files
model_path = hf_hub_download(
    repo_id="SGEthan/covis_toy",
    filename="pytorch_model.bin"
)

config_path = hf_hub_download(
    repo_id="SGEthan/covis_toy",
    filename="config.json"
)

# Load configuration
with open(config_path, 'r') as f:
    config = json.load(f)

# Initialize model (you'll need the COVIS class)
from models.covispose_model import COVIS

model = COVIS(
    backbone=config['backbone'],
    num_transformer_layers=config['num_transformer_layers'],
    transformer_ffn_dim=config['transformer_ffn_dim']
)

# Load weights
checkpoint = torch.load(model_path, map_location='cpu')
if "model_state_dict" in checkpoint:
    model.load_state_dict(checkpoint["model_state_dict"])
else:
    model.load_state_dict(checkpoint)

model.eval()

# Use for inference
with torch.no_grad():
    # Your inference code here
    # outputs1, outputs2 = model(pano1_tensor, pano2_tensor)
    pass
```

## Model Architecture

The CovisPose model consists of:

1. **Backbone Network**: resnet50 for feature extraction
2. **Transformer Encoder**: 6 layers for processing image features
3. **Prediction Heads**:
   - Covisibility mask prediction
   - Relative pose estimation
   - Boundary detection

## Task Description

**CovisPose** estimates the relative pose between two panoramic images by:

1. **Covisibility Estimation**: Predicting which parts of the images overlap
2. **Pose Regression**: Estimating relative rotation and translation
3. **Boundary Detection**: Finding floor-wall boundaries for scale estimation

## Training Data

This model was trained on panoramic image pairs with:
- Relative pose annotations
- Covisibility masks
- Floor-wall boundary labels

## Limitations

- Designed specifically for indoor panoramic images
- Requires significant visual overlap between image pairs for reliable pose estimation
- Performance may degrade on outdoor scenes or images with minimal overlap

## Citation

If you use this model, please cite the CovisPose work:

```bibtex
@article{covispose2024,
  title={CovisPose: Co-visibility Pose Estimation for Panoramic Images},
  author={Your Authors},
  journal={Conference/Journal},
  year={2024}
}
```

## License

This model is released under the MIT License.

## Repository

- **Training Code**: Available in the original repository
- **Model Upload**: Generated automatically from local checkpoint

---

*Model uploaded on 2025-06-27T09:05:36.254713 using upload_model.py*