---
viewer: true

license: cc-by-nc-sa-4.0
language:
- en
tags:
- spatial-transcriptomics
- histology
- pathology
- benchmark
task_categories:
- feature-extraction
extra_gated_prompt: >-
  - This model and associated code are released under the [CC-BY-NC-ND 4.0 license](https://creativecommons.org/licenses/by-nc-nd/4.0/) and may only be used for non-commercial, academic research purposes with proper attribution. 
  - Any commercial use, sale, or other monetization of the model and its derivatives, is prohibited and requires prior approval. 
  - By downloading the model, you attest that all information (affiliation, research use) is correct and up-to-date. Downloading the model requires prior registration on Hugging Face and agreeing to the terms of use. By downloading this model, you agree not to distribute, publish or reproduce a copy of the model. If another user within your organization wishes to use the model, they must register as an individual user and agree to comply with the terms of use. 
  - This model is provided “as-is” without warranties of any kind, express or implied. This model has not been reviewed, certified, or approved by any regulatory body, including but not limited to the FDA (U.S.), EMA (Europe), MHRA (UK), or other medical device authorities. Any application of this model in healthcare or biomedical settings must comply with relevant regulatory requirements and undergo independent validation. Users assume full responsibility for how they use this model and any resulting consequences. The authors, contributors, and distributors disclaim any liability for damages, direct or indirect, resulting from model use. Users are responsible for ensuring compliance with data protection regulations (e.g., GDPR, HIPAA) when using it in research that involves patient data.
extra_gated_fields:
  Full Name (first and last): text
  Type of Affiliation:
    type: select
    options:
      - Industry
      - Academia
      - Other
  Current Affiliation (no abbreviations): text
  Current and Official Institutional Email: text
  Main use-case:
    type: select
    options:
      - Models Benchmarking
      - Biomarker Discovery
      - Diagnostics
      - Pathology Workflows Acceleration
      - Other
  Please add information on your intended research use: text
  I agree to use this dataset for non-commercial, academic purposes only: checkbox
  I agree not to distribute the dataset: checkbox
---

# HESCAPE • DRVI models

HESCAPE (**H&E + Spatial Contrastive Pretraining Benchmark**) is a large-scale benchmark for multimodal learning in spatial transcriptomics.  
This repository hosts the **DRVI models** for HESCAPE, organized by dataset panel.

---
## Available DRVI Models
This dataset repo exposes the following drvi models and corresponding UMAP plots:

- `human-5k-panel`  
- `human-breast-panel`  
- `human-colon-panel`  
- `human-immuno-oncology-panel`  
- `human-lung-healthy-panel`  
- `human-multi-tissue-panel`

Each model corresponds to an independent HESCAPE dataset gene panel.

---
## Usage
The [HESCAPE repository](https://github.com/peng-lab/hescape) takes pretrained weights for pre-built images and genes to train the model. ***The directory structure is crucial for the training process to work correctly.*** The repository is structured as follows:

```bash
├── hescape (from github)
│   ├── README.md
│   ├── data
│   ├── experiments
│   ├── notebooks
│   ├── pyproject.toml
│   ├── src
│   ├── tests
│   ├── uv.lock
│   └── ...
├── pretrain_weights
│   ├── gene
│   │   ├── nicheformer
│   │   ├── drvi
│   │   └── <predefined gene models> ...
│   └── image
│        ├── h0-mini
│        ├── uni
│        └── <predefined image models> ...
```
Copy the drvi models as in in the pretrain_weights/gene/ folder as given in the directory. Further instructions on how to use HESCAPE are provided [here](https://github.com/peng-lab/hescape)

### How to cite:
```
@misc{gindra2025largescalebenchmarkcrossmodallearning,
      title={A Large-Scale Benchmark of Cross-Modal Learning for Histology and Gene Expression in Spatial Transcriptomics}, 
      author={Rushin H. Gindra and Giovanni Palla and Mathias Nguyen and Sophia J. Wagner and Manuel Tran and Fabian J Theis and Dieter Saur and Lorin Crawford and Tingying Peng},
      year={2025},
      eprint={2508.01490},
      archivePrefix={arXiv},
      primaryClass={q-bio.GN},
      url={https://arxiv.org/abs/2508.01490}, 
}
```
### Contact:
- <b>Rushin Gindra</b> Helmholtz Munich, Munich (`rushin.gindra@helmholtz-munich.de`)
- 
<i>The dataset is distributed under the Attribution-NonCommercial-ShareAlike 4.0 International license (CC BY-NC-SA 4.0 Deed)</i>