Improve model card: add library_name, detailed description, uses, and sample usage
#3
by
nielsr
HF Staff
- opened
README.md
CHANGED
@@ -9,28 +9,69 @@ tags:
|
|
9 |
- remote-sensing
|
10 |
- sentinel
|
11 |
- foundation-model
|
|
|
12 |
---
|
13 |
|
14 |
# Model Card for Copernicus-FM
|
15 |
|
16 |
-
|
17 |
-
|
18 |
-
[Copernicus-FM](https://github.com/zhu-xlab/Copernicus-FM) is an extension of the [DOFA](https://github.com/zhu-xlab/DOFA) foundation model, able to process any spectral or non-spectral sensor modality using extended dynamic hypernetworks and flexible metadata encoding. The model is pretrained on the Copernicus-Pretrain dataset with masked image modeling and continual distillation.
|
19 |
|
20 |
## Model Description
|
21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
22 |
## Uses
|
23 |
|
24 |
-
|
|
|
|
|
|
|
25 |
|
26 |
-
|
27 |
|
28 |
-
|
29 |
-
- **Paper:** https://arxiv.org/abs/2503.11849
|
30 |
|
31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
32 |
|
33 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
34 |
|
35 |
```bibtex
|
36 |
@misc{wang2025unifiedcopernicusfoundationmodel,
|
|
|
9 |
- remote-sensing
|
10 |
- sentinel
|
11 |
- foundation-model
|
12 |
+
library_name: transformers
|
13 |
---
|
14 |
|
15 |
# Model Card for Copernicus-FM
|
16 |
|
17 |
+
[Copernicus-FM](https://github.com/zhu-xlab/Copernicus-FM) is an extension of the [DOFA](https://github.com/zhu-xlab/DOFA) foundation model, able to process any spectral or non-spectral sensor modality using extended dynamic hypernetworks and flexible metadata encoding. The model is pretrained on the [Copernicus-Pretrain](https://huggingface.co/datasets/wangyi111/Copernicus-Pretrain) dataset with masked image modeling and continual distillation. It was introduced in the paper [Towards a Unified Copernicus Foundation Model for Earth Vision](https://arxiv.org/abs/2503.11849).
|
|
|
|
|
18 |
|
19 |
## Model Description
|
20 |
|
21 |
+
Copernicus-FM is a unified foundation model designed for Earth Observation (EO). It takes a significant step towards next-generation EO foundation models by addressing limitations of existing efforts, such as fixed spectral sensors and overlooked metadata. The model leverages extended dynamic hypernetworks and flexible metadata encoding to process any spectral or non-spectral sensor modality, from the Earth's surface to its atmosphere.
|
22 |
+
|
23 |
+
This model is part of a larger framework that includes:
|
24 |
+
- **Copernicus-Pretrain**: A massive-scale pretraining dataset comprising 18.7 million aligned images from all major Copernicus Sentinel missions (S1-S5P).
|
25 |
+
- **Copernicus-FM**: The unified foundation model pretrained on Copernicus-Pretrain using masked image modeling and continual distillation.
|
26 |
+
- **Copernicus-Bench**: A systematic evaluation benchmark featuring 15 hierarchical downstream tasks across various Sentinel missions, ranging from preprocessing to specialized applications.
|
27 |
+
- **Copernicus-Embed-025deg**: A global embedding map derived from Copernicus-FM, providing highly compressed representations of satellite observations.
|
28 |
+
|
29 |
## Uses
|
30 |
|
31 |
+
Copernicus-FM aims to learn generic representations from space, benefiting a wide range of downstream applications crucial to our planet. Its capabilities greatly improve the scalability, versatility, and multimodal adaptability of Earth observation foundation models. Key uses include:
|
32 |
+
- **Generic Feature Extraction**: Providing robust features from diverse spectral and non-spectral satellite imagery.
|
33 |
+
- **Downstream EO Applications**: Supporting tasks within the [Copernicus-Bench](https://huggingface.co/datasets/wangyi111/Copernicus-Bench) benchmark, such as land use/land cover classification and segmentation (e.g., EuroSAT, BigEarthNet, DFC2020), cloud segmentation, flood detection, and air quality regression.
|
34 |
+
- **Connecting Earth Sciences**: Facilitating new opportunities to connect Earth Observation, weather, and climate research by providing a unified model for analyzing various forms of satellite data.
|
35 |
|
36 |
+
## Sample Usage
|
37 |
|
38 |
+
You can use Copernicus-FM for image feature extraction with the Hugging Face `transformers` library.
|
|
|
39 |
|
40 |
+
```python
|
41 |
+
from transformers import AutoModel, AutoProcessor
|
42 |
+
from PIL import Image
|
43 |
+
import requests
|
44 |
+
import torch
|
45 |
+
|
46 |
+
# Load model and processor
|
47 |
+
model_name = "wangyi111/Copernicus-FM"
|
48 |
+
model = AutoModel.from_pretrained(model_name)
|
49 |
+
processor = AutoProcessor.from_pretrained(model_name)
|
50 |
+
|
51 |
+
# Example image (replace with your Sentinel image or local path)
|
52 |
+
# Using an image from the GitHub repo for demonstration purposes
|
53 |
+
image_url = "https://raw.githubusercontent.com/zhu-xlab/Copernicus-FM/main/assets/altogether-1.png"
|
54 |
+
image = Image.open(requests.get(image_url, stream=True).raw).convert("RGB")
|
55 |
|
56 |
+
# Prepare inputs
|
57 |
+
inputs = processor(images=image, return_tensors="pt")
|
58 |
+
|
59 |
+
# Get image features
|
60 |
+
with torch.no_grad():
|
61 |
+
outputs = model(**inputs)
|
62 |
+
|
63 |
+
# For image-feature-extraction models, the last_hidden_state often contains the features
|
64 |
+
image_features = outputs.last_hidden_state
|
65 |
+
print(f"Image features shape: {image_features.shape}")
|
66 |
+
# Example output shape: torch.Size([1, N, hidden_size]) where N is number of patches/tokens
|
67 |
+
```
|
68 |
+
|
69 |
+
## Related Sources
|
70 |
+
|
71 |
+
- **Repository:** https://github.com/zhu-xlab/Copernicus-FM
|
72 |
+
- **Paper:** https://arxiv.org/abs/2503.11849
|
73 |
+
|
74 |
+
## Citation
|
75 |
|
76 |
```bibtex
|
77 |
@misc{wang2025unifiedcopernicusfoundationmodel,
|