Improve model card: add library_name, detailed description, uses, and sample usage

#3
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +50 -9
README.md CHANGED
@@ -9,28 +9,69 @@ tags:
9
  - remote-sensing
10
  - sentinel
11
  - foundation-model
 
12
  ---
13
 
14
  # Model Card for Copernicus-FM
15
 
16
- <!-- Provide a quick summary of what the model is/does. -->
17
-
18
- [Copernicus-FM](https://github.com/zhu-xlab/Copernicus-FM) is an extension of the [DOFA](https://github.com/zhu-xlab/DOFA) foundation model, able to process any spectral or non-spectral sensor modality using extended dynamic hypernetworks and flexible metadata encoding. The model is pretrained on the Copernicus-Pretrain dataset with masked image modeling and continual distillation.
19
 
20
  ## Model Description
21
 
 
 
 
 
 
 
 
 
22
  ## Uses
23
 
24
- ## Related Sources
 
 
 
25
 
26
- <!-- Provide the basic links for the model. -->
27
 
28
- - **Repository:** https://github.com/zhu-xlab/Copernicus-FM
29
- - **Paper:** https://arxiv.org/abs/2503.11849
30
 
31
- ## Citation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
 
33
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
34
 
35
  ```bibtex
36
  @misc{wang2025unifiedcopernicusfoundationmodel,
 
9
  - remote-sensing
10
  - sentinel
11
  - foundation-model
12
+ library_name: transformers
13
  ---
14
 
15
  # Model Card for Copernicus-FM
16
 
17
+ [Copernicus-FM](https://github.com/zhu-xlab/Copernicus-FM) is an extension of the [DOFA](https://github.com/zhu-xlab/DOFA) foundation model, able to process any spectral or non-spectral sensor modality using extended dynamic hypernetworks and flexible metadata encoding. The model is pretrained on the [Copernicus-Pretrain](https://huggingface.co/datasets/wangyi111/Copernicus-Pretrain) dataset with masked image modeling and continual distillation. It was introduced in the paper [Towards a Unified Copernicus Foundation Model for Earth Vision](https://arxiv.org/abs/2503.11849).
 
 
18
 
19
  ## Model Description
20
 
21
+ Copernicus-FM is a unified foundation model designed for Earth Observation (EO). It takes a significant step towards next-generation EO foundation models by addressing limitations of existing efforts, such as fixed spectral sensors and overlooked metadata. The model leverages extended dynamic hypernetworks and flexible metadata encoding to process any spectral or non-spectral sensor modality, from the Earth's surface to its atmosphere.
22
+
23
+ This model is part of a larger framework that includes:
24
+ - **Copernicus-Pretrain**: A massive-scale pretraining dataset comprising 18.7 million aligned images from all major Copernicus Sentinel missions (S1-S5P).
25
+ - **Copernicus-FM**: The unified foundation model pretrained on Copernicus-Pretrain using masked image modeling and continual distillation.
26
+ - **Copernicus-Bench**: A systematic evaluation benchmark featuring 15 hierarchical downstream tasks across various Sentinel missions, ranging from preprocessing to specialized applications.
27
+ - **Copernicus-Embed-025deg**: A global embedding map derived from Copernicus-FM, providing highly compressed representations of satellite observations.
28
+
29
  ## Uses
30
 
31
+ Copernicus-FM aims to learn generic representations from space, benefiting a wide range of downstream applications crucial to our planet. Its capabilities greatly improve the scalability, versatility, and multimodal adaptability of Earth observation foundation models. Key uses include:
32
+ - **Generic Feature Extraction**: Providing robust features from diverse spectral and non-spectral satellite imagery.
33
+ - **Downstream EO Applications**: Supporting tasks within the [Copernicus-Bench](https://huggingface.co/datasets/wangyi111/Copernicus-Bench) benchmark, such as land use/land cover classification and segmentation (e.g., EuroSAT, BigEarthNet, DFC2020), cloud segmentation, flood detection, and air quality regression.
34
+ - **Connecting Earth Sciences**: Facilitating new opportunities to connect Earth Observation, weather, and climate research by providing a unified model for analyzing various forms of satellite data.
35
 
36
+ ## Sample Usage
37
 
38
+ You can use Copernicus-FM for image feature extraction with the Hugging Face `transformers` library.
 
39
 
40
+ ```python
41
+ from transformers import AutoModel, AutoProcessor
42
+ from PIL import Image
43
+ import requests
44
+ import torch
45
+
46
+ # Load model and processor
47
+ model_name = "wangyi111/Copernicus-FM"
48
+ model = AutoModel.from_pretrained(model_name)
49
+ processor = AutoProcessor.from_pretrained(model_name)
50
+
51
+ # Example image (replace with your Sentinel image or local path)
52
+ # Using an image from the GitHub repo for demonstration purposes
53
+ image_url = "https://raw.githubusercontent.com/zhu-xlab/Copernicus-FM/main/assets/altogether-1.png"
54
+ image = Image.open(requests.get(image_url, stream=True).raw).convert("RGB")
55
 
56
+ # Prepare inputs
57
+ inputs = processor(images=image, return_tensors="pt")
58
+
59
+ # Get image features
60
+ with torch.no_grad():
61
+ outputs = model(**inputs)
62
+
63
+ # For image-feature-extraction models, the last_hidden_state often contains the features
64
+ image_features = outputs.last_hidden_state
65
+ print(f"Image features shape: {image_features.shape}")
66
+ # Example output shape: torch.Size([1, N, hidden_size]) where N is number of patches/tokens
67
+ ```
68
+
69
+ ## Related Sources
70
+
71
+ - **Repository:** https://github.com/zhu-xlab/Copernicus-FM
72
+ - **Paper:** https://arxiv.org/abs/2503.11849
73
+
74
+ ## Citation
75
 
76
  ```bibtex
77
  @misc{wang2025unifiedcopernicusfoundationmodel,