Commit
Β·
e0466ed
1
Parent(s):
b54fd81
Move files
Browse filesThis view is limited to 50 files because it contains too many changes. Β
See raw diff
- README.md +45 -65
- finetuned/classification_cvd/{acdc_sax_0.safetensors β acdc_sax/acdc_sax_0.safetensors} +0 -0
- finetuned/classification_cvd/{acdc_sax_1.safetensors β acdc_sax/acdc_sax_1.safetensors} +0 -0
- finetuned/classification_cvd/{acdc_sax_2.safetensors β acdc_sax/acdc_sax_2.safetensors} +0 -0
- finetuned/classification_cvd/{acdc_sax.yaml β acdc_sax/config.yaml} +0 -0
- finetuned/classification_cvd/{mnms2_lax_4c.yaml β mnms2_lax_4c/config.yaml} +0 -0
- finetuned/classification_cvd/{mnms2_lax_4c_0.safetensors β mnms2_lax_4c/mnms2_lax_4c_0.safetensors} +0 -0
- finetuned/classification_cvd/{mnms2_lax_4c_1.safetensors β mnms2_lax_4c/mnms2_lax_4c_1.safetensors} +0 -0
- finetuned/classification_cvd/{mnms2_lax_4c_2.safetensors β mnms2_lax_4c/mnms2_lax_4c_2.safetensors} +0 -0
- finetuned/classification_cvd/{mnms2_sax.yaml β mnms2_sax/config.yaml} +0 -0
- finetuned/classification_cvd/{mnms2_sax_0.safetensors β mnms2_sax/mnms2_sax_0.safetensors} +0 -0
- finetuned/classification_cvd/{mnms2_sax_1.safetensors β mnms2_sax/mnms2_sax_1.safetensors} +0 -0
- finetuned/classification_cvd/{mnms2_sax_2.safetensors β mnms2_sax/mnms2_sax_2.safetensors} +0 -0
- finetuned/classification_cvd/{mnms_sax.yaml β mnms_sax/config.yaml} +0 -0
- finetuned/classification_cvd/{mnms_sax_0.safetensors β mnms_sax/mnms_sax_0.safetensors} +0 -0
- finetuned/classification_cvd/{mnms_sax_1.safetensors β mnms_sax/mnms_sax_1.safetensors} +0 -0
- finetuned/classification_cvd/{mnms_sax_2.safetensors β mnms_sax/mnms_sax_2.safetensors} +0 -0
- finetuned/classification_sex/{mnms_sax.yaml β mnms_sax/config.yaml} +0 -0
- finetuned/classification_sex/{mnms_sax_0.safetensors β mnms_sax/mnms_sax_0.safetensors} +0 -0
- finetuned/classification_sex/{mnms_sax_1.safetensors β mnms_sax/mnms_sax_1.safetensors} +0 -0
- finetuned/classification_sex/{mnms_sax_2.safetensors β mnms_sax/mnms_sax_2.safetensors} +0 -0
- finetuned/classification_vendor/{mnms2_lax_4c.yaml β mnms2_lax_4c/config.yaml} +0 -0
- finetuned/classification_vendor/{mnms2_lax_4c_0.safetensors β mnms2_lax_4c/mnms2_lax_4c_0.safetensors} +0 -0
- finetuned/classification_vendor/{mnms2_lax_4c_1.safetensors β mnms2_lax_4c/mnms2_lax_4c_1.safetensors} +0 -0
- finetuned/classification_vendor/{mnms2_lax_4c_2.safetensors β mnms2_lax_4c/mnms2_lax_4c_2.safetensors} +0 -0
- finetuned/classification_vendor/{mnms2_sax.yaml β mnms2_sax/config.yaml} +0 -0
- finetuned/classification_vendor/{mnms2_sax_0.safetensors β mnms2_sax/mnms2_sax_0.safetensors} +0 -0
- finetuned/classification_vendor/{mnms2_sax_1.safetensors β mnms2_sax/mnms2_sax_1.safetensors} +0 -0
- finetuned/classification_vendor/{mnms2_sax_2.safetensors β mnms2_sax/mnms2_sax_2.safetensors} +0 -0
- finetuned/landmark_coordinate/{lax_2c.yaml β lax_2c/config.yaml} +0 -0
- finetuned/landmark_coordinate/{lax_2c_0.safetensors β lax_2c/lax_2c_0.safetensors} +0 -0
- finetuned/landmark_coordinate/{lax_2c_1.safetensors β lax_2c/lax_2c_1.safetensors} +0 -0
- finetuned/landmark_coordinate/{lax_2c_2.safetensors β lax_2c/lax_2c_2.safetensors} +0 -0
- finetuned/landmark_coordinate/{lax_4c.yaml β lax_4c/config.yaml} +0 -0
- finetuned/landmark_coordinate/{lax_4c_0.safetensors β lax_4c/lax_4c_0.safetensors} +0 -0
- finetuned/landmark_coordinate/{lax_4c_1.safetensors β lax_4c/lax_4c_1.safetensors} +0 -0
- finetuned/landmark_coordinate/{lax_4c_2.safetensors β lax_4c/lax_4c_2.safetensors} +0 -0
- finetuned/landmark_heatmap/{lax_2c.yaml β lax_2c/config.yaml} +0 -0
- finetuned/landmark_heatmap/{lax_2c_0.safetensors β lax_2c/lax_2c_0.safetensors} +0 -0
- finetuned/landmark_heatmap/{lax_2c_1.safetensors β lax_2c/lax_2c_1.safetensors} +0 -0
- finetuned/landmark_heatmap/{lax_2c_2.safetensors β lax_2c/lax_2c_2.safetensors} +0 -0
- finetuned/landmark_heatmap/{lax_4c.yaml β lax_4c/config.yaml} +0 -0
- finetuned/landmark_heatmap/{lax_4c_0.safetensors β lax_4c/lax_4c_0.safetensors} +0 -0
- finetuned/landmark_heatmap/{lax_4c_1.safetensors β lax_4c/lax_4c_1.safetensors} +0 -0
- finetuned/landmark_heatmap/{lax_4c_2.safetensors β lax_4c/lax_4c_2.safetensors} +0 -0
- finetuned/regression_age/{mnms_sax.yaml β mnms_sax/config.yaml} +0 -0
- finetuned/regression_age/{mnms_sax_0.safetensors β mnms_sax/mnms_sax_0.safetensors} +0 -0
- finetuned/regression_age/{mnms_sax_1.safetensors β mnms_sax/mnms_sax_1.safetensors} +0 -0
- finetuned/regression_age/{mnms_sax_2.safetensors β mnms_sax/mnms_sax_2.safetensors} +0 -0
- finetuned/regression_bmi/{acdc_sax_0.safetensors β acdc_sax/acdc_sax_0.safetensors} +0 -0
README.md
CHANGED
|
@@ -1,90 +1,70 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
language:
|
| 4 |
-
- en
|
| 5 |
datasets:
|
| 6 |
-
- mathpluscode/ACDC
|
| 7 |
tags:
|
| 8 |
-
- medical
|
| 9 |
-
- cardiac
|
| 10 |
-
- MRI
|
| 11 |
-
- foundation model
|
| 12 |
-
- MAE
|
| 13 |
---
|
| 14 |
|
| 15 |
# CineMA - A Foundation Model for Cine Cardiac Magnetic Resonance Images π₯π«
|
| 16 |
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
- Ventricle and myocardium segmentation
|
| 24 |
-
- Cardiovascular disease (CVD) detection and classification
|
| 25 |
-
- Patient sex classification
|
| 26 |
-
- CMR machine vendor classification
|
| 27 |
-
- Ejection fraction (EF) regression
|
| 28 |
-
- Patient body mass index (BMI) regression
|
| 29 |
-
- Patient age regression
|
| 30 |
-
- Mid-ventricular and apical landmark localization
|
| 31 |
-
|
| 32 |
-
These tasks were studied across multiple datasets:
|
| 33 |
-
|
| 34 |
-
- [ACDC](https://www.creatis.insa-lyon.fr/Challenge/acdc/)
|
| 35 |
-
- [M&Ms](https://www.ub.edu/mnms/)
|
| 36 |
-
- [M&Ms2](https://www.ub.edu/mnms-2/)
|
| 37 |
-
- [Kaggle](https://www.kaggle.com/c/second-annual-data-science-bowl/data)
|
| 38 |
-
- [Rescan](https://www.ahajournals.org/doi/full/10.1161/CIRCIMAGING.119.009214)
|
| 39 |
-
- [Landmark](https://pubs.rsna.org/doi/10.1148/ryai.2021200197)
|
| 40 |
-
|
| 41 |
-
Compared to convolutional neural network baselines such as UNet and ResNet, CineMA demonstrated superior or comparable
|
| 42 |
-
performance, especially in sample efficiency and generalization to out-of-distribution data not seen during pretraining
|
| 43 |
-
or fine-tuning.
|
| 44 |
-
|
| 45 |
-
By releasing the model weights and code for pretraining, fine-tuning, and inference, CineMA aims to lower the barrier to
|
| 46 |
-
entry for cardiac imaging research, foster reproducibility, and encourage broader adoption across institutions.
|
| 47 |
-
|
| 48 |
-
β‘οΈ **Manuscript:** [TBD](https://arxiv.org/)
|
| 49 |
|
| 50 |
β‘οΈ **Code:** [mathpluscode/CineMA](https://github.com/mathpluscode/CineMA)
|
| 51 |
|
| 52 |
## Fine-tuned CineMA Models
|
| 53 |
|
| 54 |
-
The filenames of fine-tuned model weights follow the convention of
|
| 55 |
-
where number 0, 1, and 2 correspond to the different
|
|
|
|
| 56 |
|
| 57 |
Check the "Inference Example" column to see example inference scripts using these trained models.
|
| 58 |
|
| 59 |
-
| Training Task | Training Data | Input View | Input Timeframes | Model Weights and Configurations
|
| 60 |
-
| ----------------------------------------------- | ------------- | ---------- | ---------------- |
|
| 61 |
-
| Ventricle and myocardium segmentation | ACDC | SAX | 1 | [finetuned/segmentation/acdc_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/acdc_sax_0.safetensors)<br>[finetuned/segmentation/acdc_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/acdc_sax_1.safetensors)<br>[finetuned/segmentation/acdc_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/acdc_sax_2.safetensors)<br>[finetuned/segmentation/
|
| 62 |
-
| Ventricle and myocardium segmentation | M&Ms | SAX | 1 | [finetuned/segmentation/mnms_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms_sax_0.safetensors)<br>[finetuned/segmentation/mnms_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms_sax_1.safetensors)<br>[finetuned/segmentation/mnms_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms_sax_2.safetensors)<br>[finetuned/segmentation/
|
| 63 |
-
| Ventricle and myocardium segmentation | M&Ms2 | SAX | 1 | [finetuned/segmentation/mnms2_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms2_sax_0.safetensors)<br>[finetuned/segmentation/mnms2_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms2_sax_1.safetensors)<br>[finetuned/segmentation/mnms2_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms2_sax_2.safetensors)<br>[finetuned/segmentation/
|
| 64 |
-
| Ventricle and myocardium segmentation | M&Ms2 | LAX 4C | 1 | [finetuned/segmentation/mnms2_lax_4c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms2_lax_4c_0.safetensors)<br>[finetuned/segmentation/mnms2_lax_4c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms2_lax_4c_1.safetensors)<br>[finetuned/segmentation/mnms2_lax_4c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms2_lax_4c_2.safetensors)<br>[finetuned/segmentation/
|
| 65 |
-
| CVD classification | ACDC | SAX | 2 (ED and ES) | [finetuned/classification_cvd/acdc_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/acdc_sax_0.safetensors)<br>[finetuned/classification_cvd/acdc_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/acdc_sax_1.safetensors)<br>[finetuned/classification_cvd/acdc_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/acdc_sax_2.safetensors)<br>[finetuned/classification_cvd/acdc_sax.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/acdc_sax.yaml)
|
| 66 |
-
| CVD classification | M&Ms | SAX | 2 (ED and ES) | [finetuned/classification_cvd/mnms_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms_sax_0.safetensors)<br>[finetuned/classification_cvd/mnms_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms_sax_1.safetensors)<br>[finetuned/classification_cvd/mnms_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms_sax_2.safetensors)<br>[finetuned/classification_cvd/mnms_sax.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms_sax.yaml)
|
| 67 |
-
| CVD classification | M&Ms2 | SAX | 2 (ED and ES) | [finetuned/classification_cvd/mnms2_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_sax_0.safetensors)<br>[finetuned/classification_cvd/mnms2_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_sax_1.safetensors)<br>[finetuned/classification_cvd/mnms2_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_sax_2.safetensors)<br>[finetuned/classification_cvd/mnms2_sax.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_sax.yaml)
|
| 68 |
-
| CVD classification | M&Ms2 | LAX 4C | 2 (ED and ES) | [finetuned/classification_cvd/mnms2_lax_4c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_lax_4c_0.safetensors)<br>[finetuned/classification_cvd/mnms2_lax_4c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_lax_4c_1.safetensors)<br>[finetuned/classification_cvd/mnms2_lax_4c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_lax_4c_2.safetensors)<br>[finetuned/classification_cvd/mnms2_lax_4c.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_lax_4c.yaml) | [classification_cvd.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/classification_cvd.py) |
|
| 69 |
-
| Patient sex classification | M&Ms | SAX | 2 (ED and ES) | [finetuned/classification_sex/mnms_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_sex/mnms_sax_0.safetensors)<br>[finetuned/classification_sex/mnms_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_sex/mnms_sax_1.safetensors)<br>[finetuned/classification_sex/mnms_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_sex/mnms_sax_2.safetensors)<br>[finetuned/classification_sex/mnms_sax.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_sex/mnms_sax.yaml)
|
| 70 |
-
| CMR machine vendor classification | M&Ms2 | SAX | 2 (ED and ES) | [finetuned/classification_vendor/mnms2_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_sax_0.safetensors)<br>[finetuned/classification_vendor/mnms2_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_sax_1.safetensors)<br>[finetuned/classification_vendor/mnms2_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_sax_2.safetensors)<br>[finetuned/classification_vendor/mnms2_sax.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_sax.yaml)
|
| 71 |
-
| CMR machine vendor classification | M&Ms2 | LAX 4C | 2 (ED and ES) | [finetuned/classification_vendor/mnms2_lax_4c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_lax_4c_0.safetensors)<br>[finetuned/classification_vendor/mnms2_lax_4c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_lax_4c_1.safetensors)<br>[finetuned/classification_vendor/mnms2_lax_4c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_lax_4c_2.safetensors)<br>[finetuned/classification_vendor/mnms2_lax_4c.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_lax_4c.yaml) | [classification_vendor.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/classification_vendor.py) |
|
| 72 |
-
| EF regression
|
| 73 |
-
| EF regression
|
| 74 |
-
| EF regression
|
| 75 |
-
| EF regression
|
| 76 |
-
| Patient BMI regression | ACDC | SAX | 2 (ED and ES) | [finetuned/regression_bmi/acdc_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_bmi/acdc_sax_0.safetensors)<br>[finetuned/regression_bmi/acdc_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_bmi/acdc_sax_1.safetensors)<br>[finetuned/regression_bmi/acdc_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_bmi/acdc_sax_2.safetensors)<br>[finetuned/regression_bmi/acdc_sax.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_bmi/acdc_sax.yaml)
|
| 77 |
-
| Patient age regression | M&Ms | SAX | 2 (ED and ES) | [finetuned/regression_age/mnms_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_age/mnms_sax_0.safetensors)<br>[finetuned/regression_age/mnms_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_age/mnms_sax_1.safetensors)<br>[finetuned/regression_age/mnms_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_age/mnms_sax_2.safetensors)<br>[finetuned/regression_age/mnms_sax.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_age/mnms_sax.yaml)
|
| 78 |
-
| Landmark localization by heatmap regression | Landmark | LAX 2C | 1 | [finetuned/landmark_heatmap/lax_2c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_2c_0.safetensors)<br>[finetuned/landmark_heatmap/lax_2c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_2c_1.safetensors)<br>[finetuned/landmark_heatmap/lax_2c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_2c_2.safetensors)<br>[finetuned/landmark_heatmap/lax_2c.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_2c.yaml)
|
| 79 |
-
| Landmark localization by heatmap regression | Landmark | LAX 4C | 1 | [finetuned/landmark_heatmap/lax_4c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_4c_0.safetensors)<br>[finetuned/landmark_heatmap/lax_4c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_4c_1.safetensors)<br>[finetuned/landmark_heatmap/lax_4c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_4c_2.safetensors)<br>[finetuned/landmark_heatmap/lax_4c.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_4c.yaml)
|
| 80 |
-
| Landmark localization by coordinates regression | Landmark | LAX 2C | 1 | [finetuned/landmark_coordinate/lax_2c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_2c_0.safetensors)<br>[finetuned/landmark_coordinate/lax_2c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_2c_1.safetensors)<br>[finetuned/landmark_coordinate/lax_2c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_2c_2.safetensors)<br>[finetuned/landmark_coordinate/lax_2c.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_2c.yaml)
|
| 81 |
-
| Landmark localization by coordinates regression | Landmark | LAX 4C | 1 | [finetuned/landmark_coordinate/lax_4c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_4c_0.safetensors)<br>[finetuned/landmark_coordinate/lax_4c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_4c_1.safetensors)<br>[finetuned/landmark_coordinate/lax_4c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_4c_2.safetensors)<br>[finetuned/landmark_coordinate/lax_4c.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_4c.yaml)
|
| 82 |
|
| 83 |
## Pre-trained CineMA Model
|
| 84 |
|
| 85 |
The pre-trained CineMA model backbone is available at
|
| 86 |
[pretrained/cinema.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/pretrained/cinema.safetensors) with
|
| 87 |
-
configuration [pretrained/
|
| 88 |
|
| 89 |
Following scripts demonstrated how to fine-tune this backbone using
|
| 90 |
[a preprocessed version of ACDC dataset](https://huggingface.co/datasets/mathpluscode/ACDC):
|
|
@@ -97,4 +77,4 @@ Following scripts demonstrated how to fine-tune this backbone using
|
|
| 97 |
|
| 98 |
## Contact
|
| 99 |
|
| 100 |
-
For questions or collaborations, please contact Yunguan Fu ([email protected]).
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
language:
|
| 4 |
+
- en
|
| 5 |
datasets:
|
| 6 |
+
- mathpluscode/ACDC
|
| 7 |
tags:
|
| 8 |
+
- medical
|
| 9 |
+
- cardiac
|
| 10 |
+
- MRI
|
| 11 |
+
- foundation model
|
| 12 |
+
- MAE
|
| 13 |
---
|
| 14 |
|
| 15 |
# CineMA - A Foundation Model for Cine Cardiac Magnetic Resonance Images π₯π«
|
| 16 |
|
| 17 |
+
**CineMA** is a foundation model for **Cine** cardiac magnetic resonance (CMR) imaging based on
|
| 18 |
+
**M**asked-**A**utoencoder. CineMA has been pre-trained on UK Biobank data and fine-tuned on multiple clinically
|
| 19 |
+
relevant tasks such as ventricle and myocaridum segmentation, ejection fraction (EF) regression, cardiovascular disease
|
| 20 |
+
(CVD) detection and classification, and mid-valve plane and apical landmark localization. The model has been evaluated
|
| 21 |
+
on multiple datasets, including [ACDC](https://www.creatis.insa-lyon.fr/Challenge/acdc/),
|
| 22 |
+
[M&Ms](https://www.ub.edu/mnms/), [M&Ms2](https://www.ub.edu/mnms-2/),
|
| 23 |
+
[Kaggle](https://www.kaggle.com/c/second-annual-data-science-bowl/data),
|
| 24 |
+
[Rescan](https://www.ahajournals.org/doi/full/10.1161/CIRCIMAGING.119.009214), and
|
| 25 |
+
[Landmark](https://pubs.rsna.org/doi/10.1148/ryai.2021200197), etc.
|
| 26 |
|
| 27 |
+
β‘οΈ **Manuscript:** Coming soon.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
β‘οΈ **Code:** [mathpluscode/CineMA](https://github.com/mathpluscode/CineMA)
|
| 30 |
|
| 31 |
## Fine-tuned CineMA Models
|
| 32 |
|
| 33 |
+
The filenames of fine-tuned model weights follow the convention of
|
| 34 |
+
`finetuned/<task>/<data>_<view>/<data>_<view>_<seed>.safetensors` where number 0, 1, and 2 correspond to the different
|
| 35 |
+
training seeds.
|
| 36 |
|
| 37 |
Check the "Inference Example" column to see example inference scripts using these trained models.
|
| 38 |
|
| 39 |
+
| Training Task | Training Data | Input View | Input Timeframes | Model Weights and Configurations | Inference Example |
|
| 40 |
+
| ----------------------------------------------- | ------------- | ---------- | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
|
| 41 |
+
| Ventricle and myocardium segmentation | ACDC | SAX | 1 | [finetuned/segmentation/acdc_sax/acdc_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/acdc_sax/acdc_sax_0.safetensors)<br>[finetuned/segmentation/acdc_sax/acdc_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/acdc_sax/acdc_sax_1.safetensors)<br>[finetuned/segmentation/acdc_sax/acdc_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/acdc_sax/acdc_sax_2.safetensors)<br>[finetuned/segmentation/acdc_sax/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/acdc_sax/config.yaml) | [segmentation_sax.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/segmentation_sax.py) |
|
| 42 |
+
| Ventricle and myocardium segmentation | M&Ms | SAX | 1 | [finetuned/segmentation/mnms_sax/mnms_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms_sax/mnms_sax_0.safetensors)<br>[finetuned/segmentation/mnms_sax/mnms_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms_sax/mnms_sax_1.safetensors)<br>[finetuned/segmentation/mnms_sax/mnms_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms_sax/mnms_sax_2.safetensors)<br>[finetuned/segmentation/mnms_sax/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms_sax/config.yaml) | [segmentation_sax.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/segmentation_sax.py) |
|
| 43 |
+
| Ventricle and myocardium segmentation | M&Ms2 | SAX | 1 | [finetuned/segmentation/mnms2_sax/mnms2_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms2_sax/mnms2_sax_0.safetensors)<br>[finetuned/segmentation/mnms2_sax/mnms2_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms2_sax/mnms2_sax_1.safetensors)<br>[finetuned/segmentation/mnms2_sax/mnms2_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms2_sax/mnms2_sax_2.safetensors)<br>[finetuned/segmentation/mnms2_sax/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms2_sax/config.yaml) | [segmentation_sax.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/segmentation_sax.py) |
|
| 44 |
+
| Ventricle and myocardium segmentation | M&Ms2 | LAX 4C | 1 | [finetuned/segmentation/mnms2_lax_4c/mnms2_lax_4c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms2_lax_4c/mnms2_lax_4c_0.safetensors)<br>[finetuned/segmentation/mnms2_lax_4c/mnms2_lax_4c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms2_lax_4c/mnms2_lax_4c_1.safetensors)<br>[finetuned/segmentation/mnms2_lax_4c/mnms2_lax_4c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms2_lax_4c/mnms2_lax_4c_2.safetensors)<br>[finetuned/segmentation/mnms2_lax_4c/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/segmentation/mnms2_lax_4c/config.yaml) | [segmentation_lax_4c.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/segmentation_lax_4c.py) |
|
| 45 |
+
| CVD classification | ACDC | SAX | 2 (ED and ES) | [finetuned/classification_cvd/acdc_sax/acdc_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/acdc_sax/acdc_sax_0.safetensors)<br>[finetuned/classification_cvd/acdc_sax/acdc_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/acdc_sax/acdc_sax_1.safetensors)<br>[finetuned/classification_cvd/acdc_sax/acdc_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/acdc_sax/acdc_sax_2.safetensors)<br>[finetuned/classification_cvd/acdc_sax/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/acdc_sax/config.yaml) | [classification_cvd.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/classification_cvd.py) |
|
| 46 |
+
| CVD classification | M&Ms | SAX | 2 (ED and ES) | [finetuned/classification_cvd/mnms_sax/mnms_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms_sax/mnms_sax_0.safetensors)<br>[finetuned/classification_cvd/mnms_sax/mnms_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms_sax/mnms_sax_1.safetensors)<br>[finetuned/classification_cvd/mnms_sax/mnms_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms_sax/mnms_sax_2.safetensors)<br>[finetuned/classification_cvd/mnms_sax/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms_sax/config.yaml) | [classification_cvd.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/classification_cvd.py) |
|
| 47 |
+
| CVD classification | M&Ms2 | SAX | 2 (ED and ES) | [finetuned/classification_cvd/mnms2_sax/mnms2_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_sax/mnms2_sax_0.safetensors)<br>[finetuned/classification_cvd/mnms2_sax/mnms2_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_sax/mnms2_sax_1.safetensors)<br>[finetuned/classification_cvd/mnms2_sax/mnms2_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_sax/mnms2_sax_2.safetensors)<br>[finetuned/classification_cvd/mnms2_sax/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_sax/config.yaml) | [classification_cvd.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/classification_cvd.py) |
|
| 48 |
+
| CVD classification | M&Ms2 | LAX 4C | 2 (ED and ES) | [finetuned/classification_cvd/mnms2_lax_4c/mnms2_lax_4c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_lax_4c/mnms2_lax_4c_0.safetensors)<br>[finetuned/classification_cvd/mnms2_lax_4c/mnms2_lax_4c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_lax_4c/mnms2_lax_4c_1.safetensors)<br>[finetuned/classification_cvd/mnms2_lax_4c/mnms2_lax_4c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_lax_4c/mnms2_lax_4c_2.safetensors)<br>[finetuned/classification_cvd/mnms2_lax_4c/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_cvd/mnms2_lax_4c/config.yaml) | [classification_cvd.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/classification_cvd.py) |
|
| 49 |
+
| Patient sex classification | M&Ms | SAX | 2 (ED and ES) | [finetuned/classification_sex/mnms_sax/mnms_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_sex/mnms_sax/mnms_sax_0.safetensors)<br>[finetuned/classification_sex/mnms_sax/mnms_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_sex/mnms_sax/mnms_sax_1.safetensors)<br>[finetuned/classification_sex/mnms_sax/mnms_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_sex/mnms_sax/mnms_sax_2.safetensors)<br>[finetuned/classification_sex/mnms_sax/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_sex/mnms_sax/config.yaml) | [classification_sex.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/classification_sex.py) |
|
| 50 |
+
| CMR machine vendor classification | M&Ms2 | SAX | 2 (ED and ES) | [finetuned/classification_vendor/mnms2_sax/mnms2_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_sax/mnms2_sax_0.safetensors)<br>[finetuned/classification_vendor/mnms2_sax/mnms2_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_sax/mnms2_sax_1.safetensors)<br>[finetuned/classification_vendor/mnms2_sax/mnms2_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_sax/mnms2_sax_2.safetensors)<br>[finetuned/classification_vendor/mnms2_sax/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_sax/config.yaml) | [classification_vendor.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/classification_vendor.py) |
|
| 51 |
+
| CMR machine vendor classification | M&Ms2 | LAX 4C | 2 (ED and ES) | [finetuned/classification_vendor/mnms2_lax_4c/mnms2_lax_4c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_lax_4c/mnms2_lax_4c_0.safetensors)<br>[finetuned/classification_vendor/mnms2_lax_4c/mnms2_lax_4c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_lax_4c/mnms2_lax_4c_1.safetensors)<br>[finetuned/classification_vendor/mnms2_lax_4c/mnms2_lax_4c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_lax_4c/mnms2_lax_4c_2.safetensors)<br>[finetuned/classification_vendor/mnms2_lax_4c/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/classification_vendor/mnms2_lax_4c/config.yaml) | [classification_vendor.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/classification_vendor.py) |
|
| 52 |
+
| Direct EF regression | ACDC | SAX | 2 (ED and ES) | [finetuned/regression_ef/acdc_sax/acdc_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/acdc_sax/acdc_sax_0.safetensors)<br>[finetuned/regression_ef/acdc_sax/acdc_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/acdc_sax/acdc_sax_1.safetensors)<br>[finetuned/regression_ef/acdc_sax/acdc_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/acdc_sax/acdc_sax_2.safetensors)<br>[finetuned/regression_ef/acdc_sax/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/acdc_sax/config.yaml) | [regression_ef.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/regression_ef.py) |
|
| 53 |
+
| Direct EF regression | M&Ms | SAX | 2 (ED and ES) | [finetuned/regression_ef/mnms_sax/mnms_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/mnms_sax/mnms_sax_0.safetensors)<br>[finetuned/regression_ef/mnms_sax/mnms_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/mnms_sax/mnms_sax_1.safetensors)<br>[finetuned/regression_ef/mnms_sax/mnms_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/mnms_sax/mnms_sax_2.safetensors)<br>[finetuned/regression_ef/mnms_sax/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/mnms_sax/config.yaml) | [regression_ef.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/regression_ef.py) |
|
| 54 |
+
| Direct EF regression | M&Ms2 | SAX | 2 (ED and ES) | [finetuned/regression_ef/mnms2_sax/mnms2_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/mnms2_sax/mnms2_sax_0.safetensors)<br>[finetuned/regression_ef/mnms2_sax/mnms2_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/mnms2_sax/mnms2_sax_1.safetensors)<br>[finetuned/regression_ef/mnms2_sax/mnms2_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/mnms2_sax/mnms2_sax_2.safetensors)<br>[finetuned/regression_ef/mnms2_sax/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/mnms2_sax/config.yaml) | [regression_ef.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/regression_ef.py) |
|
| 55 |
+
| Direct EF regression | M&Ms2 | LAX 4C | 2 (ED and ES) | [finetuned/regression_ef/mnms2_lax_4c/mnms2_lax_4c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/mnms2_lax_4c/mnms2_lax_4c_0.safetensors)<br>[finetuned/regression_ef/mnms2_lax_4c/mnms2_lax_4c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/mnms2_lax_4c/mnms2_lax_4c_1.safetensors)<br>[finetuned/regression_ef/mnms2_lax_4c/mnms2_lax_4c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/mnms2_lax_4c/mnms2_lax_4c_2.safetensors)<br>[finetuned/regression_ef/mnms2_lax_4c/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_ef/mnms2_lax_4c/config.yaml) | [regression_ef.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/regression_ef.py) |
|
| 56 |
+
| Patient BMI regression | ACDC | SAX | 2 (ED and ES) | [finetuned/regression_bmi/acdc_sax/acdc_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_bmi/acdc_sax/acdc_sax_0.safetensors)<br>[finetuned/regression_bmi/acdc_sax/acdc_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_bmi/acdc_sax/acdc_sax_1.safetensors)<br>[finetuned/regression_bmi/acdc_sax/acdc_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_bmi/acdc_sax/acdc_sax_2.safetensors)<br>[finetuned/regression_bmi/acdc_sax/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_bmi/acdc_sax/config.yaml) | [regression_bmi.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/regression_bmi.py) |
|
| 57 |
+
| Patient age regression | M&Ms | SAX | 2 (ED and ES) | [finetuned/regression_age/mnms_sax/mnms_sax_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_age/mnms_sax/mnms_sax_0.safetensors)<br>[finetuned/regression_age/mnms_sax/mnms_sax_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_age/mnms_sax/mnms_sax_1.safetensors)<br>[finetuned/regression_age/mnms_sax/mnms_sax_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_age/mnms_sax/mnms_sax_2.safetensors)<br>[finetuned/regression_age/mnms_sax/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/regression_age/mnms_sax/config.yaml) | [regression_age.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/regression_age.py) |
|
| 58 |
+
| Landmark localization by heatmap regression | Landmark | LAX 2C | 1 | [finetuned/landmark_heatmap/lax_2c/lax_2c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_2c/lax_2c_0.safetensors)<br>[finetuned/landmark_heatmap/lax_2c/lax_2c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_2c/lax_2c_1.safetensors)<br>[finetuned/landmark_heatmap/lax_2c/lax_2c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_2c/lax_2c_2.safetensors)<br>[finetuned/landmark_heatmap/lax_2c/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_2c/config.yaml) | [landmark_heatmap.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/landmark_heatmap.py) |
|
| 59 |
+
| Landmark localization by heatmap regression | Landmark | LAX 4C | 1 | [finetuned/landmark_heatmap/lax_4c/lax_4c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_4c/lax_4c_0.safetensors)<br>[finetuned/landmark_heatmap/lax_4c/lax_4c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_4c/lax_4c_1.safetensors)<br>[finetuned/landmark_heatmap/lax_4c/lax_4c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_4c/lax_4c_2.safetensors)<br>[finetuned/landmark_heatmap/lax_4c/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_heatmap/lax_4c/config.yaml) | [landmark_heatmap.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/landmark_heatmap.py) |
|
| 60 |
+
| Landmark localization by coordinates regression | Landmark | LAX 2C | 1 | [finetuned/landmark_coordinate/lax_2c/lax_2c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_2c/lax_2c_0.safetensors)<br>[finetuned/landmark_coordinate/lax_2c/lax_2c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_2c/lax_2c_1.safetensors)<br>[finetuned/landmark_coordinate/lax_2c/lax_2c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_2c/lax_2c_2.safetensors)<br>[finetuned/landmark_coordinate/lax_2c/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_2c/config.yaml) | [landmark_coordinate.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/landmark_coordinate.py) |
|
| 61 |
+
| Landmark localization by coordinates regression | Landmark | LAX 4C | 1 | [finetuned/landmark_coordinate/lax_4c/lax_4c_0.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_4c/lax_4c_0.safetensors)<br>[finetuned/landmark_coordinate/lax_4c/lax_4c_1.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_4c/lax_4c_1.safetensors)<br>[finetuned/landmark_coordinate/lax_4c/lax_4c_2.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_4c/lax_4c_2.safetensors)<br>[finetuned/landmark_coordinate/lax_4c/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/finetuned/landmark_coordinate/lax_4c/config.yaml) | [landmark_coordinate.py](https://github.com/mathpluscode/CineMA/blob/main/examples/inference/landmark_coordinate.py) |
|
| 62 |
|
| 63 |
## Pre-trained CineMA Model
|
| 64 |
|
| 65 |
The pre-trained CineMA model backbone is available at
|
| 66 |
[pretrained/cinema.safetensors](https://huggingface.co/mathpluscode/CineMA/blob/main/pretrained/cinema.safetensors) with
|
| 67 |
+
configuration [pretrained/config.yaml](https://huggingface.co/mathpluscode/CineMA/blob/main/pretrained/config.yaml).
|
| 68 |
|
| 69 |
Following scripts demonstrated how to fine-tune this backbone using
|
| 70 |
[a preprocessed version of ACDC dataset](https://huggingface.co/datasets/mathpluscode/ACDC):
|
|
|
|
| 77 |
|
| 78 |
## Contact
|
| 79 |
|
| 80 |
+
For questions or collaborations, please contact Yunguan Fu ([email protected]).
|
finetuned/classification_cvd/{acdc_sax_0.safetensors β acdc_sax/acdc_sax_0.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{acdc_sax_1.safetensors β acdc_sax/acdc_sax_1.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{acdc_sax_2.safetensors β acdc_sax/acdc_sax_2.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{acdc_sax.yaml β acdc_sax/config.yaml}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{mnms2_lax_4c.yaml β mnms2_lax_4c/config.yaml}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{mnms2_lax_4c_0.safetensors β mnms2_lax_4c/mnms2_lax_4c_0.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{mnms2_lax_4c_1.safetensors β mnms2_lax_4c/mnms2_lax_4c_1.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{mnms2_lax_4c_2.safetensors β mnms2_lax_4c/mnms2_lax_4c_2.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{mnms2_sax.yaml β mnms2_sax/config.yaml}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{mnms2_sax_0.safetensors β mnms2_sax/mnms2_sax_0.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{mnms2_sax_1.safetensors β mnms2_sax/mnms2_sax_1.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{mnms2_sax_2.safetensors β mnms2_sax/mnms2_sax_2.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{mnms_sax.yaml β mnms_sax/config.yaml}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{mnms_sax_0.safetensors β mnms_sax/mnms_sax_0.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{mnms_sax_1.safetensors β mnms_sax/mnms_sax_1.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_cvd/{mnms_sax_2.safetensors β mnms_sax/mnms_sax_2.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_sex/{mnms_sax.yaml β mnms_sax/config.yaml}
RENAMED
|
File without changes
|
finetuned/classification_sex/{mnms_sax_0.safetensors β mnms_sax/mnms_sax_0.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_sex/{mnms_sax_1.safetensors β mnms_sax/mnms_sax_1.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_sex/{mnms_sax_2.safetensors β mnms_sax/mnms_sax_2.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_vendor/{mnms2_lax_4c.yaml β mnms2_lax_4c/config.yaml}
RENAMED
|
File without changes
|
finetuned/classification_vendor/{mnms2_lax_4c_0.safetensors β mnms2_lax_4c/mnms2_lax_4c_0.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_vendor/{mnms2_lax_4c_1.safetensors β mnms2_lax_4c/mnms2_lax_4c_1.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_vendor/{mnms2_lax_4c_2.safetensors β mnms2_lax_4c/mnms2_lax_4c_2.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_vendor/{mnms2_sax.yaml β mnms2_sax/config.yaml}
RENAMED
|
File without changes
|
finetuned/classification_vendor/{mnms2_sax_0.safetensors β mnms2_sax/mnms2_sax_0.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_vendor/{mnms2_sax_1.safetensors β mnms2_sax/mnms2_sax_1.safetensors}
RENAMED
|
File without changes
|
finetuned/classification_vendor/{mnms2_sax_2.safetensors β mnms2_sax/mnms2_sax_2.safetensors}
RENAMED
|
File without changes
|
finetuned/landmark_coordinate/{lax_2c.yaml β lax_2c/config.yaml}
RENAMED
|
File without changes
|
finetuned/landmark_coordinate/{lax_2c_0.safetensors β lax_2c/lax_2c_0.safetensors}
RENAMED
|
File without changes
|
finetuned/landmark_coordinate/{lax_2c_1.safetensors β lax_2c/lax_2c_1.safetensors}
RENAMED
|
File without changes
|
finetuned/landmark_coordinate/{lax_2c_2.safetensors β lax_2c/lax_2c_2.safetensors}
RENAMED
|
File without changes
|
finetuned/landmark_coordinate/{lax_4c.yaml β lax_4c/config.yaml}
RENAMED
|
File without changes
|
finetuned/landmark_coordinate/{lax_4c_0.safetensors β lax_4c/lax_4c_0.safetensors}
RENAMED
|
File without changes
|
finetuned/landmark_coordinate/{lax_4c_1.safetensors β lax_4c/lax_4c_1.safetensors}
RENAMED
|
File without changes
|
finetuned/landmark_coordinate/{lax_4c_2.safetensors β lax_4c/lax_4c_2.safetensors}
RENAMED
|
File without changes
|
finetuned/landmark_heatmap/{lax_2c.yaml β lax_2c/config.yaml}
RENAMED
|
File without changes
|
finetuned/landmark_heatmap/{lax_2c_0.safetensors β lax_2c/lax_2c_0.safetensors}
RENAMED
|
File without changes
|
finetuned/landmark_heatmap/{lax_2c_1.safetensors β lax_2c/lax_2c_1.safetensors}
RENAMED
|
File without changes
|
finetuned/landmark_heatmap/{lax_2c_2.safetensors β lax_2c/lax_2c_2.safetensors}
RENAMED
|
File without changes
|
finetuned/landmark_heatmap/{lax_4c.yaml β lax_4c/config.yaml}
RENAMED
|
File without changes
|
finetuned/landmark_heatmap/{lax_4c_0.safetensors β lax_4c/lax_4c_0.safetensors}
RENAMED
|
File without changes
|
finetuned/landmark_heatmap/{lax_4c_1.safetensors β lax_4c/lax_4c_1.safetensors}
RENAMED
|
File without changes
|
finetuned/landmark_heatmap/{lax_4c_2.safetensors β lax_4c/lax_4c_2.safetensors}
RENAMED
|
File without changes
|
finetuned/regression_age/{mnms_sax.yaml β mnms_sax/config.yaml}
RENAMED
|
File without changes
|
finetuned/regression_age/{mnms_sax_0.safetensors β mnms_sax/mnms_sax_0.safetensors}
RENAMED
|
File without changes
|
finetuned/regression_age/{mnms_sax_1.safetensors β mnms_sax/mnms_sax_1.safetensors}
RENAMED
|
File without changes
|
finetuned/regression_age/{mnms_sax_2.safetensors β mnms_sax/mnms_sax_2.safetensors}
RENAMED
|
File without changes
|
finetuned/regression_bmi/{acdc_sax_0.safetensors β acdc_sax/acdc_sax_0.safetensors}
RENAMED
|
File without changes
|