Update README.md
Browse files
README.md
CHANGED
@@ -6,6 +6,55 @@ tags:
|
|
6 |
- pytorch_model_hub_mixin
|
7 |
---
|
8 |
|
9 |
-
|
10 |
-
|
11 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
- pytorch_model_hub_mixin
|
7 |
---
|
8 |
|
9 |
+
|
10 |
+
# St4RTrack: Simultaneous 4D Reconstruction and Tracking in the World
|
11 |
+
|
12 |
+
```bibtex
|
13 |
+
@inproceedings{st4rtrack2025,
|
14 |
+
title={St4RTrack: Simultaneous 4D Reconstruction and Tracking in the World},
|
15 |
+
author={Feng*, Haiwen and Zhang*, Junyi and Wang, Qianqian and Ye, Yufei and Yu, Pengcheng and Black, Michael J. and Darrell, Trevor and Kanazawa, Angjoo},
|
16 |
+
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
|
17 |
+
year={2025}
|
18 |
+
}
|
19 |
+
```
|
20 |
+
|
21 |
+
## Model info
|
22 |
+
|
23 |
+
Github page: https://github.com/HavenFeng/St4rTrack-Release
|
24 |
+
Project page: https://st4rtrack.github.io/
|
25 |
+
Paper: https://arxiv.org/abs/2504.13152
|
26 |
+
|
27 |
+
|
28 |
+
## How to Use
|
29 |
+
|
30 |
+
First install [St4rTrack](https://github.com/HavenFeng/St4rTrack-Release). To load the model (Seq):
|
31 |
+
|
32 |
+
```python
|
33 |
+
from dust3r.models import AsymmetricCroCo3DStereo
|
34 |
+
|
35 |
+
# Loads default Seq checkpoint
|
36 |
+
model = AsymmetricCroCo3DStereo.from_pretrained("yupengchengg147/St4rTrack")
|
37 |
+
```
|
38 |
+
|
39 |
+
run following code to load checkpoint trained with Pair Mode:
|
40 |
+
|
41 |
+
```python
|
42 |
+
from huggingface_hub import hf_hub_download
|
43 |
+
import tempfile
|
44 |
+
import os
|
45 |
+
from dust3r.model import AsymmetricCroCo3DStereo
|
46 |
+
|
47 |
+
# Create a temporary directory for the model files
|
48 |
+
temp_dir = os.path.join(tempfile.gettempdir(), "st4rtrack_pair")
|
49 |
+
os.makedirs(temp_dir, exist_ok=True)
|
50 |
+
|
51 |
+
# Download the config and model files from the Pair subfolder
|
52 |
+
config_path = hf_hub_download(
|
53 |
+
repo_id="yupengchengg147/St4rTrack",
|
54 |
+
filename="Pair/config.json",
|
55 |
+
cache_dir=temp_dir
|
56 |
+
)
|
57 |
+
# Load the model from the downloaded path
|
58 |
+
model_dir = os.path.dirname(config_path)
|
59 |
+
model = AsymmetricCroCo3DStereo.from_pretrained(model_dir)
|
60 |
+
```
|