File size: 1,505 Bytes
55cc10c f0cf072 55cc10c 5f0f4f7 55cc10c 5f0f4f7 55cc10c 5f0f4f7 55cc10c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
---
license: mit
datasets:
- allenai/objaverse
- allenai/objaverse-xl
- qizekun/OrienText300K
- tiange/Cap3D
language:
- en
---
# PointSO
Paper_link: https://arxiv.org/abs/2502.13143
<br>
Project Page: https://qizekun.github.io/sofar/
<br>
Github Code: https://github.com/qizekun/SoFar
We introduce the concept of semantic orientation, representing the object orientation condition on open vocabulary language.
The model is part of [SoFar](https://github.com/qizekun/SoFar), a 6-DoF system for spatial reasoning and robotic manipulation.
The released weights is on [Huggingface PointSO](https://huggingface.co/qizekun/PointSO), and the code is in the [orientation](https://github.com/qizekun/SoFar/tree/main/orientation) folder.
## Pretrain
Download the PointMAE as initialization.
```bash
wget https://github.com/Pang-Yatian/Point-MAE/releases/download/main/pretrain.pth -P orientation/
```
Perpare the OrienText300K dataset following [DATASET.md](https://github.com/qizekun/SoFar/blob/main/datasets/DATASET.md).
```bash
cd orientation
sh train_ddp.sh
```
## Finetune
Perpare the Open6DOR finetuning dataset following [DATASET.md](https://github.com/qizekun/SoFar/blob/main/datasets/DATASET.md).
The dataset is generated from isaac sim with different assets from Open6DOR.
Finetune PointSO will significantly improve the performance on Open6DOR rotation track & 6-DoF track.
We recommend using this version of the model for the Open6DOR V2 evaluation.
```bash
cd orientation
sh train_ddp_ft.sh
``` |