|
--- |
|
license: mit |
|
datasets: |
|
- allenai/objaverse |
|
- allenai/objaverse-xl |
|
- qizekun/OrienText300K |
|
- tiange/Cap3D |
|
language: |
|
- en |
|
--- |
|
|
|
|
|
# PointSO |
|
|
|
Paper_link: https://arxiv.org/abs/2502.13143 |
|
<br> |
|
Project Page: https://qizekun.github.io/sofar/ |
|
<br> |
|
Github Code: https://github.com/qizekun/SoFar |
|
|
|
We introduce the concept of semantic orientation, representing the object orientation condition on open vocabulary language. |
|
The model is part of [SoFar](https://github.com/qizekun/SoFar), a 6-DoF system for spatial reasoning and robotic manipulation. |
|
The released weights is on [Huggingface PointSO](https://huggingface.co/qizekun/PointSO), and the code is in the [orientation](https://github.com/qizekun/SoFar/tree/main/orientation) folder. |
|
|
|
## Pretrain |
|
Download the PointMAE as initialization. |
|
```bash |
|
wget https://github.com/Pang-Yatian/Point-MAE/releases/download/main/pretrain.pth -P orientation/ |
|
``` |
|
Perpare the OrienText300K dataset following [DATASET.md](https://github.com/qizekun/SoFar/blob/main/datasets/DATASET.md). |
|
```bash |
|
cd orientation |
|
sh train_ddp.sh |
|
``` |
|
|
|
## Finetune |
|
Perpare the Open6DOR finetuning dataset following [DATASET.md](https://github.com/qizekun/SoFar/blob/main/datasets/DATASET.md). |
|
The dataset is generated from isaac sim with different assets from Open6DOR. |
|
Finetune PointSO will significantly improve the performance on Open6DOR rotation track & 6-DoF track. |
|
We recommend using this version of the model for the Open6DOR V2 evaluation. |
|
```bash |
|
cd orientation |
|
sh train_ddp_ft.sh |
|
``` |
