Update README.md

README.md

@@ -1,3 +1,38 @@
----
-license: mit
----
+---
+license: mit
+datasets:
+- allenai/objaverse
+- allenai/objaverse-xl
+- qizekun/OrienText300K
+- tiange/Cap3D
+language:
+- en
+---
+
+
+# PointSO
+
+We introduce the concept of semantic orientation, representing the object orientation condition on open vocabulary language.
+The model is part of [SoFar](https://github.com/qizekun/SoFar), a 6-DoF system for spatial reasoning and robotic manipulation.
+The released weights is on [Huggingface PointSO](https://huggingface.co/qizekun/PointSO), and the code is in the [orientation](./orientation) folder.
+
+## Pretrain
+Download the PointMAE as initialization.
+```bash
+wget https://github.com/Pang-Yatian/Point-MAE/releases/download/main/pretrain.pth -P orientation/
+```
+Perpare the OrienText300K dataset following [DATASET.md](./datasets/DATASET.md).
+```bash
+cd orientation
+sh train_ddp.sh
+```
+
+## Finetune
+Perpare the Open6DOR finetuning dataset following [DATASET.md](./datasets/DATASET.md).
+The dataset is generated from isaac sim with different assets from Open6DOR.
+Finetune PointSO will significantly improve the performance on Open6DOR rotation track & 6-DoF track. 
+We recommend using this version of the model for the Open6DOR V2 evaluation.
+```bash
+cd orientation
+sh train_ddp_ft.sh
+```