Update README.md

README.md

@@ -7,6 +7,11 @@ language:
 base_model:
 - allenai/Molmo-7B-D-0924
 pipeline_tag: robotics
+tags:
+- robotics
+- grasping
+- task-oriented-grasping
+- manipulation
 ---
 
 # GraspMolmo
@@ -42,4 +47,32 @@ Running the above code could result in the following output:
 In order to accomplish the task "Pour coffee from the blue mug.", the optimal grasp is described as follows: "The grasp is on the middle handle of the blue mug, with fingers grasping the sides of the handle.".
 
 <point x="28.6" y="20.7" alt="Where to grasp the object">Where to grasp the object</point>
+```
+
+## Grasp Inference
+
+To predict a grasp point *and* match it to one of the candidate grasps, refer to the [GraspMolmo](https://github.com/abhaybd/GraspMolmo/blob/main/graspmolmo/inference/grasp_predictor.py) class.
+First, install `graspmolmo` with
+
+```bash
+pip install "git+https://github.com/abhaybd/GraspMolmo.git#egg=graspmolmo[infer]"
+```
+
+and then inference can be run as follows:
+
+```python
+from graspmolmo.inference.grasp_predictor import GraspMolmo
+
+task = "..."
+rgb, depth = get_image()
+camera_intrinsics = np.array(...)
+
+point_cloud = backproject(rgb, depth, camera_intrinsics)
+# grasps are in the camera reference frame
+grasps = predict_grasps(point_cloud)  # Using your favorite grasp predictor (e.g. M2T2)
+
+gm = GraspMolmo()
+idx = gm.pred_grasp(rgb, point_cloud, task, grasps)
+
+print(f"Predicted grasp: {grasps[idx]}")
 ```