[update] better readme for release

Author: yzqin

README.md

@@ -1,25 +1,86 @@
-## Install
+# DexPoint: Generalizable Point Cloud Reinforcement Learning for
+
+Sim-to-Real Dexterous Manipulation
+
+[[Project Page]](https://yzqin.github.io/dexpoint/) [[Paper]](https://arxiv.org/abs/2211.09423) [[Poster]](https://docs.google.com/presentation/d/1dDtAPQ49k1emhETRPAib5R0wCGdwlz5l/edit?usp=sharing&ouid=108317450590466198031&rtpof=true&sd=true)
+-----
+
+[DexPoint: Generalizable Point Cloud Reinforcement Learning for
+Sim-to-Real Dexterous Manipulation ](https://yzqin.github.io/dexpoint/)
+
+Yuzhe Qin*, Binghao Huang*, Zhao-Heng Yin, Hao Su, Xiaolong Wang, CoRL 2022.
+
+DexPoint is a novel system and algorithm for RL from point cloud. This repo contains the simulated environment and
+training code for DexPoint.
+
+![Teaser](docs/teaser.png)
+
+## Bibtex
+
+```
+@article{dexpoint,
+  title          = {DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation },
+  author         = {Qin, Yuzhe and Huang, Binghao and Yin, Zhao-Heng and Su, Hao and Wang, Xiaolong},
+  journal        = {Conference on Robot Learning (CoRL)},
+  year           = {2022},
+}
+```
+
+## Installation
 
 ```shell
-# Install SAPIEN dev version, example for 3.8, you can choose a different whl file for 3.7, 3.9, 3.10
-pip3 install sapien>=2.1.0
+git clone [email protected]:yzqin/dexpoint-release.git
+cd dexart-release
+conda create --name dexpoint python=3.8
+conda activate dexpoint
+pip install -e .
 ```
 
-Download data file for hand detector and scene
+Download data file for the scene
 from [Google Drive Link](https://drive.google.com/file/d/1Xe3jgcIUZm_8yaFUsHnO7WJWr8cV41fE/view?usp=sharing).
 Place the `day.ktx` at `assets/misc/ktx/day.ktx`.
 
 ```shell
+pip install gdown
 gdown https://drive.google.com/uc?id=1Xe3jgcIUZm_8yaFUsHnO7WJWr8cV41fE
 ```
 
 ## File Structure
 
-- `hand_teleop`: main entry for the environment, utils, and other staff needs for teleoperation and RL training.
+- `dexpoint`: main content for the environment, utils, and other staff needs for RL training.
 - `assets`: robot and object models, and other static files
-- `main`: entry files
+- `example`: entry files to learn how to use the DexPoint environment
+- `docker`: dockerfile that can create container to be used for headless training on server
+
+## Quick Start
+
+### Use DexPoint environment and extend it for your project
+
+Run and explore the comments in the file below provided to familiarize yourself with the basic architecture of the
+DexPoint environment. Check the printed messages to understand the observation, action, camera, and speed for these
+environments.
+
+- [state_only_env.py](example/example_use_state_only_env.py): minimal state only environment
+- [example_use_pc_env.py](example/example_use_pc_env.py): minimal point cloud environment
+- [example_use_imagination_env.py](example/example_use_imagination_env.py): point cloud environment with imagined point
+  proposed
+  in DexPoint
+- [example_use_multi_camera_visual_env.py](example/example_use_multi_camera_visual_env.py): environment with multiple
+  different visual modalities, including depth, rgb, segmentation. We provide it for your reference, although it is not
+  used in DexPoint
+
+The environment we used in the training of DexPoint paper can be found here
+in [example_dexpoint_grasping.py](example/example_dexpoint_grasping.py).
+
+### Example extension of DexPoint environment framework in other project
+
+[DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objects (CVPR 2023)](https://github.com/Kami-code/dexart-release):
+extend DexPoint to articulated object manipulation.
+
+[From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperation (RA-L 2022)](https://yzqin.github.io/dex-teleop-imitation/):
+use teleoperation for data collection in DexPoint environment.
+
+
+
 
-## How to use
 
-Play with the `runnable` files inside the `main` directory. It provides an example of how to use PointCloud and Imaged
-Point Cloud in your environment.

docker/Dockerfile

@@ -24,5 +24,5 @@ RUN apt-get update -q \
 # Install python package
 ENV LANG C.UTF-8
 RUN pip3 install gym open3d scipy opencv-python numpy nlopt scipy transforms3d imageio nvitop setuptools opencv-contrib-python tensorboard moviepy h5py --upgrade
-RUN pip3 install https://storage1.ucsd.edu/wheels/sapien-dev/sapien-2.0.0.dev20220531-cp38-cp38-manylinux2014_x86_64.whl
+RUN pip3 install sapien==2.1.0
 RUN pip3 install torch torchvision --extra-index-url https://download.pytorch.org/whl/cu113

docs/teaser.png

@@ -0,0 +1,57 @@
+import os
+from time import time
+
+import numpy as np
+
+from dexpoint.env.rl_env.relocate_env import AllegroRelocateRLEnv
+from dexpoint.real_world import task_setting
+
+if __name__ == '__main__':
+    def create_env_fn():
+        object_names = ["mustard_bottle", "tomato_soup_can", "potted_meat_can"]
+        object_name = np.random.choice(object_names)
+        rotation_reward_weight = 0  # whether to match the orientation of the goal pose
+        use_visual_obs = True
+        env_params = dict(object_name=object_name, rotation_reward_weight=rotation_reward_weight,
+                          randomness_scale=1, use_visual_obs=use_visual_obs, use_gui=False,
+                          no_rgb=True)
+
+        # If a computing device is provided, designate the rendering device.
+        # On a multi-GPU machine, this sets the rendering GPU and RL training GPU to be the same,
+        # based on "CUDA_VISIBLE_DEVICES".
+        if "CUDA_VISIBLE_DEVICES" in os.environ:
+            env_params["device"] = "cuda"
+        environment = AllegroRelocateRLEnv(**env_params)
+
+        # Create camera
+        environment.setup_camera_from_config(task_setting.CAMERA_CONFIG["relocate"])
+
+        # Specify observation
+        environment.setup_visual_obs_config(task_setting.OBS_CONFIG["relocate_noise"])
+
+        # Specify imagination
+        environment.setup_imagination_config(task_setting.IMG_CONFIG["relocate_robot_only"])
+        return environment
+
+
+    env = create_env_fn()
+    print("Observation space:")
+    print(env.observation_space)
+    print("Action space:")
+    print(env.action_space)
+
+    obs = env.reset()
+
+    tic = time()
+    rl_steps = 1000
+    for _ in range(rl_steps):
+        action = np.zeros(env.action_space.shape)
+        action[0] = 0.002  # Moving forward ee link in x-axis
+        obs, reward, done, info = env.step(action)
+    elapsed_time = time() - tic
+
+    simulation_steps = rl_steps * env.frame_skip
+    print(f"Single process for point-cloud environment with {rl_steps} RL steps "
+          f"(= {simulation_steps} simulation steps) takes {elapsed_time}s.")
+    print("Keep in mind that using multiple processes during RL training can significantly increase the speed.")
+    env.scene = None

example/example_dexpoint_grasping.py

@@ -0,0 +1,84 @@
+import os
+from time import time
+
+import numpy as np
+import open3d as o3d
+
+from dexpoint.env.rl_env.relocate_env import AllegroRelocateRLEnv
+from dexpoint.real_world import task_setting
+
+if __name__ == '__main__':
+    def create_env_fn():
+        object_names = ["mustard_bottle", "tomato_soup_can", "potted_meat_can"]
+        object_name = np.random.choice(object_names)
+        rotation_reward_weight = 0  # whether to match the orientation of the goal pose
+        use_visual_obs = True
+        env_params = dict(object_name=object_name, rotation_reward_weight=rotation_reward_weight,
+                          randomness_scale=1, use_visual_obs=use_visual_obs, use_gui=False,
+                          no_rgb=True)
+
+        # If a computing device is provided, designate the rendering device.
+        # On a multi-GPU machine, this sets the rendering GPU and RL training GPU to be the same,
+        # based on "CUDA_VISIBLE_DEVICES".
+        if "CUDA_VISIBLE_DEVICES" in os.environ:
+            env_params["device"] = "cuda"
+        environment = AllegroRelocateRLEnv(**env_params)
+
+        # Create camera
+        environment.setup_camera_from_config(task_setting.CAMERA_CONFIG["relocate"])
+
+        # Specify observation
+        environment.setup_visual_obs_config(task_setting.OBS_CONFIG["relocate_noise"])
+
+        # Specify imagination
+        environment.setup_imagination_config(task_setting.IMG_CONFIG["relocate_goal_robot"])
+        return environment
+
+
+    env = create_env_fn()
+    print("Observation space:")
+    print(env.observation_space)
+    print("Action space:")
+    print(env.action_space)
+
+    obs = env.reset()
+    print("For state task, observation is a numpy array. For visual tasks, observation is a python dict.")
+
+    print("Observation keys")
+    print(obs.keys())
+
+    tic = time()
+    rl_steps = 1000
+    for _ in range(rl_steps):
+        action = np.zeros(env.action_space.shape)
+        action[0] = 0.002  # Moving forward ee link in x-axis
+        obs, reward, done, info = env.step(action)
+    elapsed_time = time() - tic
+
+    pc = obs["relocate-point_cloud"]
+    # The name of the key in observation is "CAMERA_NAME"-"MODALITY_NAME".
+    # While CAMERA_NAME is defined in task_setting.CAMERA_CONFIG["relocate"], name is point_cloud.
+    # See example_use_multi_camera_visual_env.py for more modalities.
+
+    simulation_steps = rl_steps * env.frame_skip
+    print(f"Single process for point-cloud environment with {rl_steps} RL steps "
+          f"(= {simulation_steps} simulation steps) takes {elapsed_time}s.")
+    print("Keep in mind that using multiple processes during RL training can significantly increase the speed.")
+    env.scene = None
+
+    # Note that in the DexPoint paper, we never use "imagination_goal" but only "imagination_robot"
+    goal_pc = obs["imagination_goal"]
+    goal_robot = obs["imagination_robot"]
+    imagination_goal_cloud = o3d.geometry.PointCloud(
+        points=o3d.utility.Vector3dVector(goal_pc))
+    imagination_goal_cloud.paint_uniform_color(np.array([0, 1, 0]))
+    imagination_robot_cloud = o3d.geometry.PointCloud(
+        points=o3d.utility.Vector3dVector(goal_robot))
+    imagination_robot_cloud.paint_uniform_color(np.array([0, 0, 1]))
+
+    obs_cloud = o3d.geometry.PointCloud(points=o3d.utility.Vector3dVector(pc))
+    obs_cloud.paint_uniform_color(np.array([1, 0, 0]))
+    coordinate = o3d.geometry.TriangleMesh.create_coordinate_frame(size=0.05, origin=[0, 0, 0])
+    o3d.visualization.draw_geometries([imagination_goal_cloud, imagination_robot_cloud, coordinate, obs_cloud])
+
+    env.scene = None

example/example_use_imagination_env.py

@@ -0,0 +1,69 @@
+import os
+
+import imageio
+import numpy as np
+from PIL import ImageColor
+
+from dexpoint.env.rl_env.relocate_env import AllegroRelocateRLEnv
+
+if __name__ == '__main__':
+    def create_env_fn():
+        object_names = ["mustard_bottle", "tomato_soup_can", "potted_meat_can"]
+        object_name = np.random.choice(object_names)
+        rotation_reward_weight = 0
+        use_visual_obs = True
+        env_params = dict(object_name=object_name, rotation_reward_weight=rotation_reward_weight,
+                          randomness_scale=1, use_visual_obs=use_visual_obs, use_gui=False, no_rgb=False)
+
+        # If a computing device is provided, designate the rendering device.
+        # On a multi-GPU machine, this sets the rendering GPU and RL training GPU to be the same,
+        # based on "CUDA_VISIBLE_DEVICES".
+        if "CUDA_VISIBLE_DEVICES" in os.environ:
+            env_params["device"] = "cuda"
+        environment = AllegroRelocateRLEnv(**env_params)
+
+        # Create camera
+        camera_cfg = {
+            "cam1": dict(position=np.array([-0.4, 0.4, 0.6]), look_at_dir=np.array([0.4, -0.4, -0.6]),
+                         right_dir=np.array([-1, -1, 0]), fov=np.deg2rad(69.4), resolution=(256, 256)),
+            "cam2": dict(position=np.array([-0.6, -0.3, 0.8]), look_at_dir=np.array([0.6, 0.3, -0.8]),
+                         right_dir=np.array([1, -2, 0]), fov=np.deg2rad(69.4), resolution=(256, 256))
+        }
+        environment.setup_camera_from_config(camera_cfg)
+
+        # Specify observation modality
+        empty_info = {}  # level empty dict for default observation setting
+        obs_cfg = {"cam1": {"rgb": empty_info, "segmentation": empty_info},
+                   "cam2": {"depth": empty_info}}
+        environment.setup_visual_obs_config(obs_cfg)
+        return environment
+
+
+    env = create_env_fn()
+    print("Observation space:")
+    print(env.observation_space)
+    print("Action space:")
+    print(env.action_space)
+
+    obs = env.reset()
+    print("For state task, observation is a numpy array. For visual tasks, observation is a python dict.")
+
+    print("Observation keys")
+    print(obs.keys())
+    rgb = obs["cam1-rgb"]
+    rgb_pic = (rgb * 255).astype(np.uint8)
+    imageio.imsave("cam1-rgb.png", rgb_pic)
+
+    # Segmentation
+    link_seg = obs["cam1-segmentation"][..., 0]
+    part_seg = obs["cam1-segmentation"][..., 1]
+    colormap = sorted(set(ImageColor.colormap.values()))
+    color_palette = np.array([ImageColor.getrgb(color) for color in colormap], dtype=np.uint8)
+    imageio.imsave("cam1-link_seg.png", color_palette[link_seg].astype(np.uint8))
+    imageio.imsave("cam1-part_seg.png", color_palette[part_seg].astype(np.uint8))
+
+    # Depth normalization
+    depth = obs["cam2-depth"] / 10 * 65535
+    imageio.imwrite("cam2-depth.png", depth[..., 0].astype(np.uint16))
+
+    env.scene = None

example/example_use_multi_camera_visual_env.py

@@ -43,20 +43,20 @@ def create_env_fn():
 
     print("Observation keys")
     print(obs.keys())
-    # The name of the key in observation is "CAMERA_NAME"-"MODALITY_NAME".
-    # While CAMERA_NAME is defined in task_setting.CAMERA_CONFIG["relocate"], name is point_cloud.
-    # See example_use_multi_camera_visual_env.py for more modalities.
 
     tic = time()
     rl_steps = 1000
     for _ in range(rl_steps):
         action = np.zeros(env.action_space.shape)
         action[0] = 0.002  # Moving forward ee link in x-axis
         obs, reward, done, info = env.step(action)
+    elapsed_time = time() - tic
 
     pc = obs["relocate-point_cloud"]
+    # The name of the key in observation is "CAMERA_NAME"-"MODALITY_NAME".
+    # While CAMERA_NAME is defined in task_setting.CAMERA_CONFIG["relocate"], name is point_cloud.
+    # See example_use_multi_camera_visual_env.py for more modalities.
 
-    elapsed_time = time() - tic
     simulation_steps = rl_steps * env.frame_skip
     print(f"Single process for point-cloud environment with {rl_steps} RL steps "
           f"(= {simulation_steps} simulation steps) takes {elapsed_time}s.")