You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi,
As discussed in the discord channel, it would be great to have the ability to use both state (privileged state information) and visual information (eg: rgb+segmentation).
Current Limitations
One can already use state + manually do env.render() to get rgb but it is currently not possible to get the segmentation data.
Use Case
The use case behind this is to train an expert policy using privileged state information so that we can then generate expert trajectories with rgb+segmentation data to obtain rgb segments of frames generated by an expert.
Current Workaround
The current workaround seems to be to :
Train a policy using state obs_mode.
Use the trained expert policy to collect and save trajectories using the RecordEpisode wrapper.
Feature Description
Hi,
As discussed in the discord channel, it would be great to have the ability to use both
state
(privileged state information) and visual information (eg:rgb+segmentation
).Current Limitations
One can already use
state
+ manually doenv.render()
to getrgb
but it is currently not possible to get the segmentation data.Use Case
The use case behind this is to train an expert policy using privileged state information so that we can then generate expert trajectories with rgb+segmentation data to obtain rgb segments of frames generated by an expert.
Current Workaround
The current workaround seems to be to :
state
obs_mode.state
observations torgb+segmentation
using Maniskill conversion tool.Ex:
The text was updated successfully, but these errors were encountered: