Input specific location for an object for a trained policy

[ZachAtkins0n]

I was wondering what would be the best way to about having a input for a block location that is read from a camera sensor for a manipulator? Would it be possible to use a trained robot model that was trained to move towards a block that was randomly spawned originally and then after training use a camera sensor to tell the robot where the block location is instead?

[RandomOakForest]

Thanks for posting this discussion. Have you considered training an object detection task using camera images and using the object detector as input to your manipulator? Isaac Sim has a list of examples that may give you good ideas about how to do this here.