target out of sight data

[FeiqiangLin]

Hi,
I'was wondering if you use any data augmentation technique to train the model in order to obtain searching behaviour when target is out of sight(looking in the direction where the target disappeared)? When i view the data collected from the dataset, it seems that the samples where target is unseen are not very common. Or the samples from the training split of the dataset are enough to acquire such capability?
Also, regarding the waypoint labels: do you use the trajectory that the robot actually followed during an episode (e.g., for a specific sample at a given time, the label consists of the poses from t1 to t8 along the actual trajectory the robot took), or do you use the A planned trajectory at each time step?

[wsakobe]

Hi,

1. Since we built our dataset using indoor home scenes from HM3D and MP3D, such scenes have a lot of occlusions (corridors, doorways, etc.), resulting in numerous cases where targets are occluded.
2. Because tracking is a dynamic task, we use A* as the planner for each step to plan the robot's trajectory at the current position, rather than simply aggregating the actions of the episode to generate the trajectory.

[FeiqiangLin]

Thankss for your reply!

1. Ok, I will check the samples collected from the dataset.
2. The waypoint plan calculated by the A* algorithm will be used as lables, or the acutal actions executed by the robot will be used as labels? I believe these two are diffenrent. Lables from A* plan will link to current target position, while lables from robot traj will somehow point to the future target position.

[wsakobe]

It's the former. We use the target position corresponding to each step for planning to obtain a trajectory consisting of 10 waypoints as the label, but when collecting data, we only execute the action corresponding to the first waypoint.

[FeiqiangLin]

It's the former. We use the target position corresponding to each step for planning to obtain a trajectory consisting of 10 waypoints as the label, but when collecting data, we only execute the action corresponding to the first waypoint.

Thanks for your quick reply, does the A* planning will also produce yaw plans?

[wsakobe]

A* cannot plan the yaw angle; we interpolate the corresponding yaw angle through the 2D waypoints it plans.

[FeiqiangLin]

Thanks!

[FeiqiangLin]

It's the former. We use the target position corresponding to each step for planning to obtain a trajectory consisting of 10 waypoints as the label, but when collecting data, we only execute the action corresponding to the first waypoint.

Hi, when i use self._sim.pathfinder.find_path(path) to plan a trajectory towards the target at each time step, most of the result paths only has 2-4 waypoints. how can i get 10 waypoints out of the planning trajectory?

[wsakobe]

Yes, you're right. The waypoints obtained by the ShortestPath() function are very sparse. We first used B-spline to interpolate the discrete waypoints obtained by the ShortestPath() function, then calculated the appropriate robot speed based on the target distance through a preset speed function, and finally selected the set of points that meet the speed requirements on the interpolated spline curve to form the trajectory.