Question about proprio information in training data

[Jianyi2004]

Question about proprio information in training data

Question

I have two questions about the training data:

1. Does the training data include relative distance information between the robot arm and objects?

2. Could you clarify what information is contained in the proprio array? From the code, I can see it's constructed from state_obs_keys in OXE_DATASET_CONFIGS, but I'm not sure about its specific contents.

Context

I'm looking at the data loading code, particularly:

• prismatic/vla/datasets/rlds/dataset.py: where proprio is constructed from state_obs_keys
• prismatic/vla/datasets/rlds/oxe/materialize.py: where dataset configs are loaded

The code shows that proprio is loaded when load_proprio=True, but I couldn't find documentation about what specific robot state information it contains.

Additional Information

• I've enabled load_proprio=True in my configuration
• I've modified the data collator to include robot_states in the batch
• I'm trying to understand what state information is available during training

Would appreciate any clarification on this. Thanks!