[Discussion] Default to noisy sensor measurements

[Danfoa]

For our machine learning and control experiments, we should be using noisy observations for all relevant observables. Currently, we use ground truth data from the simulator, which biases our learning and control algorithms to operate only on perfect, unbiased, and noiseless observations.

I would like to transition the entire quadruped_env API so that all observations returned via the step function include parametric control over the variance and bias of the sensor noise. It seems that MuJoCo leaves this functionality to users, which is the approach I suggest we adopt.

This would mean that during environment initialization, we would pass optional parameters to model the noise acting on the sensor observations (e.g., joint position, joint velocity, IMU, contact forces, joint torques, joint accelerations, etc.). The idea is to return these noisy observations by default and optionally return the ground truth observations. Something like this:

state, reward, is_terminated, is_truncated, info = env.step(action=action)
joint_pos_noisy = state['joint_pos']      # Sensor measurement with noise, bias, etc.
joint_pos_gt = state['joint_pos_gt']      # Ground truth sensor measurement

[giulioturrisi]

Hello, no problem with that. The only drawback that i see is that you don't have full control of the noise in this way, or become very convolute to add it (e.g. different noise param for different state). Instead, if you take care from yourself, it become more easy maybe? The IMU for example has directly noise in the sensor construction as far as i remember.

Is there a way to make it general? Maybe using directly the sensor provided by mujoco in the xml?

[Danfoa]

Well the most basic noise model we can use is a Brownian Motion (white noise) with the only parameter being the noise std.

We could provide this as a default noise model for the relevant observations:

• qpos: in units of meters / radians
• qvel: in units of meters/radians per second.
• qacc:
• joint_torques
• ...

Then observables computed from these noise observations would use the sensor data not the perfect observations. Hence we would have that the mass matrix, jacobians, kinetic energy, work, ... will all be non-linearly propagating the white noise in their respective data spaces.

I would propose we add a initialization parameter sensor_noise_cfg, which is a dict passing configurations for noise per sensor observanble. Populated by default by some reasonable white noise perturbations. And if the user pass None to this parameter we simply use the perfect sensor observations ?

As far as I know, mujoco is not adding any noise to sensor observations, it only has the capability to store the sensor noise model parameters within the mjData structure. Similarly we should aim to ensure reproducibility of dataset environments initialized with different noise models.

Also for IMU we would need to use another sensor model that also adds bias and variance.

[giulioturrisi]

Ok for me. The current implementation of the IMU can specify bias and variance.