Add StateVectorActionGenerator and make OptunaSensorManager compatible#1251
Add StateVectorActionGenerator and make OptunaSensorManager compatible#1251nperree-dstl wants to merge 10 commits intomainfrom
Conversation
…accurately represent the action space
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1251 +/- ##
==========================================
+ Coverage 94.58% 94.60% +0.01%
==========================================
Files 233 233
Lines 16743 16761 +18
Branches 2356 2363 +7
==========================================
+ Hits 15837 15856 +19
+ Misses 607 604 -3
- Partials 299 301 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| max_index_change = float(np.sqrt(generator.max_state_change**2 - | ||
| sum(value**2))) | ||
| max_index_change, resolution = self.sample_parameters( | ||
| max_index_change, | ||
| generator) |
There was a problem hiding this comment.
I wonder if max change could be something that is part of the action generator, as different StateVectorActionGenerators may have different ways to define constraints. Possibly min and max?
There was a problem hiding this comment.
I think having a min / max StateVector with the same length as action mapping that defines the min / max of each dimension would be useful. This specific constraint seems to be an inequality rather than a min / max, i.e. bounding a 2d position by a circle, rather than a square (which min / max would do).
To do this generally, we would need to add lower (or upper) inequalities to generators. This could be a list of callables that act on this generator's action space?
e.g for CircleSamplePositionActionGenerator and MaxSpeedPositionActionGenerator we would have a less than or equal inequality looking something like:
lambda x: np.sum( [ (x[index] - self.current_value[index]) ** 2 for index in self.action_mapping ] ) - self.maximum_travel**2
These could then be integrated into constraints for Optuna objectives / other optimisation packages more easily without having to have custom logic per generator in each sensor manager.
|
|
||
| @staticmethod | ||
| def sample_parameters(max_value, generator): | ||
| resolution = getattr(generator, "resolution", None) |
There was a problem hiding this comment.
The resolution for the NStepDirectionalGridActionGenerator is resolution * step_size. So in most cases a platform with that generator will be given an action value it doesn't contain and it will fail silently.
Introduces
StateVectorActionGeneratoras a base class for generating actions which change the state (e.g. platform location), andMovePositionActionGeneratorfor generatingMovePositionActions.Updates
OptunaSensorManagerso it is compatible withStateVectorActionGenerators.Addresses #1190.