Skip to content

Refactor Input Transforms in RL #122

Description

@fabioseel

Currently, input transforms are implemented via hijacking the "input normalization" of samplefactory.
However, this is a weird hack and confusing.
There's two better ways of doing it:

Make the transforms part of the model

The transforms are torch modules, so they can easily be added to be part of the model.
Advantages

  • simple solution (actually similar to the current one, the input transform is part of the model, but a bit cleaner)
  • easy access to 'untransformed' input eg for denoising

Problems

  • logically, they are not part of the model
  • reusing a model seems to be slightly more difficult (either the model config but especially loading / storing the weights needs some extra handling in that case)

Use environment wrappers

One could also use an environment wrapper to implement the transforms, pretty much as in #84
Advantages:

  • logic-wise / conceptually the cleanest solution

Problems:

  • moves the computation to the 'environment' processing, hence cpu, which already seems to be the bottleneck
  • need to explore how easy it is to access the 'untransformed' input in that case

We should decide and implement this before starting to work on #87 .

Metadata

Metadata

Assignees

No one assigned

    Labels

    EnhancementImprove an existing feature

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions