Add module.train() to move a module back to the autodiff backend
          #3975
        
          
      
  Add this suggestion to a batch that can be applied as a single commit.
  This suggestion is invalid because no changes were made to the code.
  Suggestions cannot be applied while the pull request is closed.
  Suggestions cannot be applied while viewing a subset of changes.
  Only one suggestion per line can be applied in a batch.
  Add this suggestion to a batch that can be applied as a single commit.
  Applying suggestions on deleted lines is not supported.
  You must change the existing code in this line in order to create a valid suggestion.
  Outdated suggestions cannot be applied.
  This suggestion has been applied or marked resolved.
  Suggestions cannot be applied from pending reviews.
  Suggestions cannot be applied on multi-line comments.
  Suggestions cannot be applied while the pull request is queued to merge.
  Suggestion cannot be applied right now. Please check back later.
  
    
  
    
Warning
Because
module.valid()is stateless, therequire_gradattribute of module params will not follow when subsequently callingmodule.train(). For default use cases, this is fine, but if params were manually changed (e.g., froze some parameters) this could lead to unintended user results if the same logic is not applied after moving the model back to autodiff.This should be documented, but we should also keep this PR pending for now for this reason.
Checklist
cargo run-checkscommand has been executed.Related Issues/PRs
From discord:
Changes
Added the inverse function to
module.valid()<>module.train(), which moves the module and all of its sub-modules to the autodiff backend.Testing
Single test for batchnorm module to illustrate usage