Pytorch Lighting Support

[agemagician]

What would you like to be added:
To add support and integration for PyTorch lighting

Why is this needed:
Accelerate research and results.

Without this feature, how does current nni work：
Using PyTorch code, which takes a lot of time.

Components that may involve changes:
I am not 100% sure.

[J-shang]

Hi @agemagician, could you give more information about where you want to use Pytorch Lighting? HPO, NAS, or Compression?

[agemagician]

Hi @J-shang ,

Thanks a lot for your reply.

I believe it will be beneficial for all use-cases, but for me, I am very interested in "Model Compression".

Two good use-cases if you could provide a PyTorch integration for NNI, and test it in:

1. Nvidia Nemo to compress the audio.
2. HuggingFace Transformers to compress NLP models.

Thanks a lot.

[xiaowu0162]

Hi @agemagician,

For compressing HuggingFace Transformers, we currently have a tool under development for v2.4. If you are interested in it, please check the examples in this pull request.

Thank you.

[agemagician]

Great, thanks.

is it possible to provide an example for Nemo + PyTorch lighting?
Or maybe a general example for using Pytorch lighting with NNI for model compression,

[mashu]

I am trying with Lightning module, but than some other non-NNI logs are produced and it's not starting a typical experiment that I could view with webgui. This happens with DARTs strategy.

[J-shang]

I am trying with Lightning module, but than some other non-NNI logs are produced and it's not starting a typical experiment that I could view with webgui. This happens with DARTs strategy.

@mashu , could you provide your log messages? you could open a new issue to descript what you have done and upload the log, seems that you are trying with NAS.