Hyper parameter schedules

[tilmankamp]

Support for hyper parameter schedules during training. They could get specified as comma-separated value sequences:

python DeepSpeech.py ... --dropout_rate 0.40,10:0.30,50:0.20 ...

This example also shows how colon-separated prefixes could be used to specify the epochs at which the prefixed values should get reached.

[reuben]

Someone said they were interested in contributing a LR scheduler a while ago, I don't remember who it was though :/

[reuben]

Also, this would be a good opportunity to refactor the training code a bit and make it more reusable so that logic like this can be implemented as an orchestration of existing functions rather than hardcoded into the training loop. For example if individual training steps and epochs are simple enough to set up and call from an external, simpler training script, the script can contain all of the schedule parsing logic, instead of it being spread throughout the code.

[nilabja-bhattacharya]

Hi, I would like to work on this. I am a new contributor, please let me know the files to get started.