Increase training speed by testing different techniques

[tfha]

1. Profiling code for time with profiling techniques to find the parts to optimize (cProfiler etc.): https://machinelearningmastery.com/profiling-python-code/
2. MKL:

• https://pypi.org/project/mkl/
• https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl.html#gs.7t60es
• https://stackoverflow.com/questions/69986869/how-to-enable-and-disable-intel-mkl-in-numpy-python

3. Running on NGI Odin Machine
4. Running on Azure cloud (if its available for use in our NGI Azure cloud) or AWS/Google cloud etc.
5. Sharing database and optuna optimize from many machines against the same database-files
6. Make it possible to automatically kick off the number of processes as the number of cpus on a machine, ie. parallelization:

• https://wiki.python.org/moin/ParallelProcessing
• https://mpi4py.readthedocs.io/en/stable/

[tfha]

Number 3, 5 and 6 are now tested and ok.

[tfha]

Point 2 is already implemented in existing libraries

[tfha]

A report from profiling the train_agent method. This highlights a number of processes to handle.

Some thoughts of signifcant processes that takes time:

• the implement_action method takes considerable time, 21 seconds to run the 50 000 times it is called. There might be something we can do with the loop in the function.
• the step function. Perhaps there are ways to reduce the time.
• the evaluate_policy function. Perhaps we should consider to reduce the number of episodes to evaluate
• collect_rollouts function in SB3. I found out that this function controls the callbacks. So the callbacks takes a significant amount of time.
• In the end it is the learn method, but I guess we cannot do much about that.

[tfha]

I have tested running on Azure but this did not work properly and I will try ones more with some new advices.