You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
pip install -e DEHB # -e stands for editable, lets you modify the code and rerun things
17
-
```
10
+
Welcome to DEHB, an algorithm for Hyperparameter Optimization (HPO). DEHB uses Differential Evolution (DE) under-the-hood as an Evolutionary Algorithm to power the black-box optimization that HPO problems pose.
18
11
19
-
### Tutorials/Example notebooks
12
+
`dehb` is a python package implementing the [DEHB](https://arxiv.org/abs/2105.09821) algorithm. It offers an intuitive interface to optimize user-defined problems using DEHB.
20
13
21
-
*[00 - A generic template to use DEHB for multi-fidelity Hyperparameter Optimization](examples/00_interfacing_DEHB.ipynb)
22
-
*[01.1 - Using DEHB to optimize 4 hyperparameters of a Scikit-learn's Random Forest on a classification dataset](examples/01.1_Optimizing_RandomForest_using_DEHB.ipynb)
23
-
*[01.2 - Using DEHB to optimize 4 hyperparameters of a Scikit-learn's Random Forest on a classification dataset using Ask & Tell interface](examples/01.2_Optimizing_RandomForest_using_Ask_Tell.ipynb)
24
-
*[02 - Optimizing Scikit-learn's Random Forest without using ConfigSpace to represent the hyperparameter space](examples/02_using%20DEHB_without_ConfigSpace.ipynb)
25
-
*[03 - Hyperparameter Optimization for MNIST in PyTorch](examples/03_pytorch_mnist_hpo.py)
26
-
27
-
To run PyTorch example: (*note additional requirements*)
14
+
### Getting Started
15
+
#### Installation
28
16
```bash
29
-
python examples/03_pytorch_mnist_hpo.py \
30
-
--min_fidelity 1 \
31
-
--max_fidelity 3 \
32
-
--runtime 60 \
33
-
--verbose
17
+
pip install dehb
34
18
```
35
-
36
-
#### Ask & Tell interface
37
-
DEHB allows users to either utilize the Ask & Tell interface for manual task distribution or leverage the built-in functionality (`run`) to set up a Dask cluster autonomously.
38
-
The Ask & Tell functionality can be utilized as follows:
19
+
#### Using DEHB
20
+
DEHB allows users to either utilize the Ask & Tell interface for manual task distribution or leverage the built-in functionality (`run`) to set up a Dask cluster autonomously. The following snippet offers a small look in to how to use DEHB. For further information, please refer to our [getting started examples](https://automl.github.io/DEHB/latest/getting_started/single_worker/) in our documentation.
39
21
```python
40
22
optimizer = DEHB(
41
-
f=your_target_function,# Here we do not need to necessarily specify the target function, but it can still be useful to call 'run' later.
23
+
f=your_target_function,
42
24
cs=config_space,
43
25
dimensions=dimensions,
44
26
min_fidelity=min_fidelity,
45
27
max_fidelity=max_fidelity)
46
28
29
+
##### Using Ask & Tell
47
30
# Ask for next configuration to run
48
31
job_info = optimizer.ask()
49
32
@@ -52,79 +35,36 @@ result = your_target_function(config=job_info["config"], fidelity=job_info["fide
52
35
53
36
# When you received the result, feed them back to the optimizer
54
37
optimizer.tell(job_info, result)
55
-
```
56
-
57
-
### Running DEHB in a parallel setting
58
-
59
-
DEHB has been designed to interface a [Dask client](https://distributed.dask.org/en/latest/api.html#distributed.Client).
60
-
DEHB can either create a Dask client during instantiation and close/kill the client during garbage collection.
61
-
Or a client can be passed as an argument during instantiation.
62
38
63
-
* Setting `n_workers` during instantiation \
64
-
If set to `1` (default) then the entire process is a sequential run without invoking Dask. \
65
-
If set to `>1` then a Dask Client is initialized with as many workers as `n_workers`. \
66
-
This parameter is ignored if `client` is not None.
67
-
* Setting `client` during instantiation \
68
-
When `None` (default), a Dask client is created using `n_workers` specified. \
69
-
Else, any custom-configured Dask Client can be created and passed as the `client` argument to DEHB.
70
-
71
-
#### Using GPUs in a parallel run
72
-
73
-
Certain target function evaluations (especially for Deep Learning) require computations to be
74
-
carried out on GPUs. The GPU devices are often ordered by device ID and if not configured, all
75
-
spawned worker processes access these devices in the same order and can either run out of memory or
76
-
not exhibit parallelism.
77
-
78
-
For `n_workers>1` and when running on a single node (or local), the `single_node_with_gpus` can be
79
-
passed to the `run()` call to DEHB. Setting it to `False` (default) has no effect on the default setup
80
-
of the machine. Setting it to `True` will reorder the GPU device IDs dynamically by setting the environment
81
-
variable `CUDA_VISIBLE_DEVICES` for each worker process executing a target function evaluation. The re-ordering
82
-
is done in a manner that the first priority device is the one with the least number of active jobs assigned
83
-
to it by that DEHB run.
84
-
85
-
To run the PyTorch MNIST example on a single node using 2 workers:
86
-
```bash
87
-
python examples/03_pytorch_mnist_hpo.py \
88
-
--min_fidelity 1 \
89
-
--max_fidelity 3 \
90
-
--runtime 60 \
91
-
--n_workers 2 \
92
-
--single_node_with_gpus \
93
-
--verbose
39
+
##### Using run()
40
+
# Run optimization for 1 bracket. Output files will be saved to ./logs
41
+
traj, runtime, history = optimizer.run(brackets=1, verbose=True)
94
42
```
95
43
96
-
#### Multi-node runs
44
+
#### Running DEHB in a parallel setting
45
+
For a more in-depth look in how-to run DEHB in a parallel setting, please have a look at our [documentation](https://automl.github.io/DEHB/latest/getting_started/parallel/).
97
46
98
-
Multi-node parallelism is often contingent on the cluster setup to be deployed on. Dask provides useful
99
-
frameworks to interface various cluster designs. As long as the `client` passed to DEHB during
100
-
instantiation is of type `dask.distributed.Client`, DEHB can interact with this client and
101
-
distribute its optimization process in a parallel manner.
102
-
103
-
For instance, `Dask-CLI` can be used to create a `dask-scheduler` which can dump its connection
104
-
details to a file on a cluster node accessible to all processes. Multiple `dask-worker` can then be
105
-
created to interface the `dask-scheduler` by connecting to the details read from the file dumped. Each
106
-
dask-worker can be triggered on any remote machine. Each worker can be configured as required,
107
-
including mapping to specific GPU devices.
47
+
### Tutorials/Example notebooks
108
48
109
-
Some helper scripts can be found [here](utils/), that can be used as a reference to run DEHB in a multi-node
110
-
manner on clusters managed by SLURM. (*not expected to work off-the-shelf*)
49
+
*[00 - A generic template to use DEHB for multi-fidelity Hyperparameter Optimization](examples/00_interfacing_DEHB.ipynb)
50
+
*[01.1 - Using DEHB to optimize 4 hyperparameters of a Scikit-learn's Random Forest on a classification dataset](examples/01.1_Optimizing_RandomForest_using_DEHB.ipynb)
51
+
*[01.2 - Using DEHB to optimize 4 hyperparameters of a Scikit-learn's Random Forest on a classification dataset using Ask & Tell interface](examples/01.2_Optimizing_RandomForest_using_Ask_Tell.ipynb)
52
+
*[02 - Optimizing Scikit-learn's Random Forest without using ConfigSpace to represent the hyperparameter space](examples/02_using%20DEHB_without_ConfigSpace.ipynb)
53
+
*[03 - Hyperparameter Optimization for MNIST in PyTorch](examples/03_pytorch_mnist_hpo.py)
111
54
112
-
To run the PyTorch MNIST example on a multi-node setup using 4 workers:
55
+
To run PyTorch example: (*note additional requirements*)
113
56
```bash
114
-
bash utils/run_dask_setup.sh \
115
-
-f dask_dump/scheduler.json \ # This is how the workers will be discovered by DEHB
116
-
-e env_name \
117
-
-n 4
118
-
119
-
# Make sure to sleep to allow the workers to setup properly
120
-
sleep 5
121
57
python examples/03_pytorch_mnist_hpo.py \
122
58
--min_fidelity 1 \
123
59
--max_fidelity 3 \
124
60
--runtime 60 \
125
-
--scheduler_file dask_dump/scheduler.json \
126
61
--verbose
127
62
```
63
+
### Documentation
64
+
For more details and features, please have a look at our [documentation](https://automl.github.io/DEHB/latest/).
65
+
66
+
### Contributing
67
+
Any contribution is greaty appreciated! Please take the time to check out our [contributing guidelines](./CONTRIBUTING.md)
0 commit comments