ModuleNotFoundError for SMAC

[ArlindKadra]

Hey all,

After having a clean environment, installing all the dependencies as suggested in the repo, the example code fails with the following error:
ModuleNotFoundError: No module named 'smac.tae.execute_ta_run'

Code:

import os
from cave.cavefacade import CAVE
  
task_path = 'path to results'
cave_output_dir = os.path.join(
    task_path,
    'cave_output',
)

cave = CAVE(
    [task_path],  # List of folders holding results
    cave_output_dir,  # Output directory
    ['.'],  # Target Algorithm Directory (only relevant for SMAC)
    file_format="APT",
    verbose="DEBUG"
)

[ArlindKadra]

Python 3.6
The environment:

argcomplete==1.12.1
async-generator==1.10
attrs==20.3.0
bokeh==1.1.0
-e git+https://github.com/automl/CAVE@f605e4f7b2a42896c047386b124c6f0c0d1b6086#egg=cave
certifi==2020.6.20
cffi==1.14.3
click==7.1.2
cloudpickle==1.6.0
ConfigSpace==0.4.16
contextvars==2.4
cryptography==3.2.1
cycler==0.10.0
Cython==0.29.21
dask==2.30.0
distributed==2.30.1
docutils==0.16
fanova==2.0.19
h11==0.11.0
HeapDict==1.0.1
hpbandster==0.7.4
idna==2.10
immutables==0.14
importlib-metadata==2.0.0
ipaddress==1.0.23
Jinja2==3.0.0a1
joblib==0.17.0
kiwisolver==1.3.1
lazy-import==0.2.2
MarkupSafe==2.0.0a1
matplotlib==3.3.3
msgpack==1.0.0
multidict==5.0.2
netifaces==0.10.9
nose==1.3.7
numpy==1.19.4
outcome==1.1.0
packaging==20.4
pandas==1.1.4
patsy==0.5.1
Pillow==8.0.1
psutil==5.7.3
pycparser==2.20
PyImp==1.1.2
pynisher==0.6.1
pyOpenSSL==19.1.0
pyparsing==2.2.0
pyrfr==0.8.0
Pyro4==4.80
python-dateutil==2.8.1
pytz==2020.4
PyYAML==5.3.1
scikit-learn==0.23.2
scipy==1.5.4
selenium==4.0.0a7
serpent==1.30.2
six==1.15.0
sklearn==0.0
smac==0.13.1
sniffio==1.2.0
sobol-seq==0.2.0
sortedcontainers==2.3.0
statsmodels==0.12.1
tblib==1.7.0
threadpoolctl==2.1.0
toolz==0.11.1
tornado==6.1
tqdm==4.52.0
trio==0.17.0
trio-websocket==0.8.1
typing-extensions==3.7.4.3
urllib3==1.26.2
wsaccel==0.6.2
wsproto==0.14.1
yarl==1.6.3
zict==2.0.0
zipp==3.4.0

[KEggensperger]

We changed some things in SMAC which have not yet been adapted by the dependencies of cave. Depending on your general setup up downgrading smac via pip install smac=0.12.1 should work.

[ArlindKadra]

@KEggensperger I already did that and it still did not work. There was an error related to pyrfr.regression.

[shukon]

@ArlindKadra can you post the traceback? It's a different error, so more information is needed.

[ArlindKadra]

@shukon I can post it later, but it was the same module not found error as with smac.

[ArlindKadra]

@KEggensperger @shukon Update, I tested it with smac == 0.12 before. With 0.12.1 it seems to work, I am currently getting a Segmentation Fault core dumped error, but that is related to swig and not to this issue anymore.

[shukon]

Yes, breaking backwards-compatibility to smac==0.12.0 when introducing support for smac==0.12.1 in CAVE 1.3.3 was not the best decision from a user-perspective, but made sense with regard to the structures that changed in SMAC3 at the time... could have done with a better versioning on CAVE's side. Anyway, glad it's working.
Depending on your environment you will find solutions to the swig-issue in #292 or automl/random_forest_run#51 while the recommended solution is using Anaconda with the working swig-version.

[ArlindKadra]

@shukon Yeah thanks, I already managed to get it running :)

[Tillhs]

Hi! I ran into the same issue when I tried to use cave. With a downgraded version of smac I didn't experience those issues. Nevertheless, I would love to use the latest version of smac for my project. Therefore, I was wondering wether you are working on a fix for this issue and if so, when do you expect to have it fixed.
Thanks a lot for your help and happy holidays!

[shukon]

I think there is a PR for this, I will look into it asap.

[Tillhs]

Thanks for your quick response. I saw the PR and did also change those imports locally. That didn't fully help though, I just ran into more Errors after that. But maybe that was just the case for my specific project.