browse_experiments() error

[filipeabperes]

When running browse_experiments() I get the following error (substituted in)

>>> from artemis.experiments.ui import browse_experiments as be
>>> be()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<HOME>/anaconda3/lib/python3.6/site-packages/artemis/experiments/ui.py", line 77, in browse_experiments
    browser.launch(command=command)
  File "<HOME>/anaconda3/lib/python3.6/site-packages/artemis/experiments/ui.py", line 279, in launch
    print(self.get_experiment_list_str(self.exp_record_dict))
  File "<HOME>/anaconda3/lib/python3.6/site-packages/artemis/experiments/ui.py", line 390, in get_experiment_list_str
    longest_row = max(max(len(r) for r in record_table_rows), max(len(r) for r in experiment_table_rows)+4) if len(record_table_rows)>0 else 0
ValueError: max() arg is an empty sequence

If I open any of the experiments directly via

>>> from file import experiment
>>> experiment.browse()

The UI pops up but no runs of variants are shown. Not sure if I'm doing anything wrong. Ideas?

[matthiasreisser]

Hey @filipeabperes, sorry for the super late response. It seems to me that you are not actually registering any experiments before calling browse_experiments. Make sure to have at least one experiments registered. I will look into providing a more sensible exception handling.

from artemis.experiments import experiment_function
from artemis.experiments.ui import browse_experiments as be

@experiment_function
def my_experiment():
    pass

be()

This is why in your second example, you don't see the error, since presumable experiment was decorated as an experiment.

Alternatively you can do:

from artemis.experiments.ui import browse_experiments as be
from file import experiment
be()

which should work as well.

Variants will show up if you run the code where they were defined ( Which is a bit cumbersome). If I want to import experiments and their variants, what I usually is do is something like this:
In file.py:

@ExperimentFunction(is_root=True)
def my_experiment():
    pass

def load_experiments():
    my_experiment.add_variant("Variant1")
    #... 

And in run_experiments.py:

from artemis.experiments.ui import browse_experiments as be
from file import load_experiments

if __name__ == '__main__':
    load_experiments() # This registers the variants within artemis
    be()

Hope that helps!

Matthias

[filipeabperes]

@matthiasreisser Thanks for the reply. So what you're saying is that for you to visualize the results of an experiment, you have to re-import such experiment each time. The same goes for variants, they have to be re-added every time you want to visualize their results.

Shouldn't there be a way to visualize experiments/variants without having to go through this? Seems like it would be feasible, since the results are already saved in the .artemis/experiments folder anyway. What I have been doing lately (since I'm mostly concerned with logs and parameters) is simply inspecting the files in the experiment folders manually

[matthiasreisser]

Yes. The idea is that the experiment_browser serves as the entry point for not only running experiments but also managing them. Deleting, comparing, visualizing, etc.
In order for the experiment_browser to know which experiments (and their variants) to "browse", you have to reimport them.

You don't have to use the experiment browser though. The Experiment class gives you an api to read records directly (thus giving you access to parameters, logs, results etc.) You can write custom code to access that data and inspect it, if you want.

browse_experiment_records in artemis.experiments.ui for example goes through your experiment directory and simply gives you all experiments you ever ran. This might be what you want since it does not filter by the experiments you registered.

[petered]

I think what Matthias said about browse_experiment_records is the way to go (though it hasn't been maintained in a while so I'm not 100% on everything working in that UI). Alternatively, to just access things programmatically, check out the ExperimentRecord object. You can instantiate an ExperimentRecord object with the directory in which that record was saved, and it has all the methods for accessing logs, results, arguments, etc.