Allow ad-hoc color changing

[martinpopel]

As discussed in #288 and #449, it is difficult to choose one color palette that suits everyone in all situations.
If there are many runs and I select just some of them (with a regex or toggling individual runs), I quite often end up with e.g., red, red, orange, red and I cannot distinguish the runs especially if the curves are crossing each other.
I suggest to add a drop-down color picker (e.g. using http://jscolor.com/) to choose a color for each run.
I plan to use this especially before posting TensorBoard screenshots to my colleagues.

[nfelt]

This could be useful. Note that jscolor itself though we couldn't use since it's GPL.

[martinpopel]

OK, jscolor is not needed (it was just an example). Choosing e.g. from 16 pre-defined colors would be enough for my needs.

[jart]

Our team probably can't do something like this at the moment. I suspect configuring color might be more appropriate if done in code/config rather than UI.

[martinpopel]

I understand there may be higher priority features to implement. Just note that configuring colors in code/config would not solve this issue.
The point is that I don't know in advance which runs I will want to compare. I want to change the colors ad hoc from the GUI.
Also in case of multiple users connecting to the same TensorBoard server, I think the colors should be per-user configurable, i.e. client-side.

It is a question whether to solve the run<-->color mapping (semi)permanently in the browser. If possible this may be useful.
Another issue with TensorBoard is that when I delete a run (e.g. to save disk space), all colors change, so I have to redo all screenshots again to have consistent colors.

[netheril96]

A way to change color in config/code/command line is better than nothing, although I'd favor a Web UI too.

[jart]

Is the root problem that colors are not predictable and might change as you add additional runs? I'm not sure how color selection works at the moment. Although one possible thing to consider is using hash(run_name) % #colors to make color selection somewhat more deterministic.

Could that solve your problem?

[martinpopel]

The color of a given run changes only if I delete any older runs. This is bothersome, but it is not the main issue I originally reported.

My issue is:
I have hundreds of runs in TensorBoard and I select various subsets using regex filters or checkboxes.
Each subset has at most eight runs, so it would be possible to use distinct color for each run, but quite often this is not the case.

Maybe there could be a button "change colors" which would automatically select colors so the currently selected runs have distinct colors - this would solve my issue.
However, I think my original proposal is more useful: by manual ad-hoc coloring, the users can choose the colors exactly as they want (e.g. using red for a "failed" run and gray for baseline).
Of course, we could have both a button for automatic re-coloring and an ad-hoc color picker.

BTW: My current workaround is to create symlinks for a given run until one of the symlinks gets assigned a suitable color. Then I keep visible just that symlink and hide all previous symlinks plus the original.

[jart]

We're currently building SQL support so Runs can be grouped into Experiments, Experiments can be grouped into Projects, which are owned by Users. The next logical step is to add a UI feature that lets you compare an intersection of experiments.

In other words, we're working towards better solutions to organizing data. Do you suspect that might solve the root problem here?

[martinpopel]

All that is great, but I don't see how it could help with this problem without adding the color picker.
I keep my runs organized in a directory structure already, e.g. training-dataset/number-of-gpus/batch-size/learning-rate/warmup-steps/etc.
In one minute I want to see all runs on a given training data, then I want to compare runs with all hyperparmeters fixed except for batch size, then I want to see the effect of number of GPUs, etc..
So there is no single way how to organize the runs in Projects/Experiments/Groups such that I would always compare only runs within one group.
Specifically for this issue, there is no fixed mapping of runs to colors which would suit all my needs.

[jart]

One thing you might be able to do is render a matplotlib image and log it as an image summary.

[martinpopel]

One thing you might be able to do is render a matplotlib image and log it as an image summary.

This is not even a workaround (and it would require hack all the TF frameworks again): I don't know in advance which colors should each run have - I decide this only when using TensorBoard and selecting the subset of runs.

My current workaround is to export the plots to csv files and plot these with gnuplot-lua-tikz (which is more suitable for academic papers than screenshots anyway). But this is not suitable for interactive analysis in TensorBoard (zooming, exact numbers it tooltips, etc.), similarly as what you suggested.

I think this discussion starts getting too long and not focused on the main problem and its solution (color picker), but rather on workarounds.
If you feel this issue has low-priority and there is no one to implement it, let's close it and go on.
A summary of workarounds (for other users):

• clicking the show/hide checkbox for each run there and back to see "which of the four red curves (crossing each other) is which one"
• changing the color of a given run using the symlink hack
• create a new directory with symlinks to the runs you are interested in and start a new TensorBoard instance with --logdir pointing to this directory
• exporting the runs to csv and re-plotting in your favorite visualization tool