UpSet plots do not render when all units were classified for one single reason

[nikhilchandra]

I previously described this issue in reply to #336 on July 2nd, but decided to submit a a new one here for better compartmentalization.

Briefly, UpSet plot generation fails under specific circumstances, with warnings like these:

Warning: Could not create noise upset plot due to library compatibility: 'Index' object has no attribute 'levels'
Warning: Could not create non-somatic upset plot due to library compatibility: 'Index' object has no attribute 'levels'
Warning: Could not create MUA upset plot due to library compatibility: 'Index' object has no attribute 'levels'

When I first reported this problem, I noted that it stems stems from the upsetplot function from_indicators(), which takes two parameters - metric_display_names and unit_type_data - as inputs. unit_type_data is a pandas DataFrame whose columns correspond to tests that were used in the classification of the unit (I think - please correct me if I'm wrong).

When more than one column in unit_type_data has True values, then from_indicators() returns a DataFrame with a multi-level index (i.e., MultiIndex), and UpSet plot generation works properly. However, should only one column in unit_type_data have True values, then from_indicators() returns a DataFrame with a regular index (type: Index) - this causes UpSet plot generation to break.

I am unsure whether this is expected behavior or bug in the upsetplot package (in which case perhaps we should submit a bug report).

@Julie-Fabre since you have previously been unable to reproduce this problem at your end, I've created a repo here - https://github.com/nikhilchandra/debugging_upset_plots

When you have a moment, perhaps you could clone this project, set up an environment with uv, and run main.py? I think you will quickly see what I'm talking about.

[Julie-Fabre]

Hi Nikhil, Thank you for all the details and for this repo! That's incredibly helpful. Are you still getting this error with your latest PR?

[nikhilchandra]

Hi Julie, yes - my PR does not stop the @jnothman's upsetplot package from breaking under the conditions I described. All it does is allow me to introduce logic in my own calling code that detects the new warning and change behavior accordingly.

So far I've hesitated from submitting a bug report (the last commit to this project was 12/31/2023), but will try to do so when I have the bandwidth.

[Julie-Fabre]

Gotcha.
Yes, I noticed that too. I don't think this project is still actively maintained (also judging by the number of recent issues) and there is to my knowledge no currently maintained upset plot package in python. I think the best solution is to fork it, fix and maintain it like that but this does require quite a bit of work. I think I will go that route, but it is low on the priority list right now. I'll keep you posted.