Exclude custom variables from variable explorer

[battaglia01]

Issue Report Checklist

• [:)] Searched the issues page for similar reports
• [:)] Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
• [:)] Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
• [:)] Could not reproduce inside jupyter qtconsole (if console-related)
• [:)] Tried basic troubleshooting (if a bug/error)
  • [:)] Restarted Spyder
  • [:)] Reset preferences with spyder --reset
  • [:)] Reinstalled the latest version of Anaconda
  • [:)] Tried the other applicable steps from the Troubleshooting Guide
• [:)] Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

Sometimes one ends up with variables in the variable explorer that are not really relevant to the current session. The current settings make it easy to hide these in some cases: for instance, if the variables are in all uppercase, or if they're private, or callable. But there always seem to be a few which are left over, for which there is no useful way to get rid of them.

Rather than go into every possible category of thing we may want to hide, could we add the ability to just hide some user-created list of custom variables? I don't even care about GUI integration; something as simple as a global variable called __hide_var_in_spyder__ would be great, as long as Spyder adjusts the variable explorer every time this is updated. This would be a simple list of variables which Spyder knows to hide from the variable explorer. One step better would be if this can take default values as well, and only put it in the variable explorer if the current value is different from the hidden default value.

This is a silly example, but just to show how the behavior should work: suppose you have a global variable called myvar which is set to the int 123. So you set

__hide_var_in_spyder__ = [{"name": "myvar", "value": 123}]

where "myvar" is a string representing the name and 123 is the actual value (e.g. not a string). Now this is hidden in the variable explorer. On the other hand, if I then type myvar = 1234567890, it then shows in the explorer, because it doesn't match the value.

Is something like this already possible to do somehow, or if not, could this be easily added? I'd be willing to devote some time to implementing this if so.

[ccordoba12]

Hey @battaglia01, thanks for reporting. This is an interesting suggestion. You said:

I don't even care about GUI integration

Unfortunately, we can't do that because we'd be designing this feature just for you. I mean, without a GUI there's no way other people could ever find out about it.

This would be a simple list of variables which Spyder knows to hide from the variable explorer

That should be pretty simple to implement. The UI could be very similar to Edit filter settings in Files, plus a filter button in the Variable Explorer toolbar. Then we could take those names and add them to the list we already exclude.

One step better would be if this can take default values as well, and only put it in the variable explorer if the current value is different from the hidden default value.

That could be harder because we'd need an editable table of name/value entries to exclude. Also, could you help us to understand in which cases this is useful? I don't see the value in it.

[battaglia01]

I only suggested the GUI-less thing because I thought it'd be easier.

The main use of having editable values would be, for instance, if a variable that you'd like to hide ends up being shadowed temporarily. Suppose I have some global variable (perhaps imported from a library) that I want to hide. That's good, but then if that same name gets assigned some different value at some point, maybe within some smaller scope, perhaps I really do want to see it in the variable explorer at that point - both to notify that the variable as changed and now because it's being treated as a variable. I'm not sure if having a "default value" is the best way to do this but it was one simple thing I'd thought of.

[ccordoba12]

In my humble opinion, that seems of little value and more confusing for users.

[battaglia01]

The main point is just that it would be useful to hide variables in certain circumstances, which would certainly be of value to me. The default value was just one starting point and I agree it is probably not optimal.

Your idea of filtering seems good; I would suggest going with a regex rather than a glob. I would probably be happy enough with that as a starting point.

FWIW, while the "default value" thing was just something I came up with on the spur of the moment, what would really be the better suggestion is the ability to hide variables based on certain custom rules, similar to the ones you have already built into Spyder. It would certainly be useful to me to hide variables which are only of a certain class, or hide capitalized or uppercase variables only in certain circumstances, or whatever, with each rule potentially involving properties of the variable beyond just its name (i.e. its value, class, whatever). The interface could be something as simple as a textbox, where each line represents a condition, and you iterate over the lines, checking each variable against the rule - perhaps similar to breakpoint conditions. This could be potentially be a different feature than the regex thing above.

[battaglia01]

Actually, the list of rules would probably be easier and possibly subsume the regex idea. The interface is just a textbox and each line is a rule, and there are two special variables, perhaps called CURRENT_VAR_NAME and CURRENT_VAR, or something like that. Then, for instance:

If you have a bunch of constants all prefixed with the same thing which you want to hide:

CURRENT_VAR_NAME == 'DEFAULT_FGCOLOR'
CURRENT_VAR_NAME in ['DEFAULT_FGCOLOR', 'DEFAULT_BGCOLOR', 'DEFAULT_WHATEVER']
re.match(r"DEFAULT_.*", CURRENT_VAR_NAME)

If you want to hide variables of a certain class:

str(type(CURRENT_VAR)) == "<class 'numpy.lib.index_tricks.RClass'>"

If you have some mathematical constants like "phi" or "e", and you only want to hide those constants, not literally every single time someone has a variable named "e" in any codebase anywhere:

CURRENT_VAR_NAME == "e" and round(CURRENT_VAR, 10) == round(2.718281828459045, 10)

If you generally want to hide all-uppercase variables, except that you are doing linear algebra and have a bunch of capitalized matrix names you don't want to hide, so you check if they're ndarrays first:

CURRENT_VAR_NAME == CURRENT_VAR_NAME.upper() and not str(type(CURRENT_VAR)) == "<class 'numpy.ndarray'>"

Or maybe it's just that you have a bunch of one-to-three letter long names representing precomputed matrix products like USV:

CURRENT_VAR_NAME == CURRENT_VAR_NAME.upper() and not re.fullmatch(r"[A-Z]{,3}", CURRENT_VAR_NAME)

OK, I could go on but you get the idea. These are all some pretty typical use cases for where you may want to customize the behavior. I'm sure that users who are already familiar with conditional breakpoints will be able to use such an interface pretty intuitively and it would probably do most things you could possibly want the variable explorer to do.

[battaglia01]

FYI @ccordoba12 , I've patched the variable explorer and have been using this for about a year and a half now. Works really well.

I went with basically something like the general method above: having a list of custom expressions above which determine if the variable is shown or not. Basically similar to the way conditional breakpoints are entered. I tried a bunch of stuff but this was simplest, as well as most powerful.

I've been using it for a while and am pretty happy with it. Are you interested in incorporating this? I'd be happy to make some kind of interface for it if so.

[battaglia01]

Should also add that what I'm really doing, which is slightly easier than the above, is just to have one custom function which returns True if the variable should be shown or False if not.

If it's possible to fit that into the interface somehow, that would absolutely be the best way to do this; simple and you can put whatever logic you want.

Maybe just add a thing to the menu called "Add Custom Exclusion Handler..." which prompts for the name of a function? You can just put the function in your startup.ipy and be good to go. Would basically be perfect.

[ccordoba12]

I've been using it for a while and am pretty happy with it. Are you interested in incorporating this? I'd be happy to make some kind of interface for it if so.

Yeah, sounds really interesting and several people have asked for this kind of functionality before. As long as we have a nice and simple UI for it, I'm in.

Should also add that what I'm really doing, which is slightly easier than the above, is just to have one custom function which returns True if the variable should be shown or False if not.

This is the most advanced way of filtering variables, which perhaps would only be used by advanced users like you. A simpler solution would be to give users text fields to introduce patterns, regular expressions or functions to filter variables.

I'm thinking in something similar to what we have in the Files pane:

So, we could have a button in the pane's toolbar to enable filtering, an entry in its Options menu to edit filters and a dialog with three fields for filters: Patterns, Regular expressions and Function.

What do you think?

[battaglia01]

No, I'm not interested in a regex-only solution... as you mention above it would break my own use cases. I don't want to spend time building something out which is designed to work for everyone but me...

[battaglia01]

Ah wait a second, apologies as I may have misread. I thought you were saying to replace my suggestion w/ regex only. You're saying to make a new dialog with both regex and some kind of custom function. Is that correct? Then yes I agree that would be a good idea.