[WIP] implement recursive analysis infrastructure

[abap34]

This PR enables JETLS to perform recursive analysis by following dependencies.

(Error detection in InlineStrings.jl when opening CSV.jl)

Overview of the Implementation

The implementation mimics Julia’s package loading: it locates the package and performs a properly handled include. In ultimatly simplified form, the process is just follows:

path = Base.locate_package(required_pkgid, required_env)
JET.handle_include(
  interp,
  Base.include,
  [Base.__toplevel__, path])

Additional Details

• A new configuration section is introduced:

[recursive_analysis]
max_depth = 0    # Limit of recursive analysis depth. Default = 0
exclude = []     # List of package names where recursion stops. Default = []

• Currently, analysis of many packages does not succeed completely. As a result, aggressively enabling recursive analysis will make the LS unstable. From a performance standpoint, the implementation is also not yet fast enough. Therefore, this feature should be disabled by default (based on discussions with aviatesk).

Open Discussions

• This implementation relies heavily on Julia’s internal package loading mechanisms. It may require maintenance to track upstream changes. If a more robust approach becomes available, it should replace the current one.

  • However, considering that the assets in loading.jl will likely be useful for future cache implementations, I think this approach is not a bad choice as the current direction to take. A alternative would be to share these assets with tools like Revise, but discussions on this have not progressed much yet.

• Also, since the current implementation performs very danger operations on loaded_modules, there is a possibility of critical errors occurring during the analysis of Base or when JETLS itself uses the same package as the one being analyzed. My view is that this can be addressed with detailed technical fixes (for example, enabling Base.register_root_module to load modules somewhere other than loaded_modules).

Future Work

• Caching

  • In the current implementation, non-stdlib packages are reloaded for each analysis.

• Config Enhancements

  • Support for settings such as regex in exclude.
  • Currently, only exact string matching is implemented, since the config manager lacks type-conversion functionality at load time. The plan is to extend the config system to support these features.

---

I opened this PR to share the direction early.

On the JET side, I plan to do some refactoring soon to remove the duplicated parts from usemodule_with_err_handling, which has become large and unwieldy. I also haven’t done much work yet on version checks or tests for a working implementation, so I will address those as well. Once these tasks are completed and the points listed under Open Discussions are resolved, I believe this can be merged.

[aviatesk]

I think we're good to go with this approach. We'll need to add tests, but that might be a bit tricky. Alternatively, we could use a test routine that directly analyzes LSInterpreter.

Besides the tests, is the plan for this PR to refactor the JET side to simplify its implementation? Or do you plan to address any items from open discussions or the future work list, which all sound reasonable to me.

[abap34]

Right now I’m working on refactoring and fixing some minor bugs related to module usage in JET. I think it will reduce rework if I fix these first and then proceed with the refactor.

Writing the overall tests themselves shouldn’t be a difficult, though the increase in test time might be a bit troublesome.