SQLMesh extension clashes with running sqlmesh in terminal

[michiel-de-muynck]

I apologize for somewhat vague description because I do not fully know the exact circumstances that cause this, or what makes it go away. I'll do my best to provide as many details as I can.

Observed behavior

• I create a Github Codespace
• In that codespace, the devcontainer's Dockerfile installs sqlmesh to the system. Note: when using a venv instead, the issue is not (fully) there - see below.
• In settings.json, I enable the sqlmesh vscode extension and Python vscode extension
• The codespace builds as normal and the sqlmesh extension works as normal
• I open a terminal
• I run a SQLMesh command. Any command, including sqlmesh --version, or even just running python, all trigger the same issue: the sqlmesh extension immediately stops working.
  • Datatype annotations (e.g. ::BIGINT) disappear immediately. Go-to-definition and other commands do not work.
  • On the bottom-right several error messages pop up, multiple per second. Error messages include:
    • Error: command 'sqlmesh.external_model_update_columns' already exists
    • Server initialization failed.
    • (many others)
  • The logs of the sqlmesh extension show the error "Error: Client got disposed and can't be restarted". Among the (many) log messages I do not see anything more concrete than this.
• Reloading VSCode (F1 > Developer: Reload Window) fixes the issue until the next time I run any sqlmesh command in the terminal

Expected behavior

Running sqlmesh / Python commands in the terminal should not affect the VSCode extension (of course).

Additional info

• When using a venv instead of installing sqlmesh to the system, some aspects of the issue are still there, but not all:
  • When running Python commands in the terminal, the syntax highlighting and datatype annotations do briefly disappear
  • I also still get the error Server initialization failed on the bottom-right pop-up
  • However, the extension seems to "recover" and start working again a few seconds later
• Update: After some more experimentation it does not seem like venv versus system env is 100% what makes the difference. Most of the time, using a venv makes the extension able to recover after a crash after a few seconds, whereas using system Python environments or a venv which is owned by root, the extension usually does not recover, but this has not always been consistent. In all cases I do get the above issues: temporary crash, Server initialization failed error messages
• If I run /usr/local/bin/python in the terminal, this does trigger the crash, but if I copy /usr/local/bin/python to e.g. /usr/local/bin/python_copy and run that, the extension does not crash.
• Running sqlmesh or python commands in a subshell also does not cause the crash:
  • If I switch to a different user (with su) before running sqlmesh, the extension does not crash
  • If I run uvx sqlmesh, the extension does not crash
  • If I run sqlmesh in a Jupyter notebook (!sqlmesh), the extension does not crash
• I suspect that the culprit is one of the Microsoft VSCode Python extensions (Python or perhaps the new "Python Environments" extension), but can't really prove it. I tried disabling a lot of settings of those extension, but none made the issue disappear. Unfortunately, I can't try disabling the Python extension because it's a dependency of the sqlmesh extension.