VScode extension lineage bug with multiple projects

[MisterWheatley]

Description

pip install sqlmesh[lsp]==0.224.0

When using the options for multiple projects introduced in #5084 the lineage view gives an error like

Error: '~/nested/project_1/models/seed_model.sql' is not in the subpath of '~/nested/project_2'

The other LSP features are working as expected with respect to e.g. finding definitions.

I suspect that the issue stems from this line in the Web API server, where it tries to take context.path relative to model._path. I believe that in the multi project setup, context.path will resolve to the last loaded project which is not necessarily relative to the current model path.

Example to reproduce bug

Setup two projects in a nested sub folder e.g.

.
└── root/
    └── nested/
        ├── project_1/
        │   ├── ...
        │   └── config.yaml
        └── project_2/
            ├── ...
            └── config.yaml

Ensure that both have different projects, but same gateway. You can setup the projects with sqlmesh init duckdb and edit the config.yaml files

Open VSCode and set workspace path to be at root/. Open settings (either UI or settings.json) and add either relative paths (./nested/project_1, ./nested/project_2) or absolute paths to the "sqlmesh.projectPaths" vscode extension variable

Open a model in project_1 and verify that e.g. go to definition part of extension still works

Open lineage panel and you should get an error like Error: '~/nested/project_1/models/seed_model.sql' is not in the subpath of '~/nested/project_2'

I would be happy to make a contribution to fix this if I could be pointed in the right direction as to how to solve it, as I am not that familiar with the Webserver API part of the project

[MisterWheatley]

The following is my best guess at the underlying logic of the problem, copied from slack thread

• LSP line uses web.server.api.endpoints.models.get_models in this line
• The get_models call serializes models based on passed Context object and tries to set a path relative to Context.path in this line
• However the Context.path attribute is not guarenteed to be relative to model._path in a multi repo context
  • The Context.path attribute is set in this line and is simply the first element from Context.configs

[MisterWheatley]

Another minimal example to reproduce the underlying bug/issue: (copied from slack thread aswell)

Setup/requirements pip install sqlmesh[web]
Navigate to sqlmesh/examples/multi

from sqlmesh.core.context import Context
from web.server.api.endpoints.models import get_models
paths = ["repo_1/","repo_2/"]

context = Context(paths = paths)

web_models = get_models(context)

Gives a ValueError from pathlib with the following message:
sqlmesh/examples/multi/repo_2/models/c.sql is not in the subpath of sqlmesh/example/multi/repo_1

By swapping the order of paths you instead get:
sqlmesh/examples/multi/repo_1/models/b.sql is not in the subpath of sqlmesh/examples/multi/repo_2

I.e. the web.server.api.endpoints.models.get_modelsmethod cannot handle a multi project context as it currently is