Local, dev (C++ built) dependencies

[gpetrovic-meltin]

What is the appropriate way of including local, dev dependencies (edit mode)?

We have a Python project with some internal dependencies that we keep as Git(hub) submodules. Out of the two submodule dependencies, one of them is a pure Python dependency, while the other includes C++ code as well (pybind is used to create a Python module out of it).

Here's how our current setup looks like:

[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"

[project]
name = "myproject"
description = "myproject"
dynamic = ['version']
requires-python = ">=3.10,<3.11"
dependencies = [
    "external-dependency=1.2.3",
    "submodule-dependency-1 @ file:///${PROJECT_ROOT}/libs/submodule-dependency-1",
    "submodule-dependency-2 @ file:///${PROJECT_ROOT}/libs/submodule-dependency-2",
]


[tool.pdm.dev-dependencies]
dev = [
    "pytest~=7.1",
    "-e file:///${PROJECT_ROOT}/libs/submodule-dependency-1#egg=submodule-dependency-1",
    "-e file:///${PROJECT_ROOT}/libs/submodule-dependency-2#egg=submodule-dependency-2",
]

[tool.hatch.version]
path = "src/__init__.py"

It's very common to work on the submodules while developing the main project, so I've listed the same submodules in both dependencies and dev-dependencies lists. This meant that pdm install -d would update them both without having to effectively nuke the .venv folder each time.

However, I'm not sure this is the right approach, and I've noticed that the C++ dependency is built from scratch each time - which is quite time consuming. I also don't think this was documented - I found it by trial and error and later rediscovered a similar solution here.

With Pipfile, it was enough to list the dependency once, and even consecutive pipenv install was sure to update it. Most C++ builds were also incremental from what I could see - they were pretty fast.