expose public regulated data types as a python package

[ot-goegelem]

Problem

I want to write a small tool that uses pycyphal and the public regulated data types.
But is quite annoying that it is not possible to just add/install the public regulated data types as a dependency.

I understand that this is a deliberate decision to make it possible to add custom data types, but my tool does not rely on custom data types and I want to being able to have one setup with all dependencies without needing some custom setup from the end user.

Solution

It is then possible to use this dependency by just specifying the git repository

pip install git+ssh://[email protected]/ot-goegelem/public_regulated_data_types.git@feature/python_package

[pavel-kirienko]

Thanks! Please see my comment inquiring if it's possible to strap down the setup.py

[pavel-kirienko]

Is the following not a valid replacement?

import setuptools
import distutils.command.build_py
from pathlib import Path

class BuildPy(distutils.command.build_py.build_py):
    def run(self):
        import pycyphal
        output_dir = Path(self.build_lib, self.pkg_name, ".demo_dsdl_compiled")
        cur_path = pathlib.Path(__file__).parent
        pycyphal.dsdl.compile_all(
            [
                cur_path / "uavcan",
                cur_path / "reg",
            ],
            output_directory=output_dir,
        )
        super().run()

setuptools.setup(cmdclass={"build_py": BuildPy})

[pavel-kirienko]

Thank you. I think the use case is very valid but I don't want to add Python-specific things to this repository because that will become cumbersome to maintain. We should create a new repository instead that is solely dedicated to republishing DSDL definitions as a Python package. There could be similar packages published for other languages. That repository could simply automatically reupload releases every week or so, running as a cron CI job.

If I created a new repo, would you be able to move your changes there? Can you also help us set up the CD pipeline for automatic publication?

@thirtytwobits @emrainey any objections?

[ot-goegelem]

I would rather not do it periodically via a cron job.

I did a quick research and it is possible to inject build settings (config_settings, see https://peps.python.org/pep-0517/) in a package build.
What is your opinion on having another repository that you can directly install via pip and you can pass the public_regulated_data_types commit hash you want to install via config_settings to the build process. It will then clone the respective public_regulated_data_types commit and build/install those types.

[pavel-kirienko]

As long as the default behavior is to install the latest master, I'm good with that. I suppose public_regulated_data_types_pip sounds like a good name?

[ot-goegelem]

Sounds like a plan 🤗

[pavel-kirienko]

okay let's give Scott and/or Erik some time to think about it as well

[emrainey]

I like it.

[pavel-kirienko]

Okay I created the new repo here: https://github.com/OpenCyphal-Garage/public_regulated_data_types_pypi

We will roll it out of the garage once it's ready.

@ot-goegelem can you please send a PR there? If you could set up a nice readme in the process that would help! Thanks!

[pavel-kirienko]

@ot-goegelem I added a repository secret called PYPI_TOKEN that allows publishing the package on https://test.pypi.org. It will not work for the main pypi.org; we will move it there later once it's done.