Packaging workflow

[jcushman]

See docs/adr/release_surface_and_automated_publishing.md. This sets us up for automated publishing of three packages: binoc and binoc-sqlite (our example plugin) on PyPI, and binoc-sdk on cargo for writing rust plugins. Currently all three are published with a single version tag. Under more active development we might want things like a binoc github org, packages in separate repos, and/or package-specific tags in the monorepo.

[cmsetzer]

Looks good overall—thanks for building this out. I included a few suggestions, mostly about consolidating shared workspace metadata and dependencies in the root Cargo.toml.

[cmsetzer]

Seems reasonable for us to start by supporting Python 3.10+ and then incrementally drop support for Python versions as they hit EOL per the schedule. Sound good?

[cmsetzer]

We should consolidate shared values in the workspace Cargo.toml and then inherit them within the individual package Cargo.tomls. I believe we can do this for the version, edition, license, repository, homepage, and documentation fields.

So, the workspace-level Cargo.toml would have:

[workspace.package]
version = "0.1.0"
edition = "2021"
license = "MIT"
repository = "https://github.com/harvard-lil/binoc"
homepage = "https://github.com/harvard-lil/binoc"
documentation = "https://github.com/harvard-lil/binoc/tree/main/docs"

And then in binoc-cli/Cargo.toml:

[package]
name = "binoc-cli"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
homepage.workspace = true
documentation.workspace = true
description = "Standalone Rust CLI for Binoc"
publish = false

[cmsetzer]

Likewise we should specify shared workspace dependencies in the root Cargo.toml, and then inherit them within individual packages.

E.g., in the workspace Cargo.toml:

[workspace.dependencies]
binoc-sdk = { path = "binoc-sdk" }
binoc-core = { path = "binoc-core" }
binoc-stdlib = { path = "binoc-stdlib" }
binoc-cli = { path = "binoc-cli" }

serde = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.149"
tempfile = "3.26.0"
pyo3 = { version = "0.27", features = ["extension-module", "abi3-py310"] }

And then in binoc-python/Cargo.toml:

[dependencies]
binoc-core = { workspace = true }
binoc-sdk = { workspace = true }
binoc-stdlib = { workspace = true }
binoc-cli = { workspace = true }
libloading = "0.8"
pyo3 = { workspace = true }
serde_json = { workspace = true }

[cmsetzer]

From my read of the PyPI docs, seems like we create the project(s) via trusted publisher first, then we can add them to the harvard-lil org?

[jcushman]

I made the DRY changes, and tightened a few security things before deploy -- bump a tar dependency, lock GH action versions, add dependabot, check for path traversal in plugins (I think that only prevents accidental path traversal, because they have full code execution anyway, but good defense in depth).