can we make individual `stcal` submodules into separate subPACKAGEs by turning this repo into a monorepo (with a shim `stcal` package on top)

[zacharyburnett]

Talking with @braingram and @WilliamJamieson and @mcara this morning, apparently several downstream users copy-paste individual algorithms and code from stcal because it's too much hassle to have the entire stcal as a dependency (due to compilation, extra dependencies, etc.). To cut off the possibility of divergence and fragmentation downstream, we could do the following:

1. convert this repo from a single-package repo into a multi-package repo ("monorepo"), with each submodule (jump, ramp_fitting, etc.) pushing to an individual package on PyPI. These packages would have their own wheels, versions, PyPI pages, etc., but all be based out of this single repository and keep the advantage of coupled issues, pull requests, and version control
2. keep a shim stcal meta-package that includes everything, bringing together the individual subpackages with common usages, for compatibility with the current functionality of stcal and existing downstream usages of stcal (especially the JWST and Roman pipelines), with the eventual goal of splitting existing downstream dependencies onto stcal into the separate subpackages

Pros

This would bring contributors back to stcal and lower the barrier to making it more of a community package, while making stcal itself less monolithic and more flexible in individual releases and versions of subpackages. Users would be able to pick and choose individual algorithms à la carte by picking jump, ramp_fitting, etc., or use the stcal package to include everything. This will also make the environments of jwst and romancal lighter, because they won't need to pull in all the dependencies from the other.

Cons

This would be a high-effort change, for perhaps not a lot of benefit? How many users use stcal (or copy stcal code) downstream?

Action Items

• Enumerate who exactly uses stcal, and if they're only using certain submodules

[EDIT: Keck doesn't copy-paste but instead pulls in the entire stcal package]

[zacharyburnett]

@spacetelescope/stcal-maintainers @spacetelescope/jwst-pipeline-developers @spacetelescope/romancal-developers we can probably think about this for the next hack day, if people think it's a good idea

[pllim]

I would say no unless Keck is promising developer time to help maintain if we do it. Otherwise we gain almost nothing from it but extra maintenance.

[braingram]

This has nothing to do with Keck.