Merge pull request finos#274 from finos/feat/k8s-legacy-consolidation

feat(k8s-legacy): consolidate legacy k8s plugin under orb k8s-legacy

Author: cyriaquem

.gitattributes

@@ -1,3 +1,10 @@
 
 # Use bd merge for beads JSONL files
 .beads/issues.jsonl merge=beads
+
+# Enforce LF line endings for shell scripts
+*.sh text eol=lf
+
+# Legacy k8s-legacy/tools/ are shell scripts without .sh extension
+# (executed by kubelet via PATH lookup inside pods).
+k8s-legacy/tools/* text eol=lf

.github/workflows/dev-pypi.yml

@@ -19,6 +19,7 @@ on:
     branches: [main]
     paths:
       - 'src/**'
+      - 'k8s-legacy/**'
       - 'pyproject.toml'
       - 'requirements*.txt'
       - 'uv.lock'
@@ -100,14 +101,16 @@ jobs:
         repository-url: https://test.pypi.org/legacy/
         verbose: true
         skip-existing: true
+        packages-dir: dist/
 
     - name: Create deployment summary
       run: |
         {
-          echo "## Package Published Successfully"
+          echo "## Packages Published Successfully"
           echo ""
           echo "**Environment:** Test PyPI"
-          echo "**Installation:** \`pip install --index-url https://test.pypi.org/simple/ ${{ needs.config.outputs.pypi-name }}\`"
+          echo "**Installation (orb-py):** \`pip install --index-url https://test.pypi.org/simple/ ${{ needs.config.outputs.pypi-name }}\`"
+          echo "**Installation (k8s-legacy extra):** \`pip install --index-url https://test.pypi.org/simple/ ${{ needs.config.outputs.pypi-name }}[k8s-legacy]\`"
           echo ""
           echo "**Version:** $VERSION"
           echo "**Commands:**"

.github/workflows/security-code.yml

@@ -128,7 +128,11 @@ jobs:
         #   pywin32                 PSF-2.0                                                        -> PSF-2.0 (permissive, equivalent to BSD)
         #   cyclonedx-python-lib    no detected license                                            -> Apache-2.0 (METADATA: License: Apache-2.0)
         #   fastapi                 no detected license                                            -> MIT (METADATA: License-Expression: MIT)
-        allow-dependencies-licenses: 'pkg:pypi/werkzeug, pkg:githubactions/trufflesecurity/trufflehog, pkg:pypi/lark, pkg:pypi/mcp, pkg:pypi/rfc3987-syntax, pkg:pypi/wcwidth, pkg:pypi/lxml, pkg:pypi/protobuf, pkg:pypi/pywin32, pkg:pypi/cyclonedx-python-lib, pkg:pypi/fastapi'
+        #   aiohappyeyeballs        0BSD AND BSD-3-Clause AND GPL-1.0-or-later AND PSF-2.0 AND Python-2.0  -> PSF-2.0 (METADATA: License-Expression: PSF-2.0; permissive)
+        #   aiohttp                 no detected license                                            -> Apache-2.0 (METADATA: License-Expression: Apache-2.0 AND MIT; dual permissive)
+        #   docutils                BSD-2-Clause AND BSD-3-Clause AND CC-PDDC AND GPL-1.0-or-later AND GPL-3.0-only AND GPL-3.0-or-later AND ...  -> public-domain + BSD-2-Clause (METADATA: License: public domain, Python, 2-Clause BSD; GPL applies to a few isolated source files in the docutils source tree)
+        #   scramp                  no detected license                                            -> MIT-0 (METADATA: License-Expression: MIT No Attribution; permissive)
+        allow-dependencies-licenses: 'pkg:pypi/werkzeug, pkg:githubactions/trufflesecurity/trufflehog, pkg:pypi/lark, pkg:pypi/mcp, pkg:pypi/rfc3987-syntax, pkg:pypi/wcwidth, pkg:pypi/lxml, pkg:pypi/protobuf, pkg:pypi/pywin32, pkg:pypi/cyclonedx-python-lib, pkg:pypi/fastapi, pkg:pypi/aiohappyeyeballs, pkg:pypi/aiohttp, pkg:pypi/docutils, pkg:pypi/scramp'
 
   codeql-analysis:
     name: CodeQL Analysis

.github/workflows/validate-workflows.yml

@@ -69,4 +69,4 @@ jobs:
     - name: Validate shell scripts with shellcheck
       uses: ludeeus/action-shellcheck@00cae500b08a931fb5698e11e79bfbd38e612a38  # 2.0.0
       with:
-        ignore_paths: '.venv .git __pycache__ node_modules k8s'
+        ignore_paths: '.venv .git __pycache__ node_modules'

CHANGELOG.md

@@ -21,6 +21,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 
 ### Changed
 
+- The legacy Symphony-on-Kubernetes HostFactory plugin is now bundled with `orb-py` as an optional install extra rather than as a separate `open-resource-broker` PyPI package.  Install with `pip install "orb-py[k8s-legacy]"` and invoke via `orb k8s-legacy <command>`.  See `docs/root/operational/from-open-resource-broker.md` for details.  Runtime semantics are unchanged.
 - `[monitoring]` extra no longer includes `opentelemetry-instrumentation-boto` (use `[monitoring-aws]`).
 - `[all]` extra now includes `[all-providers]` so `pip install orb-py[all]` still pulls everything.
 - `[dev]` shim extra now includes `[all-providers]` for full local development.

README.md

@@ -373,6 +373,27 @@ curl http://localhost:8000/health
 
 </details>
 
+## Symphony HostFactory on Kubernetes (legacy)
+
+The `k8s-legacy` module is a Symphony HostFactory custom provider plugin for Kubernetes, predating the modern multi-cloud ORB architecture.  It is now bundled with `orb-py` as an optional install extra rather than as a separate PyPI package.
+
+Install with:
+
+```bash
+pip install "orb-py[k8s-legacy]"
+```
+
+Confirm the install by listing available templates:
+
+```bash
+orb k8s-legacy get-available-templates
+```
+
+The plugin is in maintenance mode.  A modern Kubernetes provider with native ORB integration is in development; existing deployments remain fully supported.
+
+- **Upgrading from `open-resource-broker`?** See the [migration guide](docs/root/operational/from-open-resource-broker.md).
+- **Deploying the Symphony HF plugin?** See the [k8s-legacy deployment guide](k8s-legacy/README.md).
+
 ## Project
 
 Architecture, development, and documentation.

docs/mkdocs.yml

@@ -154,6 +154,7 @@ nav:
     - Operations:
       - Backup and Recovery: operational/backup_recovery.md
       - Migration: operational/migration.md
+      - Migrate from open-resource-broker: operational/from-open-resource-broker.md
       - Performance: operational/performance.md
       - Monitoring: operational/monitoring.md
       - Tools: operational/tools.md

docs/root/operational/from-open-resource-broker.md

@@ -0,0 +1,110 @@
+# Migrating from `open-resource-broker` to `orb-py[k8s-legacy]`
+
+This guide is for operators who deployed the legacy `open-resource-broker` PyPI package and need to upgrade to the consolidated `orb-py` distribution.
+
+## What changed
+
+The Symphony-on-Kubernetes HostFactory plugin previously published as a standalone `open-resource-broker` package is now bundled with `orb-py` as an optional install extra at the `orb.k8s_legacy` sub-module path.  There is a single wheel: `orb-py`.  The heavy dependencies (kubernetes, fastapi, uvicorn, sqlalchemy, alembic, pg8000, prometheus-client, watchdog, inotify, and others) are behind the `[k8s-legacy]` extra so that operators running only the modern AWS provider do not pay for them.
+
+The runtime semantics are unchanged: same filesystem workdir layout, same push-based Kubernetes watchers, same event log format, same HostFactory API output.
+
+## Install
+
+| Before | After |
+|--------|-------|
+| `pip install open-resource-broker` | `pip install "orb-py[k8s-legacy]"` |
+
+The `open-resource-broker` package is no longer published to PyPI.  Pin to an older release only as a rollback option (see [Rollback](#rollback)).
+
+## CLI commands
+
+The standalone entry-point binaries (`open-resource-broker`, `open-resource-broker-admin`, `open-resource-broker-utils`, `open-resource-broker-events`) are replaced by subcommand groups under the unified `orb` binary.
+
+### Core HostFactory verbs
+
+| Before | After |
+|--------|-------|
+| `open-resource-broker request-machines '{...}'` | `orb k8s-legacy request-machines '{...}'` |
+| `open-resource-broker get-request-status '{...}'` | `orb k8s-legacy get-request-status '{...}'` |
+| `open-resource-broker request-return-machines '{...}'` | `orb k8s-legacy request-return-machines '{...}'` |
+| `open-resource-broker get-return-requests '{...}'` | `orb k8s-legacy get-return-requests '{...}'` |
+| `open-resource-broker get-available-templates` | `orb k8s-legacy get-available-templates` |
+| `open-resource-broker watch pods` | `orb k8s-legacy watch pods` |
+| `open-resource-broker watch request-machines` | `orb k8s-legacy watch request-machines` |
+| `open-resource-broker watch request-return-machines` | `orb k8s-legacy watch request-return-machines` |
+| `open-resource-broker watch events` | `orb k8s-legacy watch events` |
+| `open-resource-broker watch nodes` | `orb k8s-legacy watch nodes` |
+| `open-resource-broker watch kube-events` | `orb k8s-legacy watch kube-events` |
+| `open-resource-broker run-cron` | `orb k8s-legacy run-cron` |
+
+### Admin
+
+| Before | After |
+|--------|-------|
+| `open-resource-broker-admin <verb>` | `orb k8s-legacy admin <verb>` |
+
+### Utils and events-db
+
+| Before | After |
+|--------|-------|
+| `open-resource-broker-utils` | `orb k8s-legacy utils` |
+| `open-resource-broker-events transform` | `orb k8s-legacy events-db transform` |
+
+The `utils` server takes its configuration via flags (`--host`, `--port`, `--workdir`, `--platform`, `--cluster`, `--region`, `--namespace`, `--bucket`); it is not a subcommand group.  Pass flags directly: `orb k8s-legacy utils --host 0.0.0.0 --port 8080`.
+
+## HF shell scripts
+
+The five HostFactory provider scripts at `${HF_CONFDIR}/providers/k8s-hf/scripts/` now call `orb k8s-legacy <verb>` internally instead of `hostfactory <verb>` (or `open-resource-broker <verb>`).
+
+These scripts are repository artefacts, not Python package data, and are not bundled in the `orb-py` wheel.  After upgrading, replace your deployed copies with the updated versions from the repository at `k8s-legacy/hostfactory/providers/k8s-hf/scripts/`:
+
+```bash
+# From a checkout of finos/open-resource-broker at the release tag:
+cp k8s-legacy/hostfactory/providers/k8s-hf/scripts/*.sh "${HF_CONFDIR}/providers/k8s-hf/scripts/"
+```
+
+Or fetch the five scripts (`requestMachines.sh`, `getRequestStatus.sh`, `requestReturnMachines.sh`, `getReturnRequests.sh`, `getAvailableTemplates.sh`) individually from the release tarball or the GitHub source tree.
+
+## Operator daemon services
+
+If you have systemd units, supervisord configs, or similar process managers running the watcher daemons, update the `ExecStart` (or equivalent) line for each daemon.
+
+| Daemon | Before | After |
+|--------|--------|-------|
+| Pod watcher | `/usr/local/bin/open-resource-broker watch pods` | `/usr/local/bin/orb k8s-legacy watch pods` |
+| Request watcher | `/usr/local/bin/open-resource-broker watch request-machines` | `/usr/local/bin/orb k8s-legacy watch request-machines` |
+| Return watcher | `/usr/local/bin/open-resource-broker watch request-return-machines` | `/usr/local/bin/orb k8s-legacy watch request-return-machines` |
+| Cron runner | `/usr/local/bin/open-resource-broker run-cron` | `/usr/local/bin/orb k8s-legacy run-cron` |
+
+Adjust the binary path to match your virtual environment if `orb` is not on the system `PATH`.
+
+After updating the service definition, reload and restart:
+
+```bash
+systemctl daemon-reload
+systemctl restart hostfactory-watch-pods.service
+# repeat for each daemon unit
+```
+
+## Python imports
+
+This applies only if you embed the legacy plugin as a library.  Operator deployments that invoke the CLI exclusively are unaffected.
+
+| Before | After |
+|--------|-------|
+| `from open_resource_broker import ...` | `from orb.k8s_legacy import ...` |
+| `from open_resource_broker.api import ...` | `from orb.k8s_legacy.api import ...` |
+
+## Runtime semantics
+
+Unchanged.  The filesystem workdir layout (`/var/tmp/hostfactory` by default), Kubernetes watcher behaviour, event log format, and HostFactory API output are identical to the legacy standalone package.
+
+## Rollback
+
+To roll back to the standalone package, pin to the last published version:
+
+```bash
+pip install "open-resource-broker<1.0"
+```
+
+Or revert to a previous `orb-py` release that predates the consolidation.  Check the project changelog at the repository root for the exact version boundary.