Skip to content

feat(telco-kpis): Implement unified lockdowns role with hub capture and parse#514

Open
ccardenosa wants to merge 2 commits into
openshift-kni:mainfrom
ccardenosa:feat/ipa-telco-kpis-prow-migration-lockdown-hub-operators
Open

feat(telco-kpis): Implement unified lockdowns role with hub capture and parse#514
ccardenosa wants to merge 2 commits into
openshift-kni:mainfrom
ccardenosa:feat/ipa-telco-kpis-prow-migration-lockdown-hub-operators

Conversation

@ccardenosa

@ccardenosa ccardenosa commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Implements a unified lockdowns role for hub/spoke operator lockdown management,
enabling repeatable OpenShift cluster deployments with locked operator versions.

Key Features:

1. Unified Role Structure (lockdowns)
   - Mode-based dispatcher (hub/spoke) with action (parse/capture)
   - Common utilities: download, validation, symlink generation
   - Separate task files for hub and spoke operations
   - Jinja2 templates for lockdown JSON generation

2. Hub Lockdown Capture
   - Queries cluster for OCP version, Subscriptions, OperatorGroups, CatalogSources
   - Detects architecture via skopeo (x86_64/arm64, multi-arch support)
   - Maps mirrored catalog names to upstream (cs-redhat-operator-index-* → redhat-operators)
   - Enriches FBC operators with mirroring metadata:
     * fbc_iib_repo: 'latest'
     * ocp_operator_mirror_fbc_image_base: quay.io/redhat-user-workloads/telco-5g-tenant/{catalog}-{version}
   - Generates dual-format OCP pull specs (digest + tag)
   - Outputs timestamped lockdown JSON with cluster metadata

3. Hub Lockdown Parse
   - Downloads lockdown JSON from URI (GitLab, Gitea, file://)
   - GitLab symlink resolver handles multi-hop symlink chains:
     * lockdown-hub-x86_64.json → lockdown-hub-4.21-x86_64.json → ../4.21/actual.json
     * Detects JSON vs symlink text, resolves relative paths
     * Max 5 hops protection
   - Uses slurp module (not lookup) for SSH compatibility (Ansible controller vs remote host)
   - Validates lockdown structure (required fields, nested hierarchy)
   - Transforms operators for upstream compatibility:
     * Adds 'nsname' field from 'namespace' (upstream role requirement)
   - Sets facts: hub_lockdown_operators, hub_ocp_pull_spec, hub_ocp_version

4. Telco-KPIs Wrapper Playbook (deploy-ocp-operators.yml)
   - Three-phase workflow:
     * Phase 1: Parse lockdown (if hub_lockdown_uri provided) OR use parameters
     * Phase 2: Call upstream deploy-ocp-operators.yml with transformed operators
     * Phase 3: Capture lockdown (if generate_hub_lockdown requested)
   - Integrates with Jenkins install-hub-operators job
   - Supports both lockdown mode and parameter mode

5. Lockdown JSON Format
   - Nested structure: {hub: {ocp: {...}, operators: [...], metadata: {...}}}
   - OCP pull_spec with both digest (immutable) and tag (human-readable)
   - Operators include: name, namespace, catalog, channel, subscription_name,
     installed_csv, install_plan_approval, og_name, og_spec
   - FBC operators include additional: fbc_iib_repo, ocp_operator_mirror_fbc_image_base
   - Metadata: cluster_name, capture_timestamp (ISO8601)

@openshift-ci openshift-ci Bot requested review from TimurMP and rdiscala June 12, 2026 15:20
@ccardenosa ccardenosa force-pushed the feat/ipa-telco-kpis-prow-migration-lockdown-hub-operators branch 7 times, most recently from cf81594 to 8666f71 Compare June 17, 2026 12:50
@openshift-ci openshift-ci Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 17, 2026
@ccardenosa

Copy link
Copy Markdown
Collaborator Author

/test images

@ccardenosa ccardenosa force-pushed the feat/ipa-telco-kpis-prow-migration-lockdown-hub-operators branch from 8666f71 to f8d9f2e Compare June 17, 2026 13:03
@openshift-ci openshift-ci Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 17, 2026
@ccardenosa ccardenosa force-pushed the feat/ipa-telco-kpis-prow-migration-lockdown-hub-operators branch 3 times, most recently from f72625c to 5fb0f9c Compare June 18, 2026 13:53
@ccardenosa ccardenosa changed the title feat(telco-kpis): Add hub operator lockdown mechanism for reproducible installations feat(telco-kpis): Implement unified lockdowns role with hub capture and parse Jun 22, 2026
@ccardenosa ccardenosa force-pushed the feat/ipa-telco-kpis-prow-migration-lockdown-hub-operators branch 6 times, most recently from 1800dfb to 6e34f8d Compare June 24, 2026 08:54
@openshift-ci

openshift-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign mrniranjan for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ccardenosa ccardenosa force-pushed the feat/ipa-telco-kpis-prow-migration-lockdown-hub-operators branch 3 times, most recently from a31559f to 18ebb46 Compare June 24, 2026 14:10
…ed iptables routing

Add spoke_hub_connectivity Ansible role to enable spoke clusters on corporate
network to reach hub clusters on isolated networks via hypervisor iptables DNAT
rules with source IP filtering.

Key Features:
- Three-tier traffic routing architecture:
  * BMC IP (10.6.x.x) → Hub cluster (172.16.x.x): metal3 ports (6183,6385,6180,6388)
  * Spoke cluster IP (10.6.x.x) → Hub cluster (172.16.x.x): HTTP/API (80,443,6443)
  * Spoke cluster IP (10.6.x.x) → Hub bastion (172.16.x.x): registry (5000)

- Dual source IP filtering:
  * Spoke BMC IP for ZTP deployment traffic
  * Spoke cluster node IP for post-deployment klusterlet traffic
  * Enables multiple spokes to use different hubs on same hypervisor

- Dynamic configuration discovery:
  * Queries hub cluster IP via oc command using hub kubeconfig on spoke bastion
  * Filters to IPv4 addresses only (no IPv6)
  * Loads hub bastion vault for disconnected registry IP
  * Extracts spoke BMC and cluster IPs from spoke vault

- Four operational modes:
  * setup: Create iptables/firewalld rules for spoke-hub pair
  * remove: Delete all rules (BMC, cluster, registry)
  * check-only: Validate specific spoke-hub pair configuration
  * list-interconnections: Display all spoke→hub mappings across hypervisor

- Persistence via firewalld:
  * iptables rules for runtime configuration
  * firewalld rich rules for persistence across reboots

Implementation Details:
- Role: playbooks/telco-kpis/roles/spoke_hub_connectivity/
- Playbook: playbooks/telco-kpis/setup-spoke-hub-connectivity.yml
- Templates: Jinja2 shell scripts for iptables/firewalld rules
- Vault-driven: All IPs and hostnames from vaults (no hardcoded values)

Fixes Applied During Implementation:
- IPv6 filtering: Only use IPv4 addresses for iptables rules
- Port routing: Split ports by target (cluster vs bastion registry)
- Complete removal: Remove all three rule sets (BMC, cluster, registry)

This enables ZTP deployment of spoke clusters that can reach hub clusters
through hypervisor port forwarding, supporting multi-hub environments on
the same hypervisor infrastructure.

Signed-off-by: Carlos Cardenosa <ccardeno@redhat.com>
@ccardenosa ccardenosa force-pushed the feat/ipa-telco-kpis-prow-migration-lockdown-hub-operators branch from 18ebb46 to 3c70ae2 Compare June 26, 2026 10:51
…nd parse

Implements a unified lockdowns role for hub/spoke operator lockdown management,
enabling repeatable OpenShift cluster deployments with locked operator versions.

Key Features:

1. Unified Role Structure (lockdowns)
   - Mode-based dispatcher (hub/spoke) with action (parse/capture)
   - Common utilities: download, validation, symlink generation
   - Separate task files for hub and spoke operations
   - Jinja2 templates for lockdown JSON generation

2. Hub Lockdown Capture
   - Queries cluster for OCP version, Subscriptions, OperatorGroups, CatalogSources
   - Detects architecture via skopeo (x86_64/arm64, multi-arch support)
   - Maps mirrored catalog names to upstream (cs-redhat-operator-index-* → redhat-operators)
   - Enriches FBC operators with mirroring metadata:
     * fbc_iib_repo: 'latest'
     * ocp_operator_mirror_fbc_image_base: quay.io/redhat-user-workloads/telco-5g-tenant/{catalog}-{version}
   - Generates dual-format OCP pull specs (digest + tag)
   - Outputs timestamped lockdown JSON with cluster metadata

3. Hub Lockdown Parse
   - Downloads lockdown JSON from URI (GitLab, Gitea, file://)
   - GitLab symlink resolver handles multi-hop symlink chains:
     * lockdown-hub-x86_64.json → lockdown-hub-4.21-x86_64.json → ../4.21/actual.json
     * Detects JSON vs symlink text, resolves relative paths
     * Max 5 hops protection
   - Uses slurp module (not lookup) for SSH compatibility (Ansible controller vs remote host)
   - Validates lockdown structure (required fields, nested hierarchy)
   - Transforms operators for upstream compatibility:
     * Adds 'nsname' field from 'namespace' (upstream role requirement)
   - Sets facts: hub_lockdown_operators, hub_ocp_pull_spec, hub_ocp_version

4. Telco-KPIs Wrapper Playbook (deploy-ocp-operators.yml)
   - Three-phase workflow:
     * Phase 1: Parse lockdown (if hub_lockdown_uri provided) OR use parameters
     * Phase 2: Call upstream deploy-ocp-operators.yml with transformed operators
     * Phase 3: Capture lockdown (if generate_hub_lockdown requested)
   - Integrates with Jenkins install-hub-operators job
   - Supports both lockdown mode and parameter mode

5. Lockdown JSON Format
   - Nested structure: {hub: {ocp: {...}, operators: [...], metadata: {...}}}
   - OCP pull_spec with both digest (immutable) and tag (human-readable)
   - Operators include: name, namespace, catalog, channel, subscription_name,
     installed_csv, install_plan_approval, og_name, og_spec
   - FBC operators include additional: fbc_iib_repo, ocp_operator_mirror_fbc_image_base
   - Metadata: cluster_name, capture_timestamp (ISO8601)

6. Testing
   - Molecule test suites for hub parse and capture
   - 10 test scenarios covering parse, validation, symlinks, comparisons
   - Fixtures for direct JSON, 1-hop/2-hop symlinks, invalid JSON

Implementation Details:
- Uses kubernetes.core.k8s_info for cluster queries (no k8s_exec)
- Hardcoded FBC metadata for relaxed repeatability (vs reading CatalogSource state)
- Directory existence checks before template writes
- Variable scoping fixes for lockdown_artifact_dir across playbook phases

Fixes:
- Jenkins builds openshift-kni#75-77, openshift-kni#79, openshift-kni#81, openshift-kni#84, openshift-kni#85 failures with hub lockdown URI
- GitLab symlink resolution (/-/raw/ endpoint returns text, not target)
- Ansible execution context (lookup vs slurp on remote hosts)
- FBC operator mirroring metadata requirements
- Field name mismatches (namespace vs nsname)

Files:
- playbooks/telco-kpis/deploy-ocp-operators.yml (new wrapper)
- playbooks/telco-kpis/roles/lockdowns/ (unified role)
- playbooks/telco-kpis/roles/lockdowns/tasks/{main,hub,spoke,common}
- playbooks/telco-kpis/roles/lockdowns/templates/{hub,spoke,resolve-gitlab-symlinks}
- playbooks/telco-kpis/roles/lockdowns/molecule/hub/default/ (tests)

Integration:
- Jenkins: jobs/Telco-KPIs/install-hub-operators.Jenkinsfile
- GitLab: ran/dev-kpi-pipeline/-/tree/prow-lockdowns/hub/
- Upstream: playbooks/deploy-ocp-operators.yml (imports this wrapper)

Related:
- Legacy branch: ipa-telco-kpis-prow-migration-20260619-before-deploy-ocp-operators-untouched
- Design docs: docs/designs/operator-lockdown-*.md
- README: playbooks/telco-kpis/roles/lockdowns/README.md
@ccardenosa ccardenosa force-pushed the feat/ipa-telco-kpis-prow-migration-lockdown-hub-operators branch from 3c70ae2 to e6bfd2c Compare June 26, 2026 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant