feat(telco-kpis): Add standalone lockdown JSON parser playbook#455
Open
ccardenosa wants to merge 2 commits intoopenshift-kni:mainfrom
Open
feat(telco-kpis): Add standalone lockdown JSON parser playbook#455ccardenosa wants to merge 2 commits intoopenshift-kni:mainfrom
ccardenosa wants to merge 2 commits intoopenshift-kni:mainfrom
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Implement role-based container image mirroring system for internal registry
management, supporting both mirror and removal operations with authentication.
## Problem
Telco-KPIs testing requires mirroring container images to internal registries
for disconnected environments and test image management. Previous approach used
inline playbook tasks without reusability.
## Solution
Created dedicated `container_image_mirror` Ansible role with playbooks for both
mirroring and removal operations:
**Role: playbooks/roles/container_image_mirror/**
- Supports 'mirror' and 'remove' operations via parameter
- Uses skopeo for image operations
- Handles authentication with pull secrets
- Continues operation even if some images fail
- Comprehensive success/failure reporting with summary
**Playbooks:**
- `playbooks/mirror-images.yml` - Mirror images to internal registry
- `playbooks/remove-images.yml` - Remove images from registry storage
## Features
**Authentication:**
- Pull secret support for private registries (via pull_secret_string or pull_secret_path)
- System default auth when no pull secret provided
- Configurable auth file location (/tmp for bastion compatibility)
- use_pull_secret flag to control authentication method
**Registry Configuration:**
- Configurable registry host/port/namespace
- TLS verification control
- Source and destination registry support
**Operations:**
- Idempotent with existence checks
- Detailed mirror/removal summary
- Error handling continues operation on failures
## Usage
**Mirror images:**
```bash
ansible-playbook playbooks/mirror-images.yml \
-e images='[{"source": "quay.io/image:tag", "dest": "registry.local/namespace/image:tag"}]' \
-e registry_host=registry.local \
-e pull_secret_string='{"auths": {...}}'
```
**Remove images:**
```bash
ansible-playbook playbooks/remove-images.yml \
-e images='[{"dest": "registry.local/namespace/image:tag"}]' \
-e registry_host=registry.local
```
## Implementation Details
**Role Structure:**
- `defaults/main.yaml` - Default variables
- `tasks/main.yaml` - Entry point with operation dispatch
- `tasks/mirror.yaml` - Mirror images using skopeo
- `tasks/remove.yaml` - Remove images from registry storage
- `meta/main.yaml` - Role metadata
- `README.md` - Comprehensive documentation
**Key Variables:**
- `container_image_mirror_operation`: "mirror" or "remove"
- `container_image_mirror_images`: List of image objects
- `container_image_mirror_registry_host`: Target registry hostname
- `container_image_mirror_pull_secret_string`: JSON pull secret
- `container_image_mirror_use_pull_secret`: Enable/disable authentication
## Benefits
- Reusable role for both mirror and removal operations
- Cleaner separation of concerns
- Easier to test and maintain
- Follows eco-ci-cd role patterns (like ocp_operator_mirror)
- Well-documented with examples
- Jenkins job compatible (uses same variable names)
## Jenkins Integration
Used by `telco-kpis-mirror-ran-test-images` Jenkins job for mirroring RAN test
images to internal registries.
Related: Telco-KPIs test infrastructure
Signed-off-by: Carlos Cardenosa <ccardeno@redhat.com>
Add parse-lockdown.yml playbook that extracts deployment parameters from
lockdown JSON files, enabling decoupled parameter management for reproducible
deployments.
## Problem
Telco-KPIs testing requires exact software versions (OCP releases, operator
channels, catalogs) for reproducible deployments. Parsing lockdown JSON inline
within deployment playbooks creates tight coupling and makes parameter reuse
across multiple jobs difficult.
## Solution
Implement standalone parser playbook that runs before deployment jobs:
1. Downloads and parses lockdown JSON from URI
2. Auto-detects lockdown type (hub vs spoke) from JSON structure
3. Extracts deployment parameters
4. Outputs in shell env and JSON formats for downstream consumption
## Changes
**New playbook: playbooks/telco-kpis/parse-lockdown.yml**
- Auto-detection logic: checks for 'hub' vs 'deployment' key in JSON
- Hub parsing: extracts OCP_RELEASE_IMAGE, ACM_CHANNEL, MCE_CHANNEL, catalogs
- Spoke parsing: extracts OCP_PULL_SPEC, ZTP_PULL_SPEC, operator configurations
- SSL certificate bypass for internal GitLab instances (validate_certs: false)
- Dynamic artifact naming using lockdown filename from URI
- Outputs three artifacts per run:
- `{lockdown-name}.json`: Original lockdown file
- `{lockdown-name}-params.env`: Shell environment variables
- `{lockdown-name}-params.json`: Structured JSON parameters
**New role: playbooks/telco-kpis/roles/lockdown_hub_config/**
- tasks/main.yml: Download, validate, and parse hub lockdown JSON
- defaults/main.yml: Default configuration values
- README.md: Comprehensive role documentation
- Used by both parse-lockdown.yml and deploy-ocp-operators.yml
## Usage Workflow
**Step 1: Parse lockdown file**
```bash
ansible-playbook playbooks/telco-kpis/parse-lockdown.yml \
-e lockdown_uri=https://gitlab.cee.redhat.com/.../lockdown-hub-x86_64.json
```
**Step 2: Use extracted parameters**
```bash
# Source env file
source lockdown-hub-x86_64-params.env
# Use in deployment
ansible-playbook playbooks/deploy-ocp-sno.yml \
-e release="${OCP_RELEASE_IMAGE}"
```
## Benefits
**Decoupling:**
- Parsing separated from deployment logic
- Parameters extracted once, reused across multiple jobs
- Easier debugging with explicit parameter artifacts
**Flexibility:**
- Supports multiple lockdown formats (hub, spoke, baseline)
- Self-documenting artifacts with actual lockdown names
- Both shell and JSON output formats
**Prow-ready:**
- Clean separation aligns with Prow step registry architecture
- Parser step can run independently, output shared via SHARED_DIR
## Key Features
**Auto-detection:**
```yaml
lockdown_type: "{{ 'hub' if ('hub' in lockdown_data) else 'spoke' }}"
```
**Dynamic artifact naming:**
```yaml
lockdown_filename: "{{ lockdown_uri | regex_replace('.*/', '') | regex_replace('.json$', '') }}"
# Result: lockdown-hub-x86_64-params.env (not generic lockdown-params.env)
```
**Hub channel transformations:**
```yaml
hub_acm_channel: "release-{{ lockdown_data.hub.acm.version_override }}"
hub_mce_channel: "{{ lockdown_data.hub.acm.mce_override | regex_replace('^v', 'stable-') | regex_replace('\\.\\d+$', '') }}"
```
## Example Artifacts
**lockdown-hub-x86_64-params.env:**
```bash
LOCKDOWN_TYPE=hub
OCP_RELEASE_IMAGE=quay.io/openshift-release-dev/ocp-release:4.20.4-x86_64
OCP_VERSION=4.20
ACM_CHANNEL=release-2.13
MCE_CHANNEL=stable-2.8
TALM_CATALOG=quay.io/.../talm-index:v4.20
GITOPS_CATALOG=quay.io/.../gitops-index:v1.15
```
**lockdown-hub-x86_64-params.json:**
```json
{
"LOCKDOWN_TYPE": "hub",
"OCP_RELEASE_IMAGE": "quay.io/openshift-release-dev/ocp-release:4.20.4-x86_64",
"OCP_VERSION": "4.20",
"ACM_CHANNEL": "release-2.13",
"MCE_CHANNEL": "stable-2.8",
"TALM_CATALOG": "quay.io/.../talm-index:v4.20",
"GITOPS_CATALOG": "quay.io/.../gitops-index:v1.15"
}
```
## Verification
Tested with both lockdown types:
- Hub lockdown: Successfully extracted OCP 4.20.4 pull spec and operator channels
- Spoke lockdown: Successfully extracted spoke deployment parameters
Artifacts correctly named with lockdown filename and contain expected parameters.
Related: Telco-KPIs reproducible deployment system
Signed-off-by: Carlos Cardenosa <ccardeno@redhat.com>
ccardenosa
force-pushed
the
refactor/ipa-telco-kpis-prow-migration-lockdown-parser
branch
from
e880769 to
507eed0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add parse-lockdown.yml playbook that extracts deployment parameters from lockdown JSON files, enabling decoupled parameter management for reproducible deployments.
Problem
Telco-KPIs testing requires exact software versions (OCP releases, operator channels, catalogs) for reproducible deployments. Parsing lockdown JSON inline within deployment playbooks creates tight coupling and makes parameter reuse across multiple jobs difficult.
Solution
Implement standalone parser playbook that runs before deployment jobs:
Changes
New playbook: playbooks/telco-kpis/parse-lockdown.yml
{lockdown-name}.json: Original lockdown file{lockdown-name}-params.env: Shell environment variables{lockdown-name}-params.json: Structured JSON parametersNew role: playbooks/telco-kpis/roles/lockdown_hub_config/
Usage Workflow
Step 1: Parse lockdown file
Step 2: Use extracted parameters
Benefits
Decoupling:
Flexibility:
Prow-ready:
Key Features
Auto-detection:
Dynamic artifact naming:
Hub channel transformations:
Example Artifacts
lockdown-hub-x86_64-params.env:
lockdown-hub-x86_64-params.json:
{ "LOCKDOWN_TYPE": "hub", "OCP_RELEASE_IMAGE": "quay.io/openshift-release-dev/ocp-release:4.20.4-x86_64", "OCP_VERSION": "4.20", "ACM_CHANNEL": "release-2.13", "MCE_CHANNEL": "stable-2.8", "TALM_CATALOG": "quay.io/.../talm-index:v4.20", "GITOPS_CATALOG": "quay.io/.../gitops-index:v1.15" }Verification
Tested with both lockdown types:
Artifacts correctly named with lockdown filename and contain expected parameters.
Related: Telco-KPIs reproducible deployment system