Skip to content

Refactor/ipa telco kpis prow migration ztp deployment timeline#463

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift-kni:mainfrom
ccardenosa:refactor/ipa-telco-kpis-prow-migration-ztp-deployment-timeline
Jun 24, 2026
Merged

Refactor/ipa telco kpis prow migration ztp deployment timeline#463
openshift-merge-bot[bot] merged 1 commit into
openshift-kni:mainfrom
ccardenosa:refactor/ipa-telco-kpis-prow-migration-ztp-deployment-timeline

Conversation

@ccardenosa

Copy link
Copy Markdown
Collaborator

No description provided.

@openshift-ci openshift-ci Bot requested review from cplacani and mkochanowski May 6, 2026 19:02
@openshift-ci openshift-ci Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 13, 2026
@ccardenosa ccardenosa force-pushed the refactor/ipa-telco-kpis-prow-migration-ztp-deployment-timeline branch from 32745c5 to 99ae072 Compare June 3, 2026 13:20
@openshift-ci openshift-ci Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 3, 2026
@ccardenosa ccardenosa force-pushed the refactor/ipa-telco-kpis-prow-migration-ztp-deployment-timeline branch 9 times, most recently from 23362ea to 7d51733 Compare June 15, 2026 15:34
@ccardenosa ccardenosa force-pushed the refactor/ipa-telco-kpis-prow-migration-ztp-deployment-timeline branch 5 times, most recently from 2d342e2 to a1c4be9 Compare June 23, 2026 07:45

@eifrach eifrach left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let me know what you think

it looks good - but managing J2 template will make it much easier to read / modify

Comment on lines +187 to +266
ZTP Deployment Timeline Summary
======================================================================
Hub Cluster: {{ hub_cluster | default('Unknown') }}
Bastion Host: {{ inventory_hostname }}
Spoke Cluster: {{ spoke_cluster }}

Deployment Features:
- ArgoCD Application Starting Point: {{ 'Present' if milestone_timestamps.argo_app != 'N/A' else 'Not Present' }}
- ClusterInstance Tracking: {{ 'Present' if milestone_timestamps.clusterinstance != 'N/A' else 'Not Present' }}
- TALM CGU Completion: {{ 'Present' if milestone_timestamps.talm_cgu_completed != 'N/A' else 'Not Present' }}
- ztp-done Label: {{ 'Present' if has_ztp_done_label | default(false) else 'Not Present' }}

Total Events Captured: {{ ztp_deployment_timeline_events | length }}

======================================================================
KEY MILESTONES
======================================================================
{% for line in milestone_table_lines %}
{{ line }}
{% endfor %}

======================================================================
WORKLOAD READINESS STATUS
======================================================================
{% if milestone_timestamps.talm_cgu_completed != 'N/A' %}
✅ Cluster ready for workloads since: {{ milestone_timestamps.talm_cgu_completed }} (since {{ time_ready_formatted }})
{% else %}
⚠️ Workload readiness not yet achieved
{% endif %}

======================================================================
MILESTONE BREAKDOWN
======================================================================
{% for group in milestone_breakdown %}

{{ group.milestone }} ({{ group.count }} events)
First: {{ group.first.timestamp }} - {{ group.first.event }}
Last: {{ group.last.timestamp }} - {{ group.last.event }}
{% endfor %}

======================================================================
FEATURE STATUS
======================================================================
1. ArgoCD Application Starting Point: {{ 'Present' if milestone_timestamps.argo_app != 'N/A' else 'Not Present' }}
{% if milestone_timestamps.argo_app != 'N/A' %}
✅ Captured GitOps deployment trigger (earliest possible start)
✅ Using ArgoCD Application creation as deployment start
{% else %}
ℹ️ ArgoCD Application not found (checked openshift-gitops and argocd namespaces)
ℹ️ Using ClusterInstance or ManagedCluster creation as starting point instead
{% endif %}

2. ClusterInstance Tracking: {{ 'Present' if milestone_timestamps.clusterinstance != 'N/A' else 'Not Present' }}
{% if milestone_timestamps.clusterinstance != 'N/A' %}
✅ Captured SiteConfig v2 operator reconciliation events
ℹ️ Available for SiteConfig v2 deployments
{% else %}
ℹ️ ClusterInstance resource not found (requires SiteConfig v2 operator)
ℹ️ Not critical - SiteConfig v1 deployments don't have ClusterInstance
{% endif %}

3. TALM CGU Completion: {{ 'Present' if milestone_timestamps.talm_cgu_completed != 'N/A' else 'Not Present' }}
{% if milestone_timestamps.talm_cgu_completed != 'N/A' %}
✅ Captured accurate policy completion timestamp (ready for workloads)
✅ Using TALM CGU completedAt as deployment success milestone
{% else %}
ℹ️ TALM ClusterGroupUpgrade not found in ztp-install namespace
ℹ️ Using individual policy compliance events as fallback
{% endif %}

======================================================================
DEPLOYMENT SUMMARY
======================================================================
{% if deployment_duration_formatted is defined %}
🚀 The deployment took {{ deployment_duration_formatted }} from ClusterInstance CR creation to TALM CGU Completed (Ready for Workloads)
{% else %}
⚠️ Unable to calculate deployment duration (missing key milestones)
{% endif %}

======================================================================

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This we will be easier to manage in a jinja2 template instead of here

Comment on lines +97 to +175
- name: Generate deployment timeline summary
ansible.builtin.copy:
content: |
======================================================================
ZTP AI DEPLOYMENT TIME TEST SUMMARY
======================================================================
Spoke Cluster: {{ spoke_cluster }}
Hub Cluster: {{ hub_cluster }}
Deployment Method: {{ ztp_deployment_timeline_deployment_method }}

DEPLOYMENT TIMELINE:
Start Time: {{ ztp_deployment_timeline_start_time }}
End Time: {{ ztp_deployment_timeline_end_time }}
Duration: {{ ztp_deployment_timeline_duration_formatted }} ({{ ztp_deployment_timeline_duration_seconds }} seconds)

TEST VALIDATION:
Threshold: {{ threshold_duration }} ({{ threshold_seconds }} seconds)
Result: {{ test_status }}
{{ 'PASS - Deployment completed within acceptable time' if test_passed else 'FAIL - Deployment exceeded time threshold' }}

{% if not test_passed %}
FAILURE DETAILS:
Actual deployment time ({{ ztp_deployment_timeline_duration_seconds }}s) exceeded threshold ({{ threshold_seconds }}s)
Difference: {{ ztp_deployment_timeline_duration_seconds | int - threshold_seconds | int }} seconds over limit
{% endif %}

{{ ztp_deployment_timeline_detailed_summary }}
dest: "{{ test_artifacts_dir.path }}/deployment-timeline-summary.txt"
mode: '0644'

- name: Generate JUnit XML test result
ansible.builtin.copy:
content: |
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="ZTP AI Deployment Time" tests="1"
failures="{{ 0 if test_passed else 1 }}" errors="0"
time="{{ ztp_deployment_timeline_duration_seconds }}">
<testcase name="ZTP AI Deployment Time Validation"
classname="telco.kpis.ztp.ai.deployment.time"
time="{{ ztp_deployment_timeline_duration_seconds }}">
{% if not test_passed %}
<failure type="TimeThresholdExceeded"
message="Deployment {{ ztp_deployment_timeline_duration_formatted }} exceeded {{ threshold_duration }} threshold">
Deployment Time Test Failed

Spoke Cluster: {{ spoke_cluster }}
Hub Cluster: {{ hub_cluster }}
Deployment Method: {{ ztp_deployment_timeline_deployment_method }}

Start Time: {{ ztp_deployment_timeline_start_time }}
End Time: {{ ztp_deployment_timeline_end_time }}

Actual Duration: {{ ztp_deployment_timeline_duration_formatted }} ({{ ztp_deployment_timeline_duration_seconds }} seconds)
Threshold: {{ threshold_duration }} ({{ threshold_seconds }} seconds)

Exceeded By: {{ ztp_deployment_timeline_duration_seconds | int - threshold_seconds | int }} seconds

The deployment took longer than the acceptable threshold for AI (Assisted Installer) deployments.
Expected: ≤ {{ threshold_seconds }}s
Actual: {{ ztp_deployment_timeline_duration_seconds }}s
</failure>
{% endif %}
<system-out>
Spoke Cluster: {{ spoke_cluster }}
Hub Cluster: {{ hub_cluster }}
Deployment Method: {{ ztp_deployment_timeline_deployment_method }}
Start Time: {{ ztp_deployment_timeline_start_time }}
End Time: {{ ztp_deployment_timeline_end_time }}
Duration: {{ ztp_deployment_timeline_duration_formatted }} ({{ ztp_deployment_timeline_duration_seconds }}s)
Threshold: {{ threshold_duration }} ({{ threshold_seconds }}s)
Result: {{ test_status }}
Events Captured: {{ ztp_deployment_timeline_events | length }}
</system-out>
</testcase>
</testsuite>
</testsuites>
dest: "{{ test_artifacts_dir.path }}/junit_ztp-ai-deployment-time.xml"
mode: '0644'

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think those can also move to a template

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Implement ZTP (Zero Touch Provisioning) Assisted Installer deployment time
validation test with comprehensive timeline tracking.

## Playbook

**playbooks/telco-kpis/ztp-ai-deployment-time.yml:**
- Validates ZTP deployment duration against threshold (default: 2h0m = 120 minutes)
- Measures from ClusterInstance creation to TALM ClusterGroupUpgrade completion
- Test result: PASS if duration ≤ threshold, FAIL otherwise

## Role

**playbooks/roles/ztp_deployment_timeline/**
- Queries ACM (Advanced Cluster Management) resources on hub cluster
- Uses kubernetes.core.k8s_info module (not bash scripts)
- Tracks deployment milestones: ClusterInstance, ManagedCluster, AgentClusterInstall, TALM CGU
- Supports both AI (Assisted Installer) and IBI (Image-based Install) methods
- Generates detailed milestone analysis with timestamps, durations, deltas
- Exports facts: ztp_deployment_timeline_duration_seconds, ztp_deployment_timeline_deployment_method

## Artifacts Generated

**Output directory:** `ztp-ai-deployment-time-{spoke}-{YYYYMMDD-HHMMSS}` (UTC timestamp)

Files:
- `deployment-timeline-summary.txt` - Human-readable summary with milestone breakdown
- `deployment-timeline.json` - Raw timeline events in JSON format
- `junit_ztp-ai-deployment-time.xml` - JUnit XML test result for CI/CD integration

## Implementation Details

**Timeline Milestones Tracked:**
1. ClusterInstance creation
2. ManagedCluster available
3. AgentClusterInstall completion
4. TALM ClusterGroupUpgrade completion

**Role Structure:**
- `tasks/main.yml` - Main timeline collection logic
- `tasks/query-resources.yml` - ACM resource queries
- `tasks/calculate-timeline.yml` - Duration calculations
- `tasks/generate-summary.yml` - Human-readable summary
- `defaults/main.yml` - Default variables (threshold, namespace)

**UTC Timestamp Consistency:**
All timestamps use UTC (`date -u +%Y%m%d-%H%M%S`) for correct chronological
ordering across different bastion timezones.

## Integration

- Artifacts copied to shared location: `/home/telcov10n/telco-kpis-artifacts/{spoke}/`
- Integrated into comprehensive reports before "Report Metadata" section
- Report generator filters based on node-info timestamp (freshness check)

Related: Telco-KPIs ZTP validation, deployment timeline tracking
Signed-off-by: Carlos Cardenosa <ccardeno@redhat.com>
@ccardenosa ccardenosa force-pushed the refactor/ipa-telco-kpis-prow-migration-ztp-deployment-timeline branch from a1c4be9 to cd874c1 Compare June 24, 2026 07:22
@eifrach

eifrach commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

/lgtm
/approve

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 24, 2026
@openshift-ci

openshift-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: eifrach

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 24, 2026
@openshift-merge-bot openshift-merge-bot Bot merged commit da06b24 into openshift-kni:main Jun 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants