Skip to content

Commit 2d342e2

Browse files
committed
feat(telco-kpis): Add ZTP AI deployment time test
Implement ZTP (Zero Touch Provisioning) Assisted Installer deployment time validation test with comprehensive timeline tracking. ## Playbook **playbooks/telco-kpis/ztp-ai-deployment-time.yml:** - Validates ZTP deployment duration against threshold (default: 2h0m = 120 minutes) - Measures from ClusterInstance creation to TALM ClusterGroupUpgrade completion - Test result: PASS if duration ≤ threshold, FAIL otherwise ## Role **playbooks/roles/ztp_deployment_timeline/** - Queries ACM (Advanced Cluster Management) resources on hub cluster - Uses kubernetes.core.k8s_info module (not bash scripts) - Tracks deployment milestones: ClusterInstance, ManagedCluster, AgentClusterInstall, TALM CGU - Supports both AI (Assisted Installer) and IBI (Image-based Install) methods - Generates detailed milestone analysis with timestamps, durations, deltas - Exports facts: ztp_deployment_timeline_duration_seconds, ztp_deployment_timeline_deployment_method ## Artifacts Generated **Output directory:** `ztp-ai-deployment-time-{spoke}-{YYYYMMDD-HHMMSS}` (UTC timestamp) Files: - `deployment-timeline-summary.txt` - Human-readable summary with milestone breakdown - `deployment-timeline.json` - Raw timeline events in JSON format - `junit_ztp-ai-deployment-time.xml` - JUnit XML test result for CI/CD integration ## Implementation Details **Timeline Milestones Tracked:** 1. ClusterInstance creation 2. ManagedCluster available 3. AgentClusterInstall completion 4. TALM ClusterGroupUpgrade completion **Role Structure:** - `tasks/main.yml` - Main timeline collection logic - `tasks/query-resources.yml` - ACM resource queries - `tasks/calculate-timeline.yml` - Duration calculations - `tasks/generate-summary.yml` - Human-readable summary - `defaults/main.yml` - Default variables (threshold, namespace) **UTC Timestamp Consistency:** All timestamps use UTC (`date -u +%Y%m%d-%H%M%S`) for correct chronological ordering across different bastion timezones. ## Integration - Artifacts copied to shared location: `/home/telcov10n/telco-kpis-artifacts/{spoke}/` - Integrated into comprehensive reports before "Report Metadata" section - Report generator filters based on node-info timestamp (freshness check) Related: Telco-KPIs ZTP validation, deployment timeline tracking Signed-off-by: Carlos Cardenosa <ccardeno@redhat.com>
1 parent 228a82a commit 2d342e2

7 files changed

Lines changed: 1202 additions & 0 deletions

File tree

Lines changed: 217 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,217 @@
1+
# ztp_deployment_timeline
2+
3+
Ansible role to collect Zero Touch Provisioning (ZTP) deployment timeline data from OpenShift hub clusters using Advanced Cluster Management (ACM).
4+
5+
## Description
6+
7+
This role queries ACM/ZTP resources on a hub cluster to build a complete deployment timeline from ClusterInstance creation to TALM ClusterGroupUpgrade completion. It provides detailed insights into every phase of the deployment process, including GitOps sync, installation, discovery, provisioning, import, and policy application.
8+
9+
The role sets facts with deployment timeline information that can be used by test playbooks or other automation to validate deployment performance, generate reports, or troubleshoot failed deployments.
10+
11+
## Requirements
12+
13+
- Hub cluster with ACM/ZTP deployed
14+
- `kubernetes.core` collection installed
15+
- Valid kubeconfig for hub cluster access
16+
- Spoke cluster deployed via ZTP (AgentClusterInstall or ImageBasedInstall)
17+
18+
## Role Variables
19+
20+
### Required Variables
21+
22+
- `spoke_cluster`: Name of the spoke cluster (ManagedCluster name)
23+
- `hub_kubeconfig`: Path to hub cluster kubeconfig file on the bastion
24+
25+
### Optional Variables
26+
27+
None. The role automatically detects deployment method (AI vs IBI) and available resources.
28+
29+
## Facts Set by Role
30+
31+
After execution, the role sets the following facts:
32+
33+
### Timeline Facts
34+
35+
- `ztp_deployment_timeline_start_time`: ISO 8601 timestamp of deployment start (ClusterInstance creation)
36+
- `ztp_deployment_timeline_end_time`: ISO 8601 timestamp of deployment completion (TALM CGU completion)
37+
- `ztp_deployment_timeline_duration_seconds`: Total deployment time in seconds
38+
- `ztp_deployment_timeline_duration_formatted`: Human-readable duration (e.g., "1h21m01s")
39+
40+
### Event Data
41+
42+
- `ztp_deployment_timeline_events`: List of timeline events, each containing:
43+
- `timestamp`: ISO 8601 timestamp
44+
- `event`: Event type identifier
45+
- `event_description`: Human-readable event description
46+
- `milestone`: Deployment phase (0-GITOPS_APPLICATION, 1-GITOPS_SYNC, etc.)
47+
48+
### Status Facts
49+
50+
- `ztp_deployment_timeline_success`: Boolean indicating if timeline data was successfully collected
51+
- `ztp_deployment_timeline_error`: Error message if collection failed
52+
- `ztp_deployment_timeline_deployment_method`: "AI" (Assisted Installer) or "IBI" (Image-based Install)
53+
54+
### Milestone Timestamps
55+
56+
- `ztp_deployment_timeline_clusterinstance_created`: ClusterInstance creation timestamp
57+
- `ztp_deployment_timeline_gitops_sync`: ManagedCluster creation timestamp
58+
- `ztp_deployment_timeline_cluster_install_created`: AgentClusterInstall/ImageBasedInstall creation timestamp
59+
- `ztp_deployment_timeline_iso_ready`: Discovery ISO ready timestamp (AI only)
60+
- `ztp_deployment_timeline_agent_registered`: Agent registration timestamp (AI only)
61+
- `ztp_deployment_timeline_installation_started`: Installation start timestamp
62+
- `ztp_deployment_timeline_installation_completed`: Installation completion timestamp
63+
- `ztp_deployment_timeline_cluster_available`: Cluster available in ACM timestamp
64+
- `ztp_deployment_timeline_talm_cgu_completed`: TALM CGU completion timestamp
65+
66+
## Example Playbook
67+
68+
### Basic Usage
69+
70+
```yaml
71+
---
72+
- name: Collect ZTP deployment timeline
73+
hosts: bastion
74+
gather_facts: false
75+
76+
tasks:
77+
- name: Collect deployment timeline for spoke cluster
78+
ansible.builtin.include_role:
79+
name: ztp_deployment_timeline
80+
vars:
81+
spoke_cluster: "spree-02"
82+
hub_kubeconfig: "/home/telcov10n/project/generated/kni-qe-71/auth/kubeconfig"
83+
84+
- name: Display timeline summary
85+
ansible.builtin.debug:
86+
msg:
87+
- "Deployment Method: {{ ztp_deployment_timeline_deployment_method }}"
88+
- "Start Time: {{ ztp_deployment_timeline_start_time }}"
89+
- "End Time: {{ ztp_deployment_timeline_end_time }}"
90+
- "Duration: {{ ztp_deployment_timeline_duration_formatted }}"
91+
- "Total Events: {{ ztp_deployment_timeline_events | length }}"
92+
```
93+
94+
### Deployment Time Validation
95+
96+
```yaml
97+
---
98+
- name: Validate ZTP deployment time
99+
hosts: bastion
100+
gather_facts: false
101+
102+
vars:
103+
threshold_duration: "2h0m" # 120 minutes for AI deployments
104+
105+
tasks:
106+
- name: Collect deployment timeline
107+
ansible.builtin.include_role:
108+
name: ztp_deployment_timeline
109+
vars:
110+
spoke_cluster: "{{ SPOKE_CLUSTER }}"
111+
hub_kubeconfig: "{{ HUB_KUBECONFIG }}"
112+
113+
- name: Parse threshold duration
114+
ansible.builtin.set_fact:
115+
threshold_seconds: "{{ (threshold_duration | regex_search('(\\d+)h') | default('0h') | regex_replace('h', '') | int * 3600) + (threshold_duration | regex_search('(\\d+)m') | default('0m') | regex_replace('m', '') | int * 60) }}"
116+
117+
- name: Validate deployment time
118+
ansible.builtin.assert:
119+
that:
120+
- ztp_deployment_timeline_success | bool
121+
- ztp_deployment_timeline_duration_seconds <= (threshold_seconds | int)
122+
fail_msg: |
123+
Deployment time exceeded threshold!
124+
Actual: {{ ztp_deployment_timeline_duration_formatted }} ({{ ztp_deployment_timeline_duration_seconds }}s)
125+
Threshold: {{ threshold_duration }} ({{ threshold_seconds }}s)
126+
success_msg: |
127+
Deployment completed within threshold
128+
Actual: {{ ztp_deployment_timeline_duration_formatted }} ({{ ztp_deployment_timeline_duration_seconds }}s)
129+
Threshold: {{ threshold_duration }} ({{ threshold_seconds }}s)
130+
```
131+
132+
### Generate Timeline Report
133+
134+
```yaml
135+
---
136+
- name: Generate deployment timeline report
137+
hosts: bastion
138+
gather_facts: false
139+
140+
tasks:
141+
- name: Collect deployment timeline
142+
ansible.builtin.include_role:
143+
name: ztp_deployment_timeline
144+
vars:
145+
spoke_cluster: "{{ SPOKE_CLUSTER }}"
146+
hub_kubeconfig: "{{ HUB_KUBECONFIG }}"
147+
148+
- name: Generate JSON timeline
149+
ansible.builtin.copy:
150+
content: "{{ ztp_deployment_timeline_events | to_nice_json }}"
151+
dest: "/tmp/{{ spoke_cluster }}-timeline.json"
152+
153+
- name: Generate summary report
154+
ansible.builtin.template:
155+
src: timeline-summary.j2
156+
dest: "/tmp/{{ spoke_cluster }}-timeline-summary.txt"
157+
```
158+
159+
## Deployment Methods Supported
160+
161+
### Assisted Installer (AI)
162+
163+
Traditional ZTP deployment using AgentClusterInstall:
164+
- Discovery ISO generation
165+
- Agent registration and binding
166+
- Assisted Service installation
167+
- Typical duration: 60-120 minutes
168+
169+
Measurement: `ClusterInstance.metadata.creationTimestamp` → `ClusterGroupUpgrade.status.status.completedAt`
170+
171+
### Image-Based Install (IBI)
172+
173+
Fast deployment using pre-built images with ImageBasedInstall:
174+
- Pre-configured cluster image
175+
- Direct installation without discovery phase
176+
- Typical duration: 15-30 minutes
177+
178+
Measurement: `ClusterInstance.metadata.creationTimestamp` → `ClusterGroupUpgrade.status.status.completedAt`
179+
180+
## Timeline Phases
181+
182+
The role categorizes events into deployment milestones:
183+
184+
- **0-GITOPS_APPLICATION**: ClusterInstance creation (SiteConfig v2 operator)
185+
- **1-GITOPS_SYNC**: ManagedCluster creation
186+
- **2-CLUSTER_INSTALL**: OpenShift installation process
187+
- **3-DISCOVERY**: ISO creation, agent registration (AI only)
188+
- **4-PROVISIONING**: BareMetalHost hardware provisioning
189+
- **6-IMPORT**: Initial ACM import
190+
- **7-MANIFESTWORK**: ACM addon deployments
191+
- **8-POLICY**: Policy application and compliance
192+
- **9-TALM_CGU_COMPLETION**: TALM recognizes all policies compliant
193+
- **10-ZTP_DONE**: ztp-done label present
194+
195+
## Error Handling
196+
197+
If timeline collection fails, the role sets:
198+
- `ztp_deployment_timeline_success: false`
199+
- `ztp_deployment_timeline_error`: Error message describing the failure
200+
201+
Common failure scenarios:
202+
- Hub cluster not accessible
203+
- Spoke cluster not found
204+
- Missing ClusterInstance resource (not a ZTP deployment)
205+
- Missing TALM ClusterGroupUpgrade (policies not applied)
206+
207+
## Dependencies
208+
209+
- `kubernetes.core` collection (for k8s_info module)
210+
211+
## License
212+
213+
Apache License 2.0
214+
215+
## Author Information
216+
217+
Red Hat Telco Verification Team
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
# No default variables - all required variables must be provided when including the role
3+
#
4+
# Required variables:
5+
# - spoke_cluster: Name of the spoke cluster (ManagedCluster name)
6+
# - hub_kubeconfig: Path to hub cluster kubeconfig file on the bastion
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
---
2+
galaxy_info:
3+
author: Red Hat Telco Verification Team
4+
description: Collect ZTP deployment timeline data from ACM hub clusters
5+
company: Red Hat
6+
license: Apache-2.0
7+
min_ansible_version: "2.9"
8+
platforms:
9+
- name: EL
10+
versions:
11+
- "8"
12+
- "9"
13+
galaxy_tags:
14+
- openshift
15+
- ztp
16+
- acm
17+
- telco
18+
- deployment
19+
- timeline
20+
21+
dependencies: []
22+
23+
# Requirements:
24+
# This role requires the kubernetes.core collection to be installed
25+
# Install with: ansible-galaxy collection install kubernetes.core

0 commit comments

Comments
 (0)