Skip to content

Commit 45ef2c2

Browse files
committed
feat(infra): Add container image mirroring infrastructure
Implement role-based container image mirroring system for internal registry management, supporting both mirror and removal operations with authentication. ## Problem Telco-KPIs testing requires mirroring container images to internal registries for disconnected environments and test image management. Previous approach used inline playbook tasks without reusability. ## Solution Created dedicated `container_image_mirror` Ansible role with playbooks for both mirroring and removal operations: **Role: playbooks/roles/container_image_mirror/** - Supports 'mirror' and 'remove' operations via parameter - Uses skopeo for image operations - Handles authentication with pull secrets - Continues operation even if some images fail - Comprehensive success/failure reporting with summary **Playbooks:** - `playbooks/mirror-images.yml` - Mirror images to internal registry - `playbooks/remove-images.yml` - Remove images from registry storage ## Features **Authentication:** - Pull secret support for private registries (via pull_secret_string or pull_secret_path) - System default auth when no pull secret provided - Configurable auth file location (/tmp for bastion compatibility) - use_pull_secret flag to control authentication method **Registry Configuration:** - Configurable registry host/port/namespace - TLS verification control - Source and destination registry support **Operations:** - Idempotent with existence checks - Detailed mirror/removal summary - Error handling continues operation on failures ## Usage **Mirror images:** ```bash ansible-playbook playbooks/mirror-images.yml \ -e images='[{"source": "quay.io/image:tag", "dest": "registry.local/namespace/image:tag"}]' \ -e registry_host=registry.local \ -e pull_secret_string='{"auths": {...}}' ``` **Remove images:** ```bash ansible-playbook playbooks/remove-images.yml \ -e images='[{"dest": "registry.local/namespace/image:tag"}]' \ -e registry_host=registry.local ``` ## Implementation Details **Role Structure:** - `defaults/main.yaml` - Default variables - `tasks/main.yaml` - Entry point with operation dispatch - `tasks/mirror.yaml` - Mirror images using skopeo - `tasks/remove.yaml` - Remove images from registry storage - `meta/main.yaml` - Role metadata - `README.md` - Comprehensive documentation **Key Variables:** - `container_image_mirror_operation`: "mirror" or "remove" - `container_image_mirror_images`: List of image objects - `container_image_mirror_registry_host`: Target registry hostname - `container_image_mirror_pull_secret_string`: JSON pull secret - `container_image_mirror_use_pull_secret`: Enable/disable authentication ## Benefits - Reusable role for both mirror and removal operations - Cleaner separation of concerns - Easier to test and maintain - Follows eco-ci-cd role patterns (like ocp_operator_mirror) - Well-documented with examples - Jenkins job compatible (uses same variable names) ## Jenkins Integration Used by `telco-kpis-mirror-ran-test-images` Jenkins job for mirroring RAN test images to internal registries. Related: Telco-KPIs test infrastructure Signed-off-by: Carlos Cardenosa <ccardeno@redhat.com>
1 parent e3b9acf commit 45ef2c2

8 files changed

Lines changed: 586 additions & 0 deletions

File tree

playbooks/mirror-images.yml

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
---
2+
# Mirror container images to internal registry
3+
#
4+
# This playbook uses the container_image_mirror role to mirror images from
5+
# source registries to a target internal registry.
6+
#
7+
# Prerequisites:
8+
# - skopeo CLI tool installed on bastion
9+
# - Authentication to source registries (via pull_secret or existing auth)
10+
# - Target registry reachable
11+
#
12+
# Usage:
13+
# ansible-playbook playbooks/mirror-images.yml \
14+
# -i inventories/ocp-deployment/build-inventory.py \
15+
# --extra-vars "images='[{\"source\":\"quay.io/org/image:tag\",\"dest\":\"namespace/image:tag\"}]'"
16+
#
17+
# # With pull secret
18+
# ansible-playbook playbooks/mirror-images.yml \
19+
# -i inventories/ocp-deployment/build-inventory.py \
20+
# --extra-vars "images='[...]' pull_secret_string=$(cat ~/.docker/config.json | base64 -w0)"
21+
#
22+
# Required variables:
23+
# images: List of image mappings with 'source' and 'dest' keys
24+
# Example: [{"source": "quay.io/org/image:tag", "dest": "namespace/image:tag"}]
25+
#
26+
# Optional variables:
27+
# registry_host: Target registry hostname (default: bastion FQDN)
28+
# registry_port: Target registry port (default: 5000)
29+
# registry_namespace: Namespace prefix for all dest images (default: none)
30+
# dest_tls_verify: Verify TLS for dest registry (default: false)
31+
# use_pull_secret: Enable pull secret authentication (default: false)
32+
# pull_secret_string: Base64-encoded pull secret (only used if use_pull_secret=true)
33+
# pull_secret_path: Path to write pull secret file (only used if use_pull_secret=true)
34+
#
35+
- name: Mirror container images to internal registry
36+
hosts: bastion
37+
gather_facts: true
38+
roles:
39+
- role: container_image_mirror
40+
vars:
41+
container_image_mirror_operation: mirror
42+
container_image_mirror_images: "{{ images }}"
43+
container_image_mirror_registry_host: "{{ registry_host | default(ansible_fqdn) }}"
44+
container_image_mirror_registry_port: "{{ registry_port | default(5000) }}"
45+
container_image_mirror_registry_namespace: "{{ registry_namespace | default('') }}"
46+
container_image_mirror_dest_tls_verify: "{{ dest_tls_verify | default(false) }}"
47+
container_image_mirror_use_pull_secret: "{{ use_pull_secret | default(false) }}"
48+
container_image_mirror_pull_secret_string: "{{ pull_secret_string | default('') }}"
49+
container_image_mirror_pull_secret_path: "{{ pull_secret_path | default('/tmp/.pull-secret-mirror.json') }}"

playbooks/remove-images.yml

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
# Remove container images from internal registry
3+
#
4+
# This playbook uses the container_image_mirror role to remove images from
5+
# an internal registry's storage.
6+
#
7+
# Prerequisites:
8+
# - sudo access to registry storage path on bastion
9+
# - Target registry reachable
10+
#
11+
# Usage:
12+
# ansible-playbook playbooks/remove-images.yml \
13+
# -i inventories/ocp-deployment/build-inventory.py \
14+
# --extra-vars "images='[{\"dest\":\"namespace/image:tag\"}]'"
15+
#
16+
# # Remove multiple images
17+
# ansible-playbook playbooks/remove-images.yml \
18+
# -i inventories/ocp-deployment/build-inventory.py \
19+
# --extra-vars 'images=[{"dest":"ran-test/old-image:v1"},{"dest":"ran-test/deprecated:latest"}]'
20+
#
21+
# Required variables:
22+
# images: List of image mappings with 'dest' key
23+
# Example: [{"dest": "namespace/image:tag"}]
24+
#
25+
# Optional variables:
26+
# registry_namespace: Namespace prefix for all dest images (default: none)
27+
# registry_data_path: Registry storage path (default: /home/kni/registry/data/docker/registry/v2/repositories)
28+
#
29+
- name: Remove container images from internal registry
30+
hosts: bastion
31+
gather_facts: true
32+
roles:
33+
- role: container_image_mirror
34+
vars:
35+
container_image_mirror_operation: remove
36+
container_image_mirror_images: "{{ images }}"
37+
container_image_mirror_registry_namespace: "{{ registry_namespace | default('') }}"
38+
container_image_mirror_registry_data_path: "{{ registry_data_path | default('/home/kni/registry/data/docker/registry/v2/repositories') }}"
Lines changed: 309 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,309 @@
1+
# Container Image Mirror Ansible Role
2+
3+
## Disclaimer
4+
This role is provided as-is, without guarantees of support or maintenance.
5+
Use at your own discretion.
6+
7+
## Overview
8+
The `container_image_mirror` role provides generic container image mirroring and removal capabilities for internal registries. It uses `skopeo` to:
9+
- **Mirror** container images from source registries to a target internal registry
10+
- **Remove** container images from an internal registry
11+
12+
This role is designed to work with any container registry and supports both connected and disconnected (air-gapped) environments.
13+
14+
## Features
15+
- Generic image mirroring using `skopeo copy`
16+
- Image removal from registry storage
17+
- Pull secret support for private registries
18+
- Configurable TLS verification
19+
- Detailed success/failure reporting with summary
20+
- Idempotent operations with existence checks
21+
- Continues mirroring all images even if some fail
22+
- Per-image success/failure tracking
23+
24+
## Requirements
25+
- Ansible 2.9+
26+
- Tools available on the target host:
27+
- `skopeo` (for mirroring operations)
28+
- For removal operations:
29+
- `sudo` access to registry storage path
30+
- Target registry must be reachable
31+
- Pull secrets for source registries (if private)
32+
33+
## Role Variables
34+
35+
### Required Variables
36+
37+
- **`container_image_mirror_images`** (list): List of image mappings
38+
- For mirror operation: `[{"source": "quay.io/org/image:tag", "dest": "namespace/image:tag"}]`
39+
- For remove operation: `[{"dest": "namespace/image:tag"}]`
40+
- No default (must be provided)
41+
42+
### Optional Variables
43+
44+
- **`container_image_mirror_operation`** (string): Operation mode
45+
- Values: `mirror` or `remove`
46+
- Default: `mirror`
47+
48+
- **`container_image_mirror_registry_host`** (string): Target registry hostname
49+
- Default: `{{ ansible_fqdn }}`
50+
51+
- **`container_image_mirror_registry_port`** (int): Target registry port
52+
- Default: `5000`
53+
54+
- **`container_image_mirror_registry_namespace`** (string): Namespace prefix for destination images
55+
- Default: `""` (empty, no prefix)
56+
- Example: `ran-test/` would prefix all destination images
57+
58+
- **`container_image_mirror_dest_tls_verify`** (bool): Verify TLS for destination registry
59+
- Default: `false`
60+
61+
- **`container_image_mirror_use_pull_secret`** (bool): Enable pull secret file authentication
62+
- Default: `false`
63+
- When `false`: skopeo uses system authentication from `/etc/containers/auth.json`
64+
- When `true`: skopeo uses pull secret file via `--authfile` parameter
65+
66+
- **`container_image_mirror_pull_secret_string`** (string): Base64-encoded pull secret JSON
67+
- Default: `""` (empty, uses existing auth)
68+
- Format: Base64-encoded Docker config JSON
69+
- Only used when `container_image_mirror_use_pull_secret=true`
70+
71+
- **`container_image_mirror_pull_secret_path`** (string): Path to store pull secret
72+
- Default: `/tmp/.pull-secret-mirror.json`
73+
- Only used when `container_image_mirror_use_pull_secret=true`
74+
75+
- **`container_image_mirror_registry_data_path`** (string): Registry storage path (for removal)
76+
- Default: `/home/kni/registry/data/docker/registry/v2/repositories`
77+
78+
## Dependencies
79+
None.
80+
81+
## Authentication Methods
82+
83+
The role supports two authentication methods:
84+
85+
### System Authentication (Recommended)
86+
87+
When `use_pull_secret=false` (default), skopeo uses system authentication from `/etc/containers/auth.json`. This is the recommended approach when:
88+
- Source registries are public
89+
- Authentication has been configured beforehand (e.g., via `podman login` or `authWithQuay()` in Jenkins)
90+
- Both source and destination credentials are in the system auth file
91+
92+
### Pull Secret File Authentication
93+
94+
When `use_pull_secret=true`, skopeo uses a dedicated pull secret file via the `--authfile` parameter. Use this when:
95+
- You need to use specific credentials different from system auth
96+
- Working with private registries requiring explicit authentication
97+
- Credentials should not persist in system auth
98+
99+
**Important**: When using pull secrets, the content is never logged and the file is automatically cleaned up after use.
100+
101+
## Example Playbooks
102+
103+
### Mirror Images with System Authentication (Recommended)
104+
105+
```yaml
106+
---
107+
- name: Mirror RAN test images to internal registry
108+
hosts: bastion
109+
gather_facts: true
110+
roles:
111+
- role: container_image_mirror
112+
vars:
113+
container_image_mirror_operation: mirror
114+
container_image_mirror_registry_host: disconnected.registry.local
115+
container_image_mirror_registry_port: 5000
116+
container_image_mirror_registry_namespace: ran-test/
117+
container_image_mirror_images:
118+
- source: quay.io/telcov10n-ci/oslat:latest
119+
dest: oslat:latest
120+
- source: quay.io/telcov10n-ci/cyclictest:latest
121+
dest: cyclictest:latest
122+
- source: quay.io/telcov10n-ci/cnf-tests:4.8
123+
dest: cnf-tests:4.8
124+
```
125+
126+
### Mirror Images with Pull Secret
127+
128+
```yaml
129+
---
130+
- name: Mirror private images to internal registry
131+
hosts: bastion
132+
gather_facts: true
133+
roles:
134+
- role: container_image_mirror
135+
vars:
136+
container_image_mirror_operation: mirror
137+
container_image_mirror_registry_host: registry.example.com
138+
container_image_mirror_use_pull_secret: true
139+
container_image_mirror_pull_secret_string: "{{ pull_secret }}"
140+
container_image_mirror_images:
141+
- source: quay.io/private-org/image:tag
142+
dest: namespace/image:tag
143+
```
144+
145+
### Remove Images Example
146+
147+
```yaml
148+
---
149+
- name: Remove old RAN test images from internal registry
150+
hosts: bastion
151+
gather_facts: true
152+
roles:
153+
- role: container_image_mirror
154+
vars:
155+
container_image_mirror_operation: remove
156+
container_image_mirror_registry_namespace: ran-test/
157+
container_image_mirror_images:
158+
- dest: oslat:old-version
159+
- dest: cyclictest:deprecated
160+
```
161+
162+
## Usage
163+
164+
### Via Playbook
165+
166+
Create a playbook that includes the role:
167+
168+
```yaml
169+
---
170+
- name: Mirror container images
171+
hosts: bastion
172+
gather_facts: true
173+
vars:
174+
images_to_mirror:
175+
- source: quay.io/org/app:v1.0
176+
dest: apps/app:v1.0
177+
roles:
178+
- role: container_image_mirror
179+
vars:
180+
container_image_mirror_operation: mirror
181+
container_image_mirror_images: "{{ images_to_mirror }}"
182+
```
183+
184+
### Command Line
185+
186+
```bash
187+
ansible-playbook mirror-images-playbook.yml \
188+
-i inventories/ocp-deployment/build-inventory.py \
189+
--extra-vars "container_image_mirror_pull_secret_string=$(cat ~/.docker/config.json | base64 -w0)"
190+
```
191+
192+
## Output Examples
193+
194+
### Successful Mirror
195+
196+
```
197+
TASK [container_image_mirror : Display detailed mirror results]
198+
ok: [bastion] => {
199+
"msg": [
200+
"==========================================",
201+
"MIRROR SUMMARY",
202+
"==========================================",
203+
"Total images: 3",
204+
"Successfully mirrored: 3",
205+
"Failed: 0",
206+
"",
207+
"SUCCESSFUL:",
208+
" ✓ oslat:latest",
209+
" ✓ cyclictest:latest",
210+
" ✓ cnf-tests:4.8",
211+
"",
212+
"FAILED:",
213+
" (none)",
214+
"=========================================="
215+
]
216+
}
217+
```
218+
219+
### Partial Failure
220+
221+
```
222+
TASK [container_image_mirror : Display detailed mirror results]
223+
ok: [bastion] => {
224+
"msg": [
225+
"==========================================",
226+
"MIRROR SUMMARY",
227+
"==========================================",
228+
"Total images: 3",
229+
"Successfully mirrored: 2",
230+
"Failed: 1",
231+
"",
232+
"SUCCESSFUL:",
233+
" ✓ oslat:latest",
234+
" ✓ cyclictest:latest",
235+
"",
236+
"FAILED:",
237+
" ✗ cnf-tests:4.8",
238+
"=========================================="
239+
]
240+
}
241+
242+
TASK [container_image_mirror : Fail if any images failed to mirror]
243+
fatal: [bastion]: FAILED! => {
244+
"msg": "1 image(s) failed to mirror. See summary above for details."
245+
}
246+
```
247+
248+
## Use Cases
249+
250+
### Telco RAN Test Image Mirroring
251+
Mirror test images from quay.io to bastion registries for spoke cluster testing:
252+
```yaml
253+
container_image_mirror_images:
254+
- {source: "quay.io/telcov10n-ci/oslat:latest", dest: "ran-test/oslat:latest"}
255+
- {source: "quay.io/telcov10n-ci/cyclictest:latest", dest: "ran-test/cyclictest:latest"}
256+
- {source: "quay.io/telcov10n-ci/stress-ng:latest", dest: "ran-test/stress-ng:latest"}
257+
```
258+
259+
### Disconnected Environment Preparation
260+
Mirror images to internal registry before deploying in air-gapped environment:
261+
```yaml
262+
container_image_mirror_operation: mirror
263+
container_image_mirror_registry_host: registry.internal.corp
264+
container_image_mirror_images:
265+
- {source: "quay.io/app/image:v1", dest: "production/app:v1"}
266+
```
267+
268+
### Registry Cleanup
269+
Remove old or deprecated images from internal registry:
270+
```yaml
271+
container_image_mirror_operation: remove
272+
container_image_mirror_images:
273+
- {dest: "deprecated/old-app:v1"}
274+
- {dest: "test/temp-image:latest"}
275+
```
276+
277+
## Notes
278+
279+
- **Continues on error**: The role continues mirroring all images even if some fail, then reports detailed results at the end
280+
- **Pull secrets**: The role cleans up pull secrets after use for security
281+
- **TLS verification**: Disabled by default for internal registries with self-signed certificates
282+
- **Idempotency**: Checks for existing images before mirroring (though still attempts to mirror for freshness)
283+
- **Namespace handling**: The `container_image_mirror_registry_namespace` is prepended to all destination images
284+
285+
## Troubleshooting
286+
287+
### Skopeo authentication errors
288+
Ensure `container_image_mirror_pull_secret_string` contains valid credentials:
289+
```bash
290+
cat ~/.docker/config.json | base64 -w0
291+
```
292+
293+
### TLS verification errors
294+
If using self-signed certificates, ensure `container_image_mirror_dest_tls_verify: false`
295+
296+
### Permission errors during removal
297+
Ensure the ansible user has sudo access to the registry storage path
298+
299+
### Image not found errors
300+
Verify source image exists and is accessible:
301+
```bash
302+
skopeo inspect docker://quay.io/org/image:tag
303+
```
304+
305+
## License
306+
Apache-2.0
307+
308+
## Author Information
309+
Telco Verification Team - Red Hat

0 commit comments

Comments
 (0)