Skip to content

Commit 17afab5

Browse files
scripts: rewrite upgrade-operator-sdk as generic YAML-driven tool
Replace the monolithic script with a generic, config-driven tool that takes an operator directory as argument. Each operator has its own config.yaml and patches/ directory under scripts/upgrade-operator-sdk/. All versions (operator-sdk, Go toolchain, k8s.io libs) are now pinned in the YAML config — the script makes no API calls for version detection. Key changes per review feedback: - YAML config per operator instead of hardcoded OPERATORS dict - GNU patch files for scaffold customizations (Dockerfile, Makefile) - Inclusion-list merge (backup_paths) instead of exclusion-list - No global mutable state (VersionInfo removed) - No SourceFix mechanism (manual fixes by the developer) - API scope (--namespaced) configurable per CRD - Empty group hack removed (operator-sdk v1.42.1 supports it) - Backup/scaffold file conflicts shown as errors with diff - Extra commands (make metalk8s) configurable in YAML
1 parent 800dc94 commit 17afab5

File tree

10 files changed

+691
-1196
lines changed

10 files changed

+691
-1196
lines changed

BUMPING.md

Lines changed: 46 additions & 111 deletions
Original file line numberDiff line numberDiff line change
@@ -131,144 +131,79 @@ This guide is applied for both `metalk8s-operator` and `storage-operator`.
131131
### Prerequisites
132132

133133
- `go`, `curl`, and `patch` in `PATH`.
134-
- A GitHub personal access token is optional but strongly recommended: without it,
135-
GitHub API calls are subject to a 60 requests/hour anonymous rate limit. The token
136-
must be **exported** so child processes inherit it:
134+
- `pyyaml` Python package: `pip install pyyaml`
137135

138-
```
139-
export GITHUB_TOKEN=<your_token>
140-
```
136+
### Updating the versions
141137

142-
Setting the variable without `export` (e.g. `GITHUB_TOKEN=xxx`) is silently
143-
ignored by the script because Python's `os.environ` only sees exported variables.
138+
Before running the script, update the target versions in the YAML config files at
139+
`scripts/upgrade-operator-sdk/<name>/config.yaml`:
140+
141+
```yaml
142+
operator_sdk_version: v1.42.1 # target operator-sdk release
143+
go_toolchain: go1.25.8 # Go toolchain (for GOTOOLCHAIN + FROM golang:X.Y)
144+
k8s_libs: v0.33.9 # k8s.io/{api,apimachinery,client-go} version
145+
```
146+
147+
The script makes **no version-detection API calls**; all versions are read from the
148+
YAML config.
144149
145150
### Running the upgrade
146151
147-
```
148-
python3 scripts/upgrade-operator-sdk.py
152+
The script processes one operator at a time. Run it once per operator:
153+
154+
```bash
155+
python3 scripts/upgrade-operator-sdk/upgrade.py operator
156+
python3 scripts/upgrade-operator-sdk/upgrade.py storage-operator
149157
```
150158

151-
The script will display the resolved versions and prompt for confirmation before
152-
making any changes. Use `--yes` to skip the confirmation (e.g. in CI). The original
153-
operator directories are preserved as `<name>.bak/` for the duration of the review.
159+
The argument is the name of the config directory next to the script
160+
(i.e. `scripts/upgrade-operator-sdk/<name>/`). A full path can also be
161+
given for configs stored elsewhere.
154162

155163
Options:
156164

157165
```
158-
--operator-only Only process operator/
159-
--storage-only Only process storage-operator/
160166
--skip-backup Reuse an existing .bak directory (no new backup)
161-
--clean-tools Delete .tmp/bin/ after the upgrade (~150 MB, re-downloaded next run)
167+
--clean-tools Delete .tmp/bin/ after the upgrade
162168
--yes, -y Skip the confirmation prompt
163169
```
164170

165-
The script caches `operator-sdk` in `.tmp/bin/` so it is not re-downloaded on
166-
repeated runs. Use `--clean-tools` to reclaim disk space once the upgrade is
167-
validated.
168-
169-
### What to review after the upgrade
170-
171-
After a successful run:
172-
173-
1. Compare the backup against the result to spot unexpected differences:
174-
175-
```
176-
diff -r operator.bak/ operator/
177-
diff -r storage-operator.bak/ storage-operator/
178-
```
179-
180-
2. Run the unit test suite for each operator:
181-
182-
```
183-
cd operator && make test
184-
cd storage-operator && make test
185-
```
186-
187-
3. Check that generated CRD scopes are correct:
188-
`config/crd/bases/``ClusterConfig` must be `Cluster`-scoped,
189-
`VirtualIPPool` must be `Namespaced`, `Volume` must be `Cluster`-scoped.
190-
191-
4. Check that the generated RBAC is complete:
192-
`config/rbac/role.yaml` in each operator.
193-
194-
5. Check that the MetalK8s manifests contain the correct Jinja template:
195-
`deploy/manifests.yaml` must contain
196-
`{{ build_image_name("metalk8s-operator") }}` / `{{ build_image_name("storage-operator") }}`.
171+
### YAML config files
197172

198-
6. Remove the backup directories once satisfied:
173+
Each operator has a config directory at `scripts/upgrade-operator-sdk/<name>/` containing
174+
`config.yaml` and a `patches/` subdirectory. The config fields are:
199175

200-
```
201-
rm -rf operator.bak/ storage-operator.bak/
202-
```
176+
- **Versions**: `operator_sdk_version`, `go_toolchain`, `k8s_libs`
177+
- **Scaffold**: `repo`, `domain`, `apis` (with `group`, `version`, `kind`, `namespaced`). The operator name is derived from the config directory name.
178+
- **Paths**: `operator_dir`, `patches_dir`, `backup_paths`
179+
- **Post-processing**: `image_placeholder`, `extra_commands`
203180

204181
### Patch files
205182

206183
MetalK8s-specific customizations to scaffold-generated files (`Dockerfile`, `Makefile`)
207-
are stored as standard GNU unified diff files in `scripts/patches/<operator>/`:
184+
are stored as GNU unified diff files in the `patches/` subdirectory next to `config.yaml`. The script
185+
applies them with `patch -p1` after scaffolding. If a patch does not apply cleanly,
186+
look for `.rej` files and resolve manually.
208187

209-
```
210-
scripts/patches/
211-
operator/
212-
Dockerfile.patch # extra COPY dirs, ldflags, Scality LABEL block
213-
Makefile.patch # GOTOOLCHAIN export, metalk8s make target
214-
storage-operator/
215-
Dockerfile.patch # extra COPY salt/, Scality LABEL block
216-
Makefile.patch # GOTOOLCHAIN export, metalk8s make target
217-
```
218-
219-
The script applies them with `patch -p1` after scaffolding. If a patch does not
220-
apply cleanly (e.g. because the scaffold changed significantly), the script warns
221-
but continues — look for `.rej` files in the operator directory and resolve manually.
222-
223-
#### Placeholders
188+
Patch files use `__PLACEHOLDER__` tokens for values from the YAML config:
224189

225-
Patch files use `__PLACEHOLDER__` tokens for values that are only known at runtime.
226-
The script replaces them after applying the patches:
190+
| Placeholder | Replaced with | Source |
191+
| ----------------- | -------------------------------------------- | ---------- |
192+
| `__GOTOOLCHAIN__` | `go_toolchain` from config (e.g. `go1.25.8`) | `Makefile` |
193+
| `__IMAGE__` | `image_placeholder` from config | `Makefile` |
227194

228-
| Placeholder | Replaced with | File |
229-
|---|---|---|
230-
| `__GOTOOLCHAIN__` | Detected Go toolchain (e.g. `go1.25.8`) | `Makefile` |
231-
| `__IMAGE__` | Jinja2 `build_image_name(...)` expression | `Makefile` |
195+
The `FROM golang:X.Y` in `Dockerfile` is derived from `go_toolchain` in the config.
232196

233-
The `FROM golang:X.Y` line in `Dockerfile` and `GOLANGCI_LINT_VERSION` in `Makefile`
234-
are updated by simple regex substitutions (not via patches), since their values change
235-
with every upgrade.
236-
237-
#### How to add or update a patch
238-
239-
Patches are plain `diff -u` output — you can edit them by hand or regenerate them.
240-
To regenerate after modifying an operator customization:
241-
242-
```bash
243-
# 1. Run the upgrade script with --skip-backup to get a fresh scaffold
244-
python3 scripts/upgrade-operator-sdk.py --operator-only --skip-backup --yes
197+
New `.patch` files in the patches directory are automatically picked up.
245198

246-
# 2. The script applies existing patches; to start fresh, reset the file:
247-
git checkout operator/Dockerfile
248-
249-
# 3. Make your changes to the scaffold file
250-
vim operator/Dockerfile
251-
252-
# 4. Generate the new patch (a/ b/ prefixes are required for patch -p1)
253-
diff -u <(git show HEAD:operator/Dockerfile) operator/Dockerfile \
254-
| sed '1s|.*|--- a/Dockerfile|;2s|.*|+++ b/Dockerfile|' \
255-
> scripts/patches/operator/Dockerfile.patch
256-
257-
# 5. Verify it applies cleanly
258-
git checkout operator/Dockerfile
259-
patch -p1 --dry-run -d operator < scripts/patches/operator/Dockerfile.patch
260-
```
261-
262-
To add a patch for a new file (e.g. `README.md`), create a new `.patch` file in
263-
the same directory — the script automatically picks up all `*.patch` files.
264-
265-
### Stale compatibility fixes
199+
### What to review after the upgrade
266200

267-
The `OPERATORS` dict in `scripts/upgrade-operator-sdk.py` contains a `fixes` tuple
268-
per operator. These entries are one-shot source-level corrections applied after the
269-
backup merge (e.g. deprecated API replacements). Once the backup no longer contains
270-
the old pattern — i.e. after the script has been run at least once — the entry
271-
becomes a no-op and should be removed to keep the script clean.
201+
1. `git diff` to review all changes
202+
2. `cd <operator> && make test` to run tests
203+
3. Check `config/crd/bases/` for correct CRD scopes
204+
4. Check `config/rbac/role.yaml` for RBAC completeness
205+
5. Check `deploy/manifests.yaml` for correct Jinja templates
206+
6. Remove backup: `rm -rf <operator>.bak/`
272207

273208
## Calico
274209

0 commit comments

Comments
 (0)