-
Notifications
You must be signed in to change notification settings - Fork 49
feat(RELEASE-1989): skopeo helper client #812
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
midnightercz
wants to merge
1
commit into
main
Choose a base branch
from
RELEASE-1989-skopeo-helper
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+2,519
−0
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,146 @@ | ||
| # Fake Skopeo Client Implementation Summary | ||
|
|
||
| ## What Was Implemented | ||
|
|
||
| A complete fake/mock implementation of `SkopeoClient` for testing purposes, based on the design discussion. | ||
|
|
||
| ## Files Created | ||
|
|
||
| 1. **`scripts/python/helpers/fake/__init__.py`** | ||
| - Exports `FakeSkopeoClient` and `patch_skopeo_client()` | ||
| - The `patch_skopeo_client()` function performs monkey-patching | ||
|
|
||
| 2. **`scripts/python/helpers/fake/skopeo.py`** | ||
| - `FakeSkopeoClient` class - main implementation | ||
| - Loads YAML config from `RELEASE_SERVICE_UTILS_FAKE_SKOPEO_SETUP` env var | ||
| - Implements `inspect()` and `copy()` with full method signatures | ||
| - Supports regex matching, first-match-wins rule evaluation | ||
| - Validates config at load time | ||
| - Raises `SkopeoClientError` when no match found | ||
|
|
||
| 3. **`scripts/python/helpers/fake/README.md`** | ||
| - Comprehensive documentation | ||
| - Usage examples | ||
| - YAML format reference | ||
| - Troubleshooting guide | ||
|
|
||
| 4. **`scripts/python/helpers/fake/example_config.yaml`** | ||
| - Demonstrates all features: exact matching, regex, success/failure cases | ||
| - Ready-to-use examples for common test scenarios | ||
|
|
||
| 5. **`scripts/python/helpers/fake/test_fake_skopeo.py`** | ||
| - 13 unit tests covering all functionality | ||
| - All tests passing | ||
| - Validates load-time checks, runtime matching, error handling | ||
|
|
||
| 6. **`scripts/python/helpers/fake/example_test.sh`** | ||
| - Working end-to-end example using bash wrapper | ||
| - Demonstrates integration with `publish_index_image` | ||
| - Shows how to use in Tekton/CI tests | ||
|
|
||
| ## Key Design Decisions | ||
|
|
||
| ### Activation Mechanism | ||
| - **Environment variable**: `RELEASE_SERVICE_UTILS_FAKE_SKOPEO_SETUP` points to YAML config | ||
| - **Monkey patching**: `patch_skopeo_client()` replaces real client before imports | ||
| - **Drop-in replacement**: Same constructor and method signatures as `SkopeoClient` | ||
|
|
||
| ### YAML Structure | ||
| - Top-level operations: `inspect`, `copy` | ||
| - Each operation has list of rules with `match` and optional `return` | ||
| - First matching rule wins (top-to-bottom evaluation) | ||
|
|
||
| ### Matching Logic | ||
| - Only fields specified in `match` must match | ||
| - Extra parameters in actual calls are ignored | ||
| - Secrets are always ignored in matching | ||
| - `None` values don't match specified patterns | ||
| - Regex support via `{regex: "pattern"}` with fullmatch semantics | ||
|
|
||
| ### Validation | ||
| - **Load-time validation**: YAML syntax, rule structure, return types | ||
| - **Runtime validation**: Regex patterns applied during matching | ||
| - **Error messages**: Detailed, showing attempted params and config file path | ||
|
|
||
| ### Return Value Handling | ||
|
|
||
| **For `inspect()`:** | ||
| - `format` specified in match → return must be string | ||
| - No `format` in match → return must be dict | ||
| - Auto-detected based on match rule | ||
|
|
||
| **For `copy()`:** | ||
| - Omit `return` section → success (returns `None`) | ||
| - `return: {success: true}` → explicit success | ||
| - `return: {success: false, ...}` → raises `SkopeoClientError` | ||
| - Default values: `returncode=1`, `stdout=""`, `stderr=""` | ||
|
|
||
| ## Usage Pattern for Tests | ||
|
|
||
| ```bash | ||
| # 1. Create mock config | ||
| cat > mock-config.yaml <<EOF | ||
| inspect: | ||
| - match: | ||
| image: "docker://quay.io/target:tag" | ||
| format: "{{.Digest}}" | ||
| return: "sha256:different" | ||
|
|
||
| copy: | ||
| - match: | ||
| source: "docker://quay.io/source@sha256:abc" | ||
| destination: "docker://quay.io/target:tag" | ||
| EOF | ||
|
|
||
| # 2. Set environment variable | ||
| export RELEASE_SERVICE_UTILS_FAKE_SKOPEO_SETUP=/path/to/mock-config.yaml | ||
|
|
||
| # 3. Create bash wrapper | ||
| publish_index_image() { | ||
| python3 - "$@" <<'PYTHON' | ||
| import sys | ||
| sys.path.insert(0, '/path/to/helpers') | ||
| sys.path.insert(0, '/path/to/tasks/internal') | ||
|
|
||
| from fake import patch_skopeo_client | ||
| patch_skopeo_client() | ||
|
|
||
| from publish_index_image import main | ||
| sys.exit(main()) | ||
| PYTHON | ||
| } | ||
|
|
||
| # 4. Run your test | ||
| publish_index_image --source-index "..." --target-index "..." | ||
| ``` | ||
|
|
||
| ## Testing the Implementation | ||
|
|
||
| ```bash | ||
| # Run unit tests | ||
| pytest scripts/python/helpers/fake/test_fake_skopeo.py -v | ||
|
|
||
| # Run integration example | ||
| ./scripts/python/helpers/fake/example_test.sh | ||
| ``` | ||
|
|
||
| ## What's NOT Included | ||
|
|
||
| Based on our design discussion, the following were explicitly excluded: | ||
|
|
||
| - Multiple config file support (only single file) | ||
| - Special debug/strict/recording modes | ||
| - Validation of dict field names (type-only validation) | ||
| - Matching on constructor parameters | ||
| - Matching on credential values | ||
| - Template parsing for `format` parameter (returns pre-formatted strings) | ||
|
|
||
| These can be added later if needed without breaking existing configs. | ||
|
|
||
| ## Next Steps | ||
|
|
||
| 1. **In your other project**: Create test YAML configs specific to your test scenarios | ||
| 2. **Update CI/Tekton tasks**: Use the bash wrapper pattern to inject fake client | ||
| 3. **Write tests**: Use `example_test.sh` as a template for your specific test cases | ||
| 4. **Iterate**: Add more rules to your YAML configs as you encounter new test scenarios | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| """Fake implementations for testing.""" | ||
|
|
||
| from fake.skopeo import FakeSkopeoClient | ||
|
|
||
|
|
||
| def patch_skopeo_client() -> None: | ||
| """Monkey-patch skopeo.SkopeoClient with FakeSkopeoClient. | ||
|
|
||
| This function replaces the real SkopeoClient with the fake implementation. | ||
| Must be called before importing any modules that use SkopeoClient. | ||
| """ | ||
| import skopeo | ||
|
|
||
| skopeo.SkopeoClient = FakeSkopeoClient | ||
|
|
||
|
|
||
| __all__ = ["FakeSkopeoClient", "patch_skopeo_client"] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| --- | ||
| # Example mock configuration for FakeSkopeoClient | ||
| # | ||
| # This file demonstrates the YAML structure for defining mock responses | ||
| # for skopeo operations (inspect and copy). | ||
| # | ||
| # Usage: | ||
| # export RELEASE_SERVICE_UTILS_FAKE_SKOPEO_SETUP=/path/to/this/file.yaml | ||
| # python -c "from fake import patch_skopeo_client; patch_skopeo_client(); ..." | ||
|
|
||
| # Inspect operation rules | ||
| inspect: | ||
| # Example 1: Return formatted string when format parameter is specified | ||
| - match: | ||
| image: "docker://quay.io/source/image@sha256:abc123def456" | ||
| format: "{{.Digest}}" | ||
| return: "sha256:abc123def456" | ||
|
|
||
| # Example 2: Return full manifest dict when no format specified | ||
| - match: | ||
| image: "docker://quay.io/target/image:v1.0" | ||
| return: | ||
| Digest: "sha256:def456abc123" | ||
| Name: "quay.io/target/image" | ||
| RepoTags: | ||
| - "v1.0" | ||
| - "latest" | ||
| Labels: | ||
| version: "1.0.0" | ||
| maintainer: "team@example.com" | ||
|
|
||
| # Example 3: Regex matching for dynamic image names | ||
| - match: | ||
| image: | ||
| regex: "docker://quay.io/.*@sha256:[a-f0-9]{64}" | ||
| format: "{{.Digest}}" | ||
| return: "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef" | ||
|
|
||
| # Example 4: Regex matching without format (returns dict) | ||
| - match: | ||
| image: | ||
| regex: "docker://registry-proxy\\.engineering\\.redhat\\.com/.*" | ||
| return: | ||
| Digest: "sha256:fedcba987654" | ||
| Name: "registry-proxy.engineering.redhat.com/rh-osbs/openshift-golang-builder" | ||
|
|
||
| # Copy operation rules | ||
| copy: | ||
| # Example 1: Successful copy (no return section = success) | ||
| - match: | ||
| source: "docker://quay.io/source/image@sha256:abc123" | ||
| destination: "docker://quay.io/dest/image:tag" | ||
|
|
||
| # Example 2: Successful copy with explicit return | ||
| - match: | ||
| source: "docker://quay.io/another/image:v1" | ||
| destination: "docker://quay.io/target/image:v1" | ||
| return: | ||
| success: true | ||
|
|
||
| # Example 3: Failed copy with custom error | ||
| - match: | ||
| source: "docker://quay.io/nonexistent/image:tag" | ||
| destination: "docker://quay.io/dest/image:tag" | ||
| return: | ||
| success: false | ||
| stderr: "Error: manifest unknown: manifest unknown" | ||
| returncode: 1 | ||
|
|
||
| # Example 4: Authentication failure | ||
| - match: | ||
| source: "docker://quay.io/private/image:tag" | ||
| destination: "docker://quay.io/dest/image:tag" | ||
| return: | ||
| success: false | ||
| stderr: "Error: authentication required" | ||
| returncode: 1 | ||
|
|
||
| # Example 5: Regex matching for source with digest | ||
| - match: | ||
| source: | ||
| regex: "docker://quay.io/source/.*@sha256:[a-f0-9]{64}" | ||
| destination: | ||
| regex: "docker://quay.io/dest/.*:.*" | ||
| # Omit return = success |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.