Skip to content

Commit 2214746

Browse files
AndresNicoAndresNico
authored
feat[168]: Add batch 2 upgrade test resources (#217)
* Add batch 2 upgrade test resources * Add ServiceInstance dependency to readme * Add ServiceCredentialBinding dependency * Move baseCrs resources to subdirectories and add annotation * Add custom assessment for base_upgrade_test * Update readme
1 parent fefedd3 commit 2214746

11 files changed

Lines changed: 239 additions & 25 deletions

File tree

‎docs/upgrade-testing.md‎

Lines changed: 71 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -60,9 +60,15 @@ You need valid CloudFoundry credentials with appropriate permissions:
6060

6161
## Quick Start
6262

63-
### ⚠️ IMPORTANT: Configure Your CF Organization First
63+
### ⚠️ IMPORTANT: Necessary configuration steps before running any tests
6464

65-
Before running any tests, you **must** update the organization name to one you have access to:
65+
Some configuration steps are necessary before you can successfully run any upgrade tests:
66+
- Configure your CF organization
67+
- Configure your CF space
68+
69+
#### CF Organization
70+
71+
Before running any tests, you **must** update the organization name to one you have access:
6672

6773
1. **List your available CF organizations:**
6874
```bash
@@ -85,6 +91,42 @@ spec:
8591
name: your-org-name-here # ← Change this to your CF org name
8692
```
8793
94+
#### CF Space
95+
96+
Before running the base tests, you **must** update the space name to one in your organization.
97+
That can either be an existing one you have atleast the SpaceDeveloper role in or you create a new one as describe below:
98+
99+
1. **Optionally create and configure a new CF Space**
100+
101+
Create a space and give your user the SpaceDeveloper role
102+
```bash
103+
cf create-space <SPACE_NAME> -o <ORG_NAME> # Create a space in your org
104+
cf set-space-role <USERNAME> <ORG_NAME> <SPACE_NAME> SpaceDeveloper # Assign your user the SpaceDeveloper role
105+
```
106+
107+
2. **List your available CF spaces:**
108+
```bash
109+
cf spaces # List spaces
110+
```
111+
3. **Update the spaces name** in test manifests:
112+
- For base tests: `test/upgrade/testdata/baseCrs/import.yaml`:
113+
- For custom tests: `test/upgrade/testdata/customCRs/*/import.yaml` (if applicable)
114+
```yaml
115+
apiVersion: cloudfoundry.crossplane.io/v1alpha1
116+
kind: Space
117+
metadata:
118+
name: upgrade-test-import-space
119+
spec:
120+
managementPolicies:
121+
- Observe
122+
forProvider:
123+
name: upgrade-test-space-donotdelete # ← Change this to you CF space name
124+
orgRef:
125+
name: upgrade-test-org
126+
providerConfigRef:
127+
name: default
128+
```
129+
88130
### 1. Set Environment Variables
89131
90132
#### Option A: Use the provided template
@@ -190,14 +232,23 @@ make test-upgrade-custom
190232
Base tests use YAML manifests from `test/upgrade/testdata/baseCrs/`. Currently tested resources:
191233

192234
- **Organization** (import) - Uses `managementPolicies: [Observe]` to import existing org
235+
- **Space** (import) - Uses `managementPolicies: [Observe]` to import existing space
193236
- **Space** - Lightweight resource for testing basic upgrade flow
194237
- **Domain**
195238
- **SpaceQuota**
196239
- **SpaceRole**
240+
- **SpaceMembers**
241+
- **ServiceInstance**
242+
- **ServiceCredentialBinding**
197243

198244
#### Test Base Resource Dependencies
199245
- **SpaceRole:** A space role can only be assigned to a user if the user is also a member of the space's organization.\
200246
🠊 Assign a user to the space's organization by either creating a SpaceMembers/SpaceRole resource or by using the BTP Cockpit
247+
- **ServiceInstance:** A managed service instance requires a ServicePlan specifying an offering and a plan.
248+
If the combination of offering and plan is not available in your space change it something different.\
249+
🠊 Run `cf marketplace` and update the values in test/upgrade/testdata/baseCrs/service_instance.yaml
250+
- **ServiceCredentialBinding:** The ServiceCredentialBinding directly depends on the ServiceInstance it is referencing \
251+
🠊 The `base_upgrade_test` includes dedicated pre- and post-upgrade assessment for the ServiceInstance resources and its dependents. These assessments verify the ServiceInstance first, and only then thedependent resources such as ServiceCredentialBinding. This ordering makes dependency failures easier to diagnose and test less flaky when the upstream ServiceInstance is not healthy.
201252

202253
#### Adding New Base Test Resources
203254

@@ -317,15 +368,26 @@ test/
317368
├── upgrade/
318369
│ ├── testdata/
319370
│ │ ├── baseCrs/ # Base upgrade test resources
320-
│ │ │ ├── import.yaml # Organization (observe)
321-
│ │ │ ├── space.yaml # Space (create)
322-
| │ │ ├── domain.yaml
323-
| │ │ ├── space_quota.yaml
324-
| │ │ └── space_role.yaml
325-
│ │ └── customCRs/ # Custom upgrade test resources
371+
│ │ │ ├── import/
372+
│ │ │ │ └── import_org.yaml # Organization (observe)
373+
│ │ │ ├── space/
374+
│ │ │ │ └── space.yaml # Space (create)
375+
│ │ │ ├── domain/
376+
│ │ │ │ └── domain.yaml
377+
│ │ │ ├── spaceQuota/
378+
│ │ │ │ └── space_quota.yaml
379+
│ │ │ ├── spaceRole/
380+
│ │ │ │ └── space_role.yaml
381+
│ │ │ ├── serviceCredentialBinding/
382+
│ │ │ │ └── service_credential_binding.yaml
383+
│ │ │ ├── serviceInstance/
384+
│ │ │ │ └── service_instance.yaml
385+
│ │ │ └── spaceMembers/
386+
│ │ │ └── space_members.yaml
387+
│ │ └── customCrs/ # Custom upgrade test resources
326388
│ │ └── externalNames/ # External-name validation test
327389
│ │ ├── space.yaml
328-
| | └── import.yaml
390+
│ │ └── import.yaml
329391
│ ├── main_test.go # Test environment setup
330392
│ ├── upgrade_test.go # Base upgrade test logic
331393
│ ├── base_upgrade_test.go # Custom upgrade test framework

‎test/upgrade/base_upgrade_test.go‎

Lines changed: 74 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,20 +15,92 @@
1515
package upgrade
1616

1717
import (
18+
"context"
19+
"path/filepath"
20+
"slices"
21+
"strings"
1822
"testing"
23+
"time"
24+
25+
"github.com/crossplane-contrib/xp-testing/pkg/resources"
26+
"k8s.io/klog/v2"
27+
"sigs.k8s.io/e2e-framework/klient/wait"
28+
"sigs.k8s.io/e2e-framework/pkg/envconf"
1929
)
2030

2131
func TestUpgradeProvider(t *testing.T) {
2232
fromTag, toTag := loadTags()
2333

34+
serviceInstanceDir := filepath.Join(resourceDirectoryRoot, "serviceInstance")
35+
serviceCredentialBindingDir := filepath.Join(resourceDirectoryRoot, "serviceCredentialBinding")
36+
dependentDirs := []string{serviceCredentialBindingDir}
37+
38+
requiredDirs := removeFromDirs(resourceDirectories, dependentDirs)
39+
2440
upgradeTest := NewCustomUpgradeTest("baseline-upgrade-test").
2541
FromVersion(fromTag).
2642
ToVersion(toTag).
27-
WithResourceDirectories(resourceDirectories)
28-
43+
WithResourceDirectories(resourceDirectories).
44+
SkipDefaultResourceVerification().
45+
WithCustomPreUpgradeAssessment(
46+
"Check all required resources are healthy before upgrade",
47+
func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
48+
return verifyResources(ctx, t, cfg, requiredDirs, verifyTimeout)
49+
},
50+
).
51+
WithCustomPreUpgradeAssessment(
52+
"Check service instance and dependent resources are healthy before upgrade",
53+
func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
54+
return verifyServiceInstanceWithDependents(ctx, t, cfg, serviceInstanceDir, dependentDirs, verifyTimeout)
55+
},
56+
).
57+
WithCustomPostUpgradeAssessment(
58+
"Check all required resources are healthy after upgrade",
59+
func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
60+
return verifyResources(ctx, t, cfg, requiredDirs, verifyTimeout)
61+
},
62+
).
63+
WithCustomPostUpgradeAssessment(
64+
"Check service instance and dependent resources are healthy after upgrade",
65+
func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
66+
return verifyServiceInstanceWithDependents(ctx, t, cfg, serviceInstanceDir, dependentDirs, verifyTimeout)
67+
},
68+
)
2969
testenv.Test(t, upgradeTest.Feature())
3070
}
3171

72+
// verifyResources waits for resources in dirs to be ready
73+
func verifyResources(ctx context.Context, t *testing.T, cfg *envconf.Config, dirs []string, timeout time.Duration) context.Context {
74+
for _, dir := range dirs {
75+
klog.V(4).Infof("verify resources of directory %s", dir)
76+
if err := resources.WaitForResourcesToBeSynced(ctx, cfg, dir, nil, wait.WithTimeout(timeout)); err != nil {
77+
t.Errorf("verify resources of directory %s failed: %v", dir, err)
78+
}
79+
}
80+
81+
return ctx
82+
}
83+
84+
// verifyServiceInstanceWithDependents verifies the service instance directory first and
85+
// if successful dependent directories
86+
func verifyServiceInstanceWithDependents(ctx context.Context, t *testing.T, cfg *envconf.Config, serviceInstanceDir string, dependentDirs []string, timeout time.Duration) context.Context {
87+
klog.V(4).Infof("verify service instance")
88+
if err := resources.WaitForResourcesToBeSynced(ctx, cfg, serviceInstanceDir, nil, wait.WithTimeout(timeout)); err != nil {
89+
t.Errorf("verify service instance failed: %v — skipping verification of: %s", err, strings.Join(dependentDirs, ", "))
90+
return ctx
91+
}
92+
return verifyResources(ctx, t, cfg, dependentDirs, timeout)
93+
}
94+
95+
// removeFromDirs is a helper function to remove directories from the list of directories to verify,
96+
// allowing dependent resources to be verified separately
97+
func removeFromDirs(dirs []string, remove []string) []string {
98+
result := slices.Clone(dirs)
99+
return slices.DeleteFunc(result, func(d string) bool {
100+
return slices.Contains(remove, d)
101+
})
102+
}
103+
32104
// loadTags is a helper function to load FROM and TO tags for tests
33105
// This allows custom tests to reuse the same version configuration
34106
func loadTags() (string, string) {

test/upgrade/testdata/baseCrs/domain.yaml renamed to test/upgrade/testdata/baseCrs/domain/domain.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ apiVersion: cloudfoundry.crossplane.io/v1alpha1
22
kind: Domain
33
metadata:
44
name: upgrade-test-domain
5+
annotations:
6+
crossplane.io/paused: "false"
57
spec:
68
forProvider:
79
name: cfupgradetest.eu12.hana.ondemand.com

‎test/upgrade/testdata/baseCrs/import.yaml‎

Lines changed: 0 additions & 13 deletions
This file was deleted.

‎test/upgrade/testdata/baseCrs/import/import.yaml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
apiVersion: cloudfoundry.crossplane.io/v1alpha1
2+
kind: Organization
3+
metadata:
4+
name: upgrade-test-org
5+
annotations:
6+
crossplane.io/paused: "false"
7+
spec:
8+
managementPolicies:
9+
- Observe
10+
forProvider:
11+
# IMPORTANT: Change this to an org you have access to
12+
# Run `cf orgs` to see available organizations
13+
name: cf-ci-e2e # ← TODO: Change this to an existing org name
14+
providerConfigRef:
15+
name: default
16+
---
17+
apiVersion: cloudfoundry.crossplane.io/v1alpha1
18+
kind: Space
19+
metadata:
20+
name: upgrade-test-import-space
21+
annotations:
22+
crossplane.io/paused: "false"
23+
spec:
24+
managementPolicies:
25+
- Observe
26+
forProvider:
27+
name: upgrade-test-space-donotdelete
28+
orgRef:
29+
name: upgrade-test-org # ← References the imported org
30+
providerConfigRef:
31+
name: default

‎test/upgrade/testdata/baseCrs/serviceCredentialBinding/service_credential_binding.yaml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
apiVersion: cloudfoundry.crossplane.io/v1alpha1
2+
kind: ServiceCredentialBinding
3+
metadata:
4+
name: upgrade-test-service-credential-binding
5+
annotations:
6+
crossplane.io/paused: "false"
7+
spec:
8+
forProvider:
9+
name: upgrade-test-service-credential-binding
10+
type: key
11+
serviceInstanceRef:
12+
name: upgrade-test-service-instance
13+
policy:
14+
resolve: Always
15+
providerConfigRef:
16+
name: default

‎test/upgrade/testdata/baseCrs/serviceInstance/service_instance.yaml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
apiVersion: cloudfoundry.crossplane.io/v1alpha1
2+
kind: ServiceInstance
3+
metadata:
4+
name: upgrade-test-service-instance
5+
annotations:
6+
crossplane.io/paused: "false"
7+
spec:
8+
forProvider:
9+
type: managed
10+
name: upgrade-test-service-instance
11+
spaceRef:
12+
name: upgrade-test-import-space
13+
policy:
14+
resolve: Always
15+
servicePlan:
16+
offering: destination # Change the offering if not available
17+
plan: lite # Change the plan if not available
18+
providerConfigRef:
19+
name: default

test/upgrade/testdata/baseCrs/space.yaml renamed to test/upgrade/testdata/baseCrs/space/space.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ apiVersion: cloudfoundry.crossplane.io/v1alpha1
22
kind: Space
33
metadata:
44
name: upgrade-test-space
5+
annotations:
6+
crossplane.io/paused: "false"
57
spec:
68
forProvider:
79
name: upgrade-test-space

‎test/upgrade/testdata/baseCrs/spaceMembers/space_members.yaml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
apiVersion: cloudfoundry.crossplane.io/v1alpha1
2+
kind: SpaceMembers
3+
metadata:
4+
name: upgrade-test-space-members
5+
annotations:
6+
crossplane.io/paused: "false"
7+
spec:
8+
forProvider:
9+
roleType: Developers
10+
members:
11+
- username: upgrade-test-space-member-1@example.com
12+
- username: upgrade-test-space-member-2@example.com
13+
spaceRef:
14+
name: upgrade-test-import-space
15+
policy:
16+
resolve: Always
17+
enforcementPolicy: Lax
18+
providerConfigRef:
19+
name: default

test/upgrade/testdata/baseCrs/space_quota.yaml renamed to test/upgrade/testdata/baseCrs/spaceQuota/space_quota.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ apiVersion: cloudfoundry.crossplane.io/v1alpha1
22
kind: SpaceQuota
33
metadata:
44
name: upgrade-test-space-quota
5+
annotations:
6+
crossplane.io/paused: "false"
57
spec:
68
forProvider:
79
allowPaidServicePlans: false

0 commit comments

Comments
 (0)