Skip to content

Generate namespaced managed resources#1751

Closed
jbw976 wants to merge 31 commits intocrossplane-contrib:crossplane-v2from
negz:spatial
Closed

Generate namespaced managed resources#1751
jbw976 wants to merge 31 commits intocrossplane-contrib:crossplane-v2from
negz:spatial

Conversation

@jbw976
Copy link
Copy Markdown
Member

@jbw976 jbw976 commented Mar 21, 2025

Description of your changes

This PR continues the work from draft PR #1689 that @negz started. It essentially updates this provider to start generating namespaced managed resources in addition to cluster scoped ones, which will be featured heavily in the current proposal of Crossplane v2.

This should not be taken as a final product. This is a practical update to this provider as a proof of concept for what we think a provider could look like in Crossplane v2. There are many short cuts taken, it is not production ready, and we are very open to feedback to change directions or improve the architecture.

The work in this PR should be merged into the new crossplane-v2 branch, which is based on commit 1ecd9cb, the latest commit when we started this work. We will need to rebase this effort in the future to include latest changes from main, but that should not block this PR.

I have:

  • Read and followed Crossplane's contribution process.
  • Run make reviewable to ensure this PR is ready for review.

General steps this PR has taken

The following list describes the general process that was taken to update this provider to support namespaced MRs:

  • update to latest upjet version that supports namespaced resources
  • duplicate content into both cluster and namespaced copies and make some minor updates for api groups and import paths
    • config, apis, controllers
  • remove all legacy conversion/migration logic, because only the latest version needs to be supported
  • update the provider main.go template to setup both cluster and namespaced apis and controllers
  • update code generation comment markers to run on both cluster and namespaced types
  • update the generator cmd to init both cluster and namespaced config to pass to code gen pipeline
  • manually copy and update the handful of manual api and controller files to namespaced dirs

How has this code been tested

Besides generating, building and publishing successfully, this code has also been tested manually by installing the config, s3, rds, and ec2 packages into a control plane, then creating both cluster scoped and namespaced S3 buckets. Ex.:

❯ { head -n 1; grep '^buckets '; } < <(kubectl api-resources)
NAME                                        SHORTNAMES   APIVERSION                            NAMESPACED   KIND
buckets                                                  s3.aws.upbound.io/v1beta2             false        Bucket
buckets                                                  s3.m.aws.upbound.io/v1beta1           true         Bucket

❯ k get managed -A
NAMESPACE   NAME                                               SYNCED   READY   EXTERNAL-NAME             AGE
            bucket.s3.aws.upbound.io/crossplane-bucket-tmm54   True     True    crossplane-bucket-tmm54   21s

NAMESPACE   NAME                                                 SYNCED   READY   EXTERNAL-NAME             AGE
xp          bucket.s3.m.aws.upbound.io/crossplane-bucket-mhv6k   True     True    crossplane-bucket-mhv6k   18s

negz and others added 30 commits February 26, 2025 19:45
Signed-off-by: Nic Cope <nicc@rk0n.org>
It doesn't fit well in apis given it also generates controllers,
commands, etc.

Signed-off-by: Nic Cope <nicc@rk0n.org>
Ideally the code generation configuration wouldn't depend on generated
code. That should be fixed in a future commit.

Signed-off-by: Nic Cope <nicc@rk0n.org>
Signed-off-by: Nic Cope <nicc@rk0n.org>
Signed-off-by: Nic Cope <nicc@rk0n.org>
Using:

sed -i -e 's/scope=Cluster/scope=Namespaced/' $(find . -name '*.go'|xargs)

Signed-off-by: Nic Cope <nicc@rk0n.org>
Unchanged, for now.

Signed-off-by: Nic Cope <nicc@rk0n.org>
This should make the namespaced APIs and controllers actually namespaced

Signed-off-by: Nic Cope <nicc@rk0n.org>
Signed-off-by: Nic Cope <nicc@rk0n.org>
The controller import package is now configurable.

Signed-off-by: Nic Cope <nicc@rk0n.org>
Signed-off-by: Nic Cope <nicc@rk0n.org>
Signed-off-by: Jared Watts <jbw976@gmail.com>
Signed-off-by: Jared Watts <jbw976@gmail.com>
Signed-off-by: Jared Watts <jbw976@gmail.com>
…o pass to upjet pipeline

Signed-off-by: Jared Watts <jbw976@gmail.com>
Signed-off-by: Jared Watts <jbw976@gmail.com>
Signed-off-by: Jared Watts <jbw976@gmail.com>
Signed-off-by: Jared Watts <jbw976@gmail.com>
Signed-off-by: Jared Watts <jbw976@gmail.com>
Signed-off-by: Jared Watts <jbw976@gmail.com>
…lly cased in singleton list handling

These specific controllers were handled specially in the singleton list conversion code,
since that has now been removed, their versions are bumped during code gen. Not entirely
sure this is a good thing just yet, but we'll see.

Signed-off-by: Jared Watts <jbw976@gmail.com>
Signed-off-by: Jared Watts <jbw976@gmail.com>
Signed-off-by: Jared Watts <jbw976@gmail.com>
Signed-off-by: Jared Watts <jbw976@gmail.com>
* refactor GetProvider so that xpprovider.GetProvider() is only
  called once at runtime and then reused for both cluster and
  namespaced
* update provider main.go template to use both cluster and namespaced
  config during start up. This is important as both of their setup,
  conversions, etc. need to be honored

Signed-off-by: Jared Watts <jbw976@gmail.com>
Signed-off-by: Jared Watts <jbw976@gmail.com>
The CRD file names don't conflict with each other because they have
their full group names embedded in the file name. When the kustomize
tooling runs over the CRDs, it's easier to process them all in one single
directory, so we don't end up with wierd duplicates.

Signed-off-by: Jared Watts <jbw976@gmail.com>
This isn't an ideal name because it doesn't exactly match the same pattern
of other groups, e.g. s3.m.aws.upbound.io, but it is a quick way to make
publishing work for the family/config provider. That provider is special
cased to take all aws.* CRDs. Updating the group for namespaced core/config
CRDs to aws.m.upbound.io means they are also included in the family/config
provider along with the cluster scoped aws.upbound.io CRDs. We should likely
revisit this with more time to maybe update the up xpkg batch tooling.

Signed-off-by: Jared Watts <jbw976@gmail.com>
…tory

Signed-off-by: Jared Watts <jbw976@gmail.com>
…atial

Signed-off-by: Jared Watts <jbw976@gmail.com>
@jbw976
Copy link
Copy Markdown
Member Author

jbw976 commented Mar 23, 2025

I lost the ability to push to @negz's fork/branch when I opened this new PR with my as the author. I'll open a new PR from my fork.

@jbw976 jbw976 closed this Mar 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants