Skip to content

[#2379] Add Schema CRD for managing Protobuf schemas - #2452

Open
tristantarrant wants to merge 4 commits into
infinispan:mainfrom
tristantarrant:2379/schema_cr
Open

[#2379] Add Schema CRD for managing Protobuf schemas#2452
tristantarrant wants to merge 4 commits into
infinispan:mainfrom
tristantarrant:2379/schema_cr

Conversation

@tristantarrant

@tristantarrant tristantarrant commented Mar 6, 2026

Copy link
Copy Markdown
Member

Closes #2379

@ryanemerson

ryanemerson commented Mar 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

It's probably a follow-up issue, but it would be great if we could make it so that a Cache CR depends on a Schema existing on the server so that Cache creation is not attempted until the Schema exists. Something like:

apiVersion: infinispan.org/v2alpha1
kind: Schema
metadata:
  name: myschema
spec:
  clusterName: infinispan
  schema: ...
----
apiVersion: infinispan.org/v2alpha1
kind: Cache
metadata:
  name: mycachedefinition
spec:
  clusterName: infinispan
  schema: myschema

@tristantarrant

tristantarrant commented Mar 6, 2026

Copy link
Copy Markdown
Member Author

It's probably a follow-up issue, but it would be great if we could make it so that a Cache CR depends on a Schema existing on the server so that Cache creation is not attempted until the Schema exists. Something like:

apiVersion: infinispan.org/v2alpha1
kind: Schema
metadata:
  name: myschema
spec:
  clusterName: infinispan
  schema: ...
----
apiVersion: infinispan.org/v2alpha1
kind: Cache
metadata:
  name: mycachedefinition
spec:
  clusterName: infinispan
  schema: myschema

Good idea. Let's do it here

@tristantarrant
tristantarrant force-pushed the 2379/schema_cr branch 2 times, most recently from dd2b22d to 7182043 Compare March 6, 2026 16:44
@tristantarrant

tristantarrant commented Mar 6, 2026

Copy link
Copy Markdown
Member Author

Done

@Crumby

Crumby commented Mar 9, 2026

Copy link
Copy Markdown
Collaborator

E2E tests are completely missing, including integration with the CI

@tristantarrant
tristantarrant force-pushed the 2379/schema_cr branch 3 times, most recently from b0530b6 to 41f9aff Compare March 9, 2026 12:03
@tristantarrant

tristantarrant commented Mar 9, 2026

Copy link
Copy Markdown
Member Author

E2E tests are here now

@tristantarrant tristantarrant mentioned this pull request Mar 9, 2026
Closed
ryanemerson
ryanemerson requested changes Mar 16, 2026
View reviewed changes
Comment thread api/v2alpha1/schema_webhook.go
Comment thread api/v2alpha1/schema_webhook.go
Comment thread controllers/cache_controller.go Outdated
Comment thread controllers/cache_controller.go Outdated
ryanemerson
ryanemerson reviewed Mar 16, 2026
View reviewed changes
Comment thread documentation/asciidoc/topics/con_schemas.adoc
Crumby
Crumby reviewed Mar 16, 2026
View reviewed changes
Comment thread documentation/asciidoc/topics/yaml/cache_schema_ref.yaml Outdated
@tristantarrant
tristantarrant force-pushed the 2379/schema_cr branch 4 times, most recently from 1280641 to 29ec109 Compare March 24, 2026 09:56
@tristantarrant

tristantarrant commented Apr 1, 2026

Copy link
Copy Markdown
Member Author

Schema CRs now have a status flag explaining whether bidi sync is enabled (operand >=16.1)

ryanemerson
ryanemerson approved these changes Apr 1, 2026
View reviewed changes

@ryanemerson ryanemerson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM assuming CI turns 🟢

(The previous run failed with an unrelated rate-limiter issue)

Crumby
Crumby reviewed Apr 1, 2026
View reviewed changes
Comment thread test/e2e/schema/schema_test.go Outdated
Crumby
Crumby reviewed Apr 1, 2026
View reviewed changes
Comment thread config/crd/bases/infinispan.org_caches.yaml
Crumby
Crumby reviewed Apr 1, 2026
View reviewed changes
Comment thread controllers/schema_controller.go Outdated
Crumby
Crumby reviewed Apr 1, 2026
View reviewed changes
Comment thread test/e2e/schema/schema_test.go Outdated
Crumby
Crumby reviewed Apr 1, 2026
View reviewed changes
Comment thread test/e2e/schema/schema_test.go Outdated
@tristantarrant
tristantarrant force-pushed the 2379/schema_cr branch 5 times, most recently from cba40d5 to 50355fa Compare August 20, 2026 09:33
@tristantarrant

tristantarrant commented Aug 20, 2026

Copy link
Copy Markdown
Member Author

Rebased and green

Crumby
Crumby requested changes Aug 24, 2026
View reviewed changes

@Crumby Crumby left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, few requested changes, mainly the usage of generatedName and updating the ServiceAccount docs. I'll leave the rest to your judgment

Comment thread pkg/reconcile/pipeline/infinispan/handler/provision/config_listener.go
Comment thread controllers/schema_controller.go Outdated
Comment thread controllers/cache_controller.go Outdated
Comment thread controllers/schema_controller.go Outdated
Comment thread controllers/schema_controller.go
Comment thread controllers/schema_controller.go Outdated
Crumby
Crumby requested changes Aug 25, 2026
View reviewed changes
Comment thread api/v2alpha1/schema_webhook_test.go Outdated
Comment thread controllers/schema_controller_test.go Outdated
@tristantarrant
tristantarrant force-pushed the 2379/schema_cr branch 4 times, most recently from d1edd04 to faa64fa Compare August 28, 2026 06:30
@tristantarrant

tristantarrant commented Aug 28, 2026

Copy link
Copy Markdown
Member Author

@Crumby it's green

Crumby
Crumby requested changes Aug 28, 2026
View reviewed changes

@Crumby Crumby left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tristantarrant Few more issues around error handling and missing reconciliation pausing feature in the controller.

Comment thread controllers/schema_controller.go Outdated
Comment thread controllers/schema_controller.go Outdated
Comment thread controllers/schema_controller.go
Crumby
Crumby requested changes Aug 28, 2026
View reviewed changes
Comment thread controllers/schema_controller.go Outdated
@tristantarrant
[infinispan#2379] Add Schema CRD for managing Protobuf schemas
dfb9812 
Introduce a new Schema custom resource that allows users to register,
update, and delete Protobuf schemas on Infinispan clusters via the
REST API (/rest/v2/schemas). The implementation follows the same
patterns as the existing Cache CRD including:

- Schema types, webhooks (defaulting + validation), and controller
- REST API client for schema CRUD operations
- Bidirectional sync via SSE event listener (create/update/remove-schema)
- Finalizer-based cleanup on CR deletion
- Webhook tests, type utility tests, and event parsing tests
- E2E tests covering lifecycle, server sync, cluster recreate, and
  cache-schema dependency with indexed protostream caches
- CI workflow job and Makefile target for schema e2e tests
- Asciidoc documentation and sample CR
Crumby
Crumby requested changes Aug 31, 2026
View reviewed changes

@Crumby Crumby left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tristantarrant So increasing the CL annotation wasn't the way to go and probably not even the source of the issue. Another issues popped up

Comment thread controllers/schema_controller.go Outdated
Comment thread controllers/schema_controller.go Outdated
Comment thread controllers/schema_controller.go
@tristantarrant
[infinispan#2379] Schema CR: Operator owns status, CL owns spec
74aa89b 
- Add checkSchemaStatus() that always verifies schema status on the
  server before setting the Ready condition
- Remove status updates from ConfigListener CreateOrUpdate, eliminating
  the race where Operator and CL fight over the Ready condition
- Fix nil map panic in RemoveStaleResources when annotations are nil
- Remove redundant post-registration status re-check from createOrUpdate
  (now handled by checkSchemaStatus in the reconcile loop)
Crumby
Crumby reviewed Aug 31, 2026
View reviewed changes

@Crumby Crumby left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tristantarrant Let's remove the previously added resource/status verbs to Config Listener (and documentation) added previously and we are good to go.

Comment thread pkg/reconcile/pipeline/infinispan/handler/provision/config_listener.go Outdated
Comment thread documentation/asciidoc/topics/yaml/config_listener_rbac.yaml Outdated
tristantarrant and others added 2 commits August 31, 2026 18:11
@tristantarrant @Crumby
Update documentation/asciidoc/topics/yaml/config_listener_rbac.yaml
01e7b8c 
Co-authored-by: Pavel Drobek <drobekpavel@gmail.com>
@tristantarrant @Crumby
Update pkg/reconcile/pipeline/infinispan/handler/provision/config_lis…
d0caf4f 
…tener.go

Co-authored-by: Pavel Drobek <drobekpavel@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ryanemerson ryanemerson ryanemerson approved these changes
@Crumby Crumby Crumby approved these changes

Assignees

No one assigned

Labels

pr/on hold

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Schema CR

3 participants

@tristantarrant @ryanemerson @Crumby