Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Draft] Change store to unparsed_schema and remove upsert validation #25217

Draft
wants to merge 3 commits into
base: dev
Choose a base branch
from

Conversation

IoannisRP
Copy link
Contributor

@IoannisRP IoannisRP commented Feb 28, 2025

Fixed: CORE-9166

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v24.3.x
  • v24.2.x
  • v24.1.x

Release Notes

  • none

@github-actions github-actions bot added area/redpanda area/wasm WASM Data Transforms labels Feb 28, 2025
@IoannisRP IoannisRP force-pushed the CORE-9242/deferred-schema-validation branch 2 times, most recently from 89c4bda to 0f61d25 Compare March 5, 2025 12:51
Copy link
Member

@BenPope BenPope left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@IoannisRP IoannisRP force-pushed the CORE-9242/deferred-schema-validation branch from 0f61d25 to 80c5d8b Compare March 5, 2025 16:47
@IoannisRP IoannisRP self-assigned this Mar 6, 2025
@IoannisRP IoannisRP force-pushed the CORE-9242/deferred-schema-validation branch from 80c5d8b to 3ad2e44 Compare March 6, 2025 17:35
Update /subjects/subject/versions endpoint, for schemas being read
from store to be processed based on the normalization argument.
- Replace canonical schemas for unparsed schemas stored in store and
move all schema processing to sharded_store. This allows to have
lazy schema validation on startup and relax requirements for valid
schema. With this changes, a schema that had been imported before its
dependencies will be found valid when it is requested.

- Add backport to sharded_store to be able to access an unparsed schema
as-is, without processing it. This is needed as now there might be an
invalid schema loaded at startup. Operations like
delete_subject_version, which needs to reference a schema before
deleting it would be problematic.
@IoannisRP IoannisRP force-pushed the CORE-9242/deferred-schema-validation branch from 3ad2e44 to 4e7c1dc Compare March 14, 2025 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/redpanda area/wasm WASM Data Transforms
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants