Skip to content

[fix][schema] Reject unsupported Avro schema types during schema registration #24103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

poorbarcode
Copy link
Contributor

@poorbarcode poorbarcode commented Mar 21, 2025

Motivation

Issue

  • User can upload an Avro-Union schema by pulsar-admin API
  • However, the schema uploaded can not be used, users will get an error "Not a record" when creating a producer with AUTO_PRODUCE schema.

Modifications

  • Deny uploading an Avro-Union schema
  • Improve the logs when creating a producer failed due to an invalidated schema

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: x

@poorbarcode poorbarcode added type/bug The PR fixed a bug or issue reported a bug release/4.0.4 release/3.0.11 release/3.3.6 labels Mar 21, 2025
@poorbarcode poorbarcode added this to the 4.1.0 milestone Mar 21, 2025
@poorbarcode poorbarcode self-assigned this Mar 21, 2025
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Mar 21, 2025
@poorbarcode poorbarcode requested review from lhotari and removed request for lhotari March 26, 2025 01:22
@poorbarcode poorbarcode requested review from lhotari and removed request for lhotari March 26, 2025 10:03
Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, good work @poorbarcode

@lhotari lhotari changed the title [fix][schema]Pulsar does not support avro-union schema, but it allows it to be registered [fix][schema] Reject unsupported Avro schema types during schema registration Mar 26, 2025
@lhotari
Copy link
Member

lhotari commented Mar 26, 2025

I renamed the title to "Reject unsupported Avro schema types during schema registration" since good PR titles are optimally expressed in imperative mood, like good commit messages. The PR title shouldn't be a description of the problem it's fixing. It might be better to file a separate issue where that could be a title of the issue itself.

@poorbarcode
Copy link
Contributor Author

/pulsarbot rerun-failure-checks

@poorbarcode
Copy link
Contributor Author

/pulsarbot rerun-failure-checks

@lhotari
Copy link
Member

lhotari commented Apr 9, 2025

There's a test failure:

  Error:  Tests run: 175, Failures: 1, Errors: 0, Skipped: 165, Time elapsed: 46.633 s <<< FAILURE! - in org.apache.pulsar.broker.admin.AdminApiSchemaTest
  Error:  org.apache.pulsar.broker.admin.AdminApiSchemaTest.testInvalidSchemaDataException[V1](4)  Time elapsed: 0.01 s  <<< FAILURE!
  java.lang.AssertionError: expected [422] but found [500]
  	at org.testng.Assert.fail(Assert.java:110)
  	at org.testng.Assert.failNotEquals(Assert.java:1577)
  	at org.testng.Assert.assertEqualsImpl(Assert.java:149)
  	at org.testng.Assert.assertEquals(Assert.java:131)
  	at org.testng.Assert.assertEquals(Assert.java:1418)
  	at org.testng.Assert.assertEquals(Assert.java:1382)
  	at org.testng.Assert.assertEquals(Assert.java:1428)
  	at org.apache.pulsar.broker.admin.AdminApiSchemaTest.testInvalidSchemaDataException(AdminApiSchemaTest.java:282)
  	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
  	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
  	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
  	at org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47)
  	at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76)
  	at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
  	at java.base/java.lang.Thread.run(Thread.java:1583)

It seems that 422 Unprocessable Content status is expected.

@github-actions github-actions bot added the PIP label Apr 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc-not-needed Your PR changes do not impact docs PIP ready-to-test release/3.0.12 release/3.3.7 release/4.0.5 type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants