Allow comparison between embedded java schemas vs apicurio schemas without java specific attributes (avro.java.string)

[jamesHau]

Feature or Problem Description

This feature is related to #1592 .
In our case we are using Kafka together with Apicurio but I believe more cases apply.

When generating java classes from Avro schemas there will be a schema(represented as a string) embedded in the class.
If you generated your java classes using

            <plugin>
                <groupId>org.apache.avro</groupId>
                <artifactId>avro-maven-plugin</artifactId>
                <version>${avro.version}</version>
                <executions>
                    <execution>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>schema</goal>
                        </goals>
                        <configuration>
                            <sourceDirectory>${project.basedir}/src/main/resources/schemas</sourceDirectory>
                            <includes>
                                <include>**/*.avsc</include>
                            </includes>
                            <outputDirectory>${project.build.directory}/generated-sources/schemas</outputDirectory>
                            <stringType>String</stringType>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

The schema in the generated class will have an additional property for fields with the type string. We use this configuration so we can work with Strings in java rather than the default CharSequence.

{
      "name": "name",
      "type": {
        "type": "string",
        "avro.java.string": "String"
      }

This creates an issue when trying to use a schema registry which has a schema registered without language specific properties (without "avro.java.string" property). The matching of schemas won't match and we won't be able to produce a message.

Work-arounds today

• Set EXPLICIT_ARTIFACT_VERSION for every artifact and service you are using. This is cumbersome when updating schemas since you have to update the EXPLICIT_ARTIFACT_VERSION as well
• Set AUTO_REGISTER_ARTIFACT however you will publish schemas with java specific properties. If you are using 1 schema registry you don't want to auto-register and affect production.

Proposed Solution

Introduce apicurio.registry.avro-remove-java-properties which will remove avro.java.string attributes from the schemas when doing lookups towards registry.

It will allow matching between java generated avro classes with embedded schemas vs published schemas which does not have language specific properties. "avro.java.string" will be removed when performing comparison.

Additional Context

• Related to Avro Producer considers "avro.java.string" as part of the schema comparations #1592
• Pull request as an example implementation Enable option to avro.java.string when comparing avro schemas #3911

[apicurio-bot]

Thank you for reporting an issue!

Pinging @carlesarnal to respond or triage.