Various issues migrating apicurio-registry-maven-plugin v2 to v3

[creckord]

Description

Registry Version: 3.0.4
Persistence type: sql (Postgres 15)

As part of migrating our Apicurio Registry over to v3, I am trying to migrate our Maven builds from apicurio-registry-maven-plugin v2 to v3 and running into various issues. I'll collect them here at first, as to not spam the tracker. But feel free to request separate reports if they seem warranted.

Environment

Windows 11
Maven 3.9.9
JDK 21.0.5 (Temurin 21.0.5+11-LTS)
apicurio-registry-maven-plugin 3.0.4

1. Missing documentation of plugin changes

Just a minor thing, but I found no good documentation on the changes in the Maven plugin (side note, a reference of the plugin's configuration options and goals in the typical maven plugin documentation style would be awesome - the documentation under https://www.apicur.io/registry/docs/apicurio-registry/3.0.x/getting-started/assembly-managing-registry-artifacts-maven.html is rather spotty), and it isn't covered in the migration guide at all.

Some things I stumbled over:

• <type> is <artifactType> now
• <registryUrl> now has to point to /apis/registry/v3 instead of the server root
• <ifExists> enum values have changed

2. Mismatch on existing schemas

I have successfully migrated my schemas over from the v2 registry following the migration guide.

However, when I now try to re-register the exact, unchanged version of an artifact again, I get a 409 VersionAlreadyExistsException. Doing the same with the v2 client against the v2 server is successful. This seems to be the case for all artifacts that have references.

Artifact definition:

    <artifact>
        <groupId>default</groupId>
        <artifactId>MyExampleEvent</artifactId>
        <version>9</version>
        <artifactType>AVRO</artifactType>
        <file>
            ${project.build.directory}/avro/MyExampleEvent.avsc
        </file>
        <ifExists>FIND_OR_CREATE_VERSION</ifExists>
        <canonicalize>true</canonicalize>
        <references>
            <reference>
                <groupId>default</groupId>
                <artifactId>MyExampleReference</artifactId>
                <version>3</version>
                <artifactType>AVRO</artifactType>
                <ifExists>FIND_OR_CREATE_VERSION</ifExists>
                <canonicalize>true</canonicalize>
                <file>
                    ${project.build.directory}/avro/MyExampleReference.avsc
                </file>
                <name>
                    org.example.MyExampleReference
                </name>
            </reference>
        </references>
    </artifact>

Due to #5627 I wasn't able to verify what would happen if I used the v2 plugin against the v3 server. But as I understand, canonicalization and content matching is done completely server-side, so I would expect it's actually the same.

3. Rule violation causes not visible

When the upload fails due to a rule violation (e.g. the aforementioned VersionAlreadyExistsException), the only information I get is a generic "rule violation" message devoid of any information as to the rule or the cause of its violation:

[ERROR] Exception while registering artifact [default] / [MyExampleEvent]
io.apicurio.registry.rest.client.models.RuleViolationProblemDetails
    at io.apicurio.registry.rest.client.models.RuleViolationProblemDetails.createFromDiscriminatorValue (RuleViolationProblemDetails.java:62)
    at io.kiota.serialization.json.JsonParseNode.getObjectValue (JsonParseNode.java:247)
    at io.kiota.http.vertx.VertXRequestAdapter.lambda$throwIfFailedResponse$0 (VertXRequestAdapter.java:400)
    ...

I had to debug into the plugin to find out which rule was violated here. Not sure if it's the same for violations other than VersionAlreadyExists.

4. Vert.x not shut down cleanly

Not sure if this is caused by the preceding upload errors, but the Maven build always ends with an exception that is apparently caused by Vert.x still running in the background while the plugin's class realm is torn down:

Exception in thread "Thread-4" java.lang.NoClassDefFoundError: io/vertx/core/file/impl/FileSystemImpl$11
        at io.vertx.core.file.impl.FileSystemImpl.delete(FileSystemImpl.java:898)
        at io.vertx.core.file.impl.FileCache.deleteCacheDir(FileCache.java:140)
        at io.vertx.core.file.impl.FileCache.lambda$runHook$0(FileCache.java:92)
        at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.ClassNotFoundException: io.vertx.core.file.impl.FileSystemImpl$11
        at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:42)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:225)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:210)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:205)
        ... 4 more

5. Vert.x incompatibility on Windows

More of an fyi, but I wasn't able to use the v3 plugin at all at first, because the netty client used under the hood wasn't able to configure the correct DNS servers from the OS. This seems to be a problem of running Vert.x inside of Maven, because outside of it, this works perfectly.

I filed codehaus-plexus/plexus-classworlds#119 for this one.

Workaround was: MAVEN_OPTS="--add-opens jdk.naming.dns/com.sun.jndi.dns=java.naming"

[constantinpopa10]

Regarding point 3. Rule violation causes not visible I noticed the same issue regarding compatibility rule checks, namely the violation reason is missing from the client response, instead is hidden in the server logs.

For example if I'm trying to register a new version for a json schema that is incompatible with the previous version the maven plugin returns the following response:

io.apicurio.registry.rest.client.models.RuleViolationProblemDetails
    at io.apicurio.registry.rest.client.models.RuleViolationProblemDetails.createFromDiscriminatorValue (RuleViolationProblemDetails.java:62)
    at io.kiota.serialization.json.JsonParseNode.getObjectValue (JsonParseNode.java:247)
    at io.kiota.http.vertx.VertXRequestAdapter.lambda$throwIfFailedResponse$0 (VertXRequestAdapter.java:400)
    at com.microsoft.kiota.ApiExceptionBuilder.<init> (ApiExceptionBuilder.java:26)
    at io.kiota.http.vertx.VertXRequestAdapter.throwIfFailedResponse (VertXRequestAdapter.java:399)
    at io.kiota.http.vertx.VertXRequestAdapter.send (VertXRequestAdapter.java:170)
    at io.apicurio.registry.rest.client.groups.item.artifacts.ArtifactsRequestBuilder.post (ArtifactsRequestBuilder.java:127)
    at io.apicurio.registry.maven.RegisterRegistryMojo.registerArtifact (RegisterRegistryMojo.java:340)
    at io.apicurio.registry.maven.RegisterRegistryMojo.registerArtifact (RegisterRegistryMojo.java:277)

Whereas the server logs contain the actual violation message:

apicurio-registry-1 | 2024-12+00-03 15:32:46 INFO [io.apicurio.registry.logging.audit.AuditLogService] (executor-thread-120) apicurio.audit action="createArtifact" result="failure" src_ip="172.22.0.1" if_exists="FIND_OR_CREATE_VERSION" error_msg="Incompatible artifact: JsonArtifact [JSON], num of incompatible diffs: {1}, list of diff types: [OBJECT_TYPE_REQUIRED_PROPERTIES_MEMBER_ADDED at /required] Causes: OBJECT_TYPE_REQUIRED_PROPERTIES_MEMBER_ADDED at /required" group_id="mdmsfx" canonical="true"

[EricWittmann]

Thanks for the feedback! This is extremely helpful. 🙏

[EricWittmann]

I'll fix (3) today, since that's easy - just need to unwrap the problem object and output the results in a friendly way. That is one of the annoying things about the new code-generated clients: they do not propagate the errors very well by default. That said, the benefits of the new SDK approach far outweigh the annoyances. :)

I think I can also fix both (4) and (5) by not using the vertx handler. I can just use the standard JDK handler instead. I'll give that a try.

[EricWittmann]

So it turns out that I was wrong and the JDK handler is only meant for demos. So no joy there. Instead I have updated the Mojo impl to properly close the Vertx object (didn't realize this was necessary). The PR is here:

• fix(vertx-client): Make sure to close the Vertx instance in the maven plugin #5676

That only fixes (4) I think.

I didn't have time to work on (3) but will hopefully do it tomorrow.

[EricWittmann]

Looks like properly closing the vertx instance does nothing to help with (4). I'm going to break that one out into a separate issue and link it here.

[EricWittmann]

• Maven Plugin : Vert.x not shut down cleanly #5683

[EricWittmann]

I've done a bit of work on (3) and the new maven plugin output will look like this (for a Validity rule violation):

[INFO] --------------< io.apicurio.issues:issue-5683-reproducer >--------------
[INFO] Building reproducer 1
[INFO]   from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO] 
[INFO] --- apicurio-registry:3.0.6-SNAPSHOT:register (register-artifact) @ issue-5683-reproducer ---
[ERROR] ---
[ERROR] Registry rule validation failure: RuleViolationException
[ERROR] The OpenAPI artifact is not semantically valid. 3 problems found.
[ERROR] 	-> /components/schemas[VersionMetaData]/properties[type]
[ERROR] 	   Schema Reference must refer to a valid Schema Definition.
[ERROR] 	-> /paths[/ids/{globalId}]/parameters[0]
[ERROR] 	   Path Parameter "localId" not found in path template.
[ERROR] 	-> /paths[/ids/{globalId}]/get
[ERROR] 	   No definition found for path variable "globalId" for path '/ids/{globalId}' and method 'GET'.
[ERROR] ---
[ERROR] Exception while registering artifact [OpenAPI] / [RegistryAPI]
org.apache.maven.plugin.MojoFailureException: Registry rule validation failure: RuleViolationException
    at io.apicurio.registry.maven.AbstractRegistryMojo.logAndThrow (AbstractRegistryMojo.java:177)
    at io.apicurio.registry.maven.AbstractRegistryMojo.logAndThrow (AbstractRegistryMojo.java:150)
    at io.apicurio.registry.maven.RegisterRegistryMojo.registerArtifact (RegisterRegistryMojo.java:380)

Obviously a big improvement. Interested in additional feedback once others have a chance to try it out.

In addition, I've added a dryRun option to the register mojo and deprecated the test-update mojo. This is more consistent with Registry 3 semantics, and it makes it easier to maintain going forward.