Skip to content

NIFI-15229 - Downgrade reactor dependencies #10542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Nov 18, 2025
Merged

NIFI-15229 - Downgrade reactor dependencies #10542

merged 2 commits into from
Nov 18, 2025
+21 −0

Conversation

@pvillard31
Copy link
Contributor

@pvillard31 pvillard31 commented Nov 18, 2025

Summary

NIFI-15229 - Downgrade reactor dependencies

Commit from NIFI-15199 bumped io.projectreactor.netty:reactor-netty-http to 1.3.0. That release now eagerly initializes Netty’s HTTP/3 QUIC support, which requires native libraries NiFi does not ship. Every Azure Blob checkpoint call from ConsumeAzureEventHub therefore throws java.lang.NoClassDefFoundError: reactor.netty.http.client.HttpClientSecure.

ConsumeAzureEventHub[id=5d6fb1d3-019a-1000-ffff-ffffb2454981] Processing halted: yielding [1 sec]: java.lang.NoClassDefFoundError: Could not initialize class reactor.netty.http.client.HttpClientSecure
- Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.UnsatisfiedLinkError: failed to load the required native library [in thread "Timer-Driven Process Thread-3"]

java.lang.NoClassDefFoundError: Could not initialize class reactor.netty.http.client.HttpClientSecure
	at reactor.netty.http.client.HttpClientConnect$MonoHttpConnect.lambda$subscribe$0(HttpClientConnect.java:226)
	at reactor.core.publisher.MonoCreate.subscribe(MonoCreate.java:61)
	at reactor.core.publisher.FluxRetryWhen.subscribe(FluxRetryWhen.java:81)
	at reactor.core.publisher.MonoRetryWhen.subscribeOrReturn(MonoRetryWhen.java:46)
	at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:63)
	at reactor.netty.http.client.HttpClientConnect$MonoHttpConnect.subscribe(HttpClientConnect.java:300)
	at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
	at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
	at reactor.core.publisher.Mono.subscribe(Mono.java:4576)
	at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:205)
	at reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
	at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:63)
	at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
	at reactor.core.publisher.Mono.subscribe(Mono.java:4576)
	at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:205)
	at reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
	at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:63)
	at reactor.core.publisher.MonoDeferContextual.subscribe(MonoDeferContextual.java:55)
	at reactor.core.publisher.Mono.subscribe(Mono.java:4576)
	at reactor.core.publisher.Mono.block(Mono.java:1778)
	at org.apache.nifi.processors.azure.eventhub.position.LegacyBlobStorageEventPositionProvider.containerExists(LegacyBlobStorageEventPositionProvider.java:145)
	at org.apache.nifi.processors.azure.eventhub.position.LegacyBlobStorageEventPositionProvider.getInitialPartitionEventPosition(LegacyBlobStorageEventPositionProvider.java:72)
	at org.apache.nifi.processors.azure.eventhub.ConsumeAzureEventHub.getLegacyPartitionEventPosition(ConsumeAzureEventHub.java:859)
	at org.apache.nifi.processors.azure.eventhub.ConsumeAzureEventHub.createClient(ConsumeAzureEventHub.java:574)
	at org.apache.nifi.processors.azure.eventhub.ConsumeAzureEventHub.onTrigger(ConsumeAzureEventHub.java:524)
	at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1274)
	at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:229)
	at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
	at org.apache.nifi.engine.FlowEngine.lambda$wrap$1(FlowEngine.java:105)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.UnsatisfiedLinkError: failed to load the required native library [in thread "Timer-Driven Process Thread-3"]
	at io.netty.handler.codec.quic.Quic.ensureAvailability(Quic.java:87)
	at io.netty.handler.codec.quic.QuicheQuicSslContext.<init>(QuicheQuicSslContext.java:160)
	at io.netty.handler.codec.quic.QuicSslContextBuilder.build(QuicSslContextBuilder.java:404)
	at reactor.netty.http.Http3SslContextSpec.sslContext(Http3SslContextSpec.java:105)
	at reactor.netty.tcp.SslProvider.<init>(SslProvider.java:343)
	at reactor.netty.tcp.SslProvider$Build.build(SslProvider.java:650)
	at reactor.netty.http.client.HttpClientSecure.<clinit>(HttpClientSecure.java:82)
	... 35 more

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using ./mvnw clean install -P contrib-check
    • JDK 21
    • JDK 25

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

@pvillard31
Copy link
Contributor Author

pvillard31 commented Nov 18, 2025

As a side note, another option would be to hide HTTP/3 from Reactor. Meaning exclude io.netty:netty-codec-http3 and io.netty:netty-codec-classes-quic from the Azure SDK dependencies so that Http3.isHttp3Available() never finds the marker class. Reactor 1.3.0 would immediately fall back to HTTP/1.1+HTTP/2, so there is no QUIC bootstrap and Blob checkpoints work. Thoughts @exceptionfactory ?

@pvillard31
Copy link
Contributor Author

pvillard31 commented Nov 18, 2025

failures are due to the cloudflare worldwide issue

@exceptionfactory
Copy link
Contributor

exceptionfactory commented Nov 18, 2025

As a side note, another option would be to hide HTTP/3 from Reactor. Meaning exclude io.netty:netty-codec-http3 and io.netty:netty-codec-classes-quic from the Azure SDK dependencies so that Http3.isHttp3Available() never finds the marker class. Reactor 1.3.0 would immediately fall back to HTTP/1.1+HTTP/2, so there is no QUIC bootstrap and Blob checkpoints work. Thoughts @exceptionfactory ?

Thanks for the background @pvillard31.

Excluding the netty-codec-http3 sounds like a more focused option for now, versus downgrading.

Adding the native libraries is probably the ultimate path forward, but that would require a bit more evaluation.

@pvillard31
Copy link
Contributor Author

pvillard31 commented Nov 18, 2025

OK, thanks @exceptionfactory - I made the change.

In the NAR, before:

  META-INF/bundled-dependencies/azure-core-http-netty-1.16.2.jar
  META-INF/bundled-dependencies/netty-codec-base-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-codec-classes-quic-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-codec-compression-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-codec-dns-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-codec-http-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-codec-http2-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-codec-http3-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-codec-native-quic-4.2.7.Final-linux-aarch_64.jar
  META-INF/bundled-dependencies/netty-codec-native-quic-4.2.7.Final-linux-x86_64.jar
  META-INF/bundled-dependencies/netty-codec-native-quic-4.2.7.Final-osx-aarch_64.jar
  META-INF/bundled-dependencies/netty-codec-native-quic-4.2.7.Final-osx-x86_64.jar
  META-INF/bundled-dependencies/netty-codec-native-quic-4.2.7.Final-windows-x86_64.jar
  META-INF/bundled-dependencies/netty-codec-socks-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-handler-proxy-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-resolver-dns-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-resolver-dns-classes-macos-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-resolver-dns-native-macos-4.2.7.Final-osx-x86_64.jar
  META-INF/bundled-dependencies/netty-tcnative-boringssl-static-2.0.74.Final-linux-aarch_64.jar
  META-INF/bundled-dependencies/netty-tcnative-boringssl-static-2.0.74.Final-linux-x86_64.jar
  META-INF/bundled-dependencies/netty-tcnative-boringssl-static-2.0.74.Final-osx-aarch_64.jar
  META-INF/bundled-dependencies/netty-tcnative-boringssl-static-2.0.74.Final-osx-x86_64.jar
  META-INF/bundled-dependencies/netty-tcnative-boringssl-static-2.0.74.Final-windows-x86_64.jar
  META-INF/bundled-dependencies/netty-tcnative-boringssl-static-2.0.74.Final.jar
  META-INF/bundled-dependencies/netty-tcnative-classes-2.0.74.Final.jar
  META-INF/bundled-dependencies/netty-transport-classes-epoll-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-transport-classes-kqueue-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-transport-native-epoll-4.2.7.Final-linux-x86_64.jar
  META-INF/bundled-dependencies/netty-transport-native-kqueue-4.2.7.Final-osx-x86_64.jar
  META-INF/bundled-dependencies/reactor-core-3.8.0.jar
  META-INF/bundled-dependencies/reactor-netty-core-1.3.0.jar
  META-INF/bundled-dependencies/reactor-netty-http-1.3.0.jar

Now:

  META-INF/bundled-dependencies/azure-core-http-netty-1.16.2.jar
  META-INF/bundled-dependencies/netty-codec-base-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-codec-compression-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-codec-dns-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-codec-http-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-codec-http2-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-codec-socks-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-handler-proxy-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-resolver-dns-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-resolver-dns-classes-macos-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-resolver-dns-native-macos-4.2.7.Final-osx-x86_64.jar
  META-INF/bundled-dependencies/netty-tcnative-boringssl-static-2.0.74.Final-linux-aarch_64.jar
  META-INF/bundled-dependencies/netty-tcnative-boringssl-static-2.0.74.Final-linux-x86_64.jar
  META-INF/bundled-dependencies/netty-tcnative-boringssl-static-2.0.74.Final-osx-aarch_64.jar
  META-INF/bundled-dependencies/netty-tcnative-boringssl-static-2.0.74.Final-osx-x86_64.jar
  META-INF/bundled-dependencies/netty-tcnative-boringssl-static-2.0.74.Final-windows-x86_64.jar
  META-INF/bundled-dependencies/netty-tcnative-boringssl-static-2.0.74.Final.jar
  META-INF/bundled-dependencies/netty-tcnative-classes-2.0.74.Final.jar
  META-INF/bundled-dependencies/netty-transport-classes-epoll-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-transport-classes-kqueue-4.2.7.Final.jar
  META-INF/bundled-dependencies/netty-transport-native-epoll-4.2.7.Final-linux-x86_64.jar
  META-INF/bundled-dependencies/netty-transport-native-kqueue-4.2.7.Final-osx-x86_64.jar
  META-INF/bundled-dependencies/reactor-core-3.8.0.jar
  META-INF/bundled-dependencies/reactor-netty-core-1.3.0.jar
  META-INF/bundled-dependencies/reactor-netty-http-1.3.0.jar

exceptionfactory
exceptionfactory approved these changes Nov 18, 2025
View reviewed changes
Copy link
Contributor

@exceptionfactory exceptionfactory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adjusting the approach @pvillard31, excluding the netty-codec-http3 dependencies looks like the best way forward right now.

@exceptionfactory exceptionfactory merged commit fd7c412 into apache:main Nov 18, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@exceptionfactory exceptionfactory exceptionfactory approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@pvillard31 @exceptionfactory