Skip to content

[build] fluss-fs-s3 uber-jar bundles 1,114 unshaded jackson classes that can shadow downstream jackson #4072

Description

@Jackeyzhe

Search before asking

  • I searched in the issues and found nothing similar.

Fluss version

main (development)

Please describe the bug 🐞

Describe the bug

The fluss-fs-s3 uber-jar bundles hadoop-common and aws-java-sdk-*
dependencies without any shade relocations. As a result, 1,114 unshaded
jackson class files leak into the jar at com/fasterxml/* paths:

Package Approx. count Source
com/fasterxml/jackson/databind/** ~800 hadoop-common
com/fasterxml/jackson/core/** ~200 hadoop-common
com/fasterxml/jackson/annotation/** ~20 hadoop-common
com/fasterxml/jackson/dataformat/cbor/** ~50 aws-java-sdk-core
META-INF/versions/{11,17,19}/com/fasterxml/** (MRJ) ~25 jackson-core MRJ

When a downstream application puts fluss-fs-s3 on its classpath alongside its
own jackson-core (e.g. 2.16+), the leaked classes can shadow the application's
jackson and cause runtime failures such as NoSuchMethodError — the same
failure mode reported in #3553 for the MRJ entries, but here affecting the full
jackson-databind/core/annotations/cbor surface, not just MRJ entries.

In contrast, fluss-fs-hadoop-shaded already relocates these same packages
(com.fasterxml, org.codehaus, com.ctc, com.google.re2j,
org.apache.htrace) into org.apache.fluss.fs.shaded.hadoop3.*.

To Reproduce

./mvnw -pl fluss-filesystems/fluss-fs-s3 -am -DskipTests package
unzip -l fluss-filesystems/fluss-fs-s3/target/fluss-fs-s3-*.jar | grep -c " com/fasterxml"
# -> 1114

Expected behavior

All bundled third-party dependencies should be relocated under Fluss's shaded
namespace (mirroring the fluss-fs-hadoop-shaded pattern), so the uber-jar
never exposes com/fasterxml/* classes on the classpath.

Additional context

Solution

Mirror the fluss-fs-hadoop-shaded approach: add maven-shade-plugin
relocations for com.fasterxml.jackson.*, com.fasterxml.jackson.databind.*,
com.fasterxml.jackson.dataformat.cbor.* to org.apache.fluss.fs.shaded.s3.*,
and add a <filter> to exclude META-INF/versions/*/com/fasterxml/** (MRJ
entries are not relocated by the shade plugin). A draft PR is in progress.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions