Skip to content

add an integration test with extension diretories as symlinks so that we can validate the version bump for commons-io #11393

Open
@zachjsh

Description

@zachjsh

An extension loading issue was found when using commons-io version 2.9.0. In particular When the an extension directory is configured as a symlink, the symlink is not walked properly, and no jars are found, resulting in the extension not being loaded properly. An example of such a configuration for the druid-hdfs-storage extension is shown below:

  % pwd
  /Users/user1/apache-druid-0.22.0-SNAPSHOT/extensions
  %
  % ls -al
  total 0
  drwxr-xr-x  29 user1 staff   928 Jun 29 12:38 .
  drwxr-xr-x  12 user1  staff   384 Jun 29 12:35 ..
  drwxr-xr-x  28 user1  staff   896 Jun 29 12:35 druid-avro-extensions
  drwxr-xr-x  13 user1  staff   416 Jun 29 12:35 druid-aws-rds-extensions
  drwxr-xr-x  10 user1  staff   320 Jun 29 12:35 druid-azure-extensions
  drwxr-xr-x   3 user1  staff    96 Jun 29 12:35 druid-basic-security
  drwxr-xr-x   5 user1  staff   160 Jun 29 12:35 druid-bloom-filter
  drwxr-xr-x   4 user1  staff   128 Jun 29 12:35 druid-datasketches
  drwxr-xr-x   3 user1 staff    96 Jun 29 12:35 druid-ec2-extensions
  drwxr-xr-x   4 user1 staff   128 Jun 29 12:35 druid-google-extensions
  lrwxr-xr-x   1 user1 staff    49 Jun 29 12:38 druid-hdfs-storage -> /Users/user1/druid-hdfs-storage
  drwxr-xr-x   3 user1  staff    96 Jun 29 12:35 druid-histogram
  drwxr-xr-x  10 user1  staff   320 Jun 29 12:35 druid-kafka-extraction-namespace
  drwxr-xr-x   7 user1  staff   224 Jun 29 12:35 druid-kafka-indexing-service
  drwxr-xr-x  11 user1  staff   352 Jun 29 12:35 druid-kerberos
  drwxr-xr-x   6 user1  staff   192 Jun 29 12:35 druid-kinesis-indexing-service
  drwxr-xr-x  38 user1  staff  1216 Jun 29 12:35 druid-kubernetes-extensions
  drwxr-xr-x   5 user1  staff   160 Jun 29 12:35 druid-lookups-cached-global
  drwxr-xr-x   7 user1  staff   224 Jun 29 12:35 druid-lookups-cached-single
  drwxr-xr-x  20 user1  staff   640 Jun 29 12:35 druid-orc-extensions
  drwxr-xr-x  15 user1  staff   480 Jun 29 12:35 druid-pac4j
  drwxr-xr-x  24 user1  staff   768 Jun 29 12:35 druid-parquet-extensions
  drwxr-xr-x   3 user1  staff    96 Jun 29 12:35 druid-protobuf-extensions
  drwxr-xr-x  73 user1  staff  2336 Jun 29 12:35 druid-ranger-security
  drwxr-xr-x  13 user1  staff   416 Jun 29 12:35 druid-s3-extensions
  drwxr-xr-x   4 user1  staff   128 Jun 29 12:35 druid-stats
  drwxr-xr-x   3 user1  staff    96 Jun 29 12:35 mysql-metadata-storage
  drwxr-xr-x   4 user1  staff   128 Jun 29 12:35 postgresql-metadata-storage
  drwxr-xr-x   3 user1  staff    96 Jun 29 12:35 simple-client-sslcontext
  %

An issue was filed with the apache commons-io team https://issues.apache.org/jira/browse/IO-741.

We should add an integration test which tests this scenario so that we can bump the version of this dependency in the future with confidence.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions