Skip to content

Upstream: 295e7014fbd6e2b9b3d7075497daf95d6ca53e23#763

Merged
kgyrtkirk merged 6 commits intomasterfrom
up-295e7014fbd6e2b9b3d7075497daf95d6ca53e23
Apr 2, 2025
Merged

Upstream: 295e7014fbd6e2b9b3d7075497daf95d6ca53e23#763
kgyrtkirk merged 6 commits intomasterfrom
up-295e7014fbd6e2b9b3d7075497daf95d6ca53e23

Conversation

@kgyrtkirk
Copy link
Copy Markdown
Owner

No description provided.

kfaraz and others added 6 commits April 1, 2025 11:42
* Introduce overlordNamespace label to filter out task pods from other namespaces

* Fixing changed unit tests

* Fix all unit tests in kubernetes overlord extension

* Add table to specify new configs

* Table with correct formatting

* Documentation for allowing Druid to run overlord in seperate namespaces

* Apply suggestions from code review

Co-authored-by: Frank Chen <frankchen@apache.org>

* Change Constant name to druid.overlord.namespace

* Fix checkstyle

* Documentation to clearer explain druid.indexer.runner.overlordNamespace

* Only support deployment to other namespaces using Pod Template Adapter

* Edit documentation to reflect only requiring using custom pod template adapter for deploying jobs in other namespaces

* Remove unused imports

---------

Co-authored-by: Frank Chen <frankchen@apache.org>
Co-authored-by: asishupadhyay <akulabs8@gmail.com>
* Improve S3 upload speeds using aws transfer manager

* Pass correct amazonS3Client to ServerSideEncryptingAmazonS3

* Add Unit Test Cases

* Turn on transfer manager by default

* Add Druid documentation
private MockAmazonS3Client()
{
super(new AmazonS3Client(), new NoopServerSideEncryption());
super(new AmazonS3Client(), new NoopServerSideEncryption(), new S3TransferConfig());

Check notice

Code scanning / CodeQL

Deprecated method or constructor invocation Note test

Invoking
AmazonS3Client.AmazonS3Client
should be avoided because it has been deprecated.

Copilot Autofix

AI about 1 year ago

To fix the problem, we need to replace the deprecated AmazonS3Client constructor with the recommended alternative. According to the AWS SDK documentation, the AmazonS3ClientBuilder should be used to create an instance of AmazonS3Client. This change will ensure that the code is using the latest and supported method for creating an AmazonS3Client instance.

  1. Replace the new AmazonS3Client() call with AmazonS3ClientBuilder.standard().build().
  2. Ensure that the necessary import for AmazonS3ClientBuilder is added.
Suggested changeset 1
extensions-core/s3-extensions/src/test/java/org/apache/druid/storage/s3/S3DataSegmentMoverTest.java

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/extensions-core/s3-extensions/src/test/java/org/apache/druid/storage/s3/S3DataSegmentMoverTest.java b/extensions-core/s3-extensions/src/test/java/org/apache/druid/storage/s3/S3DataSegmentMoverTest.java
--- a/extensions-core/s3-extensions/src/test/java/org/apache/druid/storage/s3/S3DataSegmentMoverTest.java
+++ b/extensions-core/s3-extensions/src/test/java/org/apache/druid/storage/s3/S3DataSegmentMoverTest.java
@@ -21,3 +21,3 @@
 
-import com.amazonaws.services.s3.AmazonS3Client;
+import com.amazonaws.services.s3.AmazonS3ClientBuilder;
 import com.amazonaws.services.s3.model.AccessControlList;
@@ -203,3 +203,3 @@
     {
-      super(new AmazonS3Client(), new NoopServerSideEncryption(), new S3TransferConfig());
+      super(AmazonS3ClientBuilder.standard().build(), new NoopServerSideEncryption(), new S3TransferConfig());
     }
EOF
@@ -21,3 +21,3 @@

import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.AccessControlList;
@@ -203,3 +203,3 @@
{
super(new AmazonS3Client(), new NoopServerSideEncryption(), new S3TransferConfig());
super(AmazonS3ClientBuilder.standard().build(), new NoopServerSideEncryption(), new S3TransferConfig());
}
Copilot is powered by AI and may make mistakes. Always verify output.
@kgyrtkirk kgyrtkirk merged commit 295e701 into master Apr 2, 2025
75 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants