Skip to content

feat:[PEA-2706] Refresh credentials implementation for data shipper #2407

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
arunpaladin merged 1 commit into from
Aug 5, 2026
Merged

feat:[PEA-2706] Refresh credentials implementation for data shipper #2407

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 27 additions & 29 deletions ...a-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/config/CredentialProvider.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,7 @@
******************************************************************************/
package com.tmobile.cso.pacman.datashipper.config;

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.BasicSessionCredentials;
import com.amazonaws.auth.*;
import com.amazonaws.services.securitytoken.AWSSecurityTokenService;
import com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClientBuilder;
import com.amazonaws.services.securitytoken.model.AssumeRoleRequest;
Expand All @@ -35,47 +33,47 @@ public class CredentialProvider {
* @param roleName the role name
* @return the credentials
*/
public BasicSessionCredentials getCredentials(String account, String roleName) {
BasicSessionCredentials baseAccntCreds = getBaseAccountCredentials(roleName);
public AWSCredentialsProvider getCredentials(String account, String roleName) {
AWSCredentialsProvider baseProvider = getBaseAccountCredentials(roleName);
if (baseAccount.equals(account)) {
return baseAccntCreds;
return baseProvider;
}
AWSSecurityTokenServiceClientBuilder stsBuilder = AWSSecurityTokenServiceClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(baseAccntCreds)).withRegion(baseRegion);
AWSSecurityTokenService stsClient = stsBuilder.build();
AssumeRoleRequest assumeRequest = new AssumeRoleRequest().withRoleArn(getRoleArn(account, roleName)).withRoleSessionName("pic-ro-" + account).withDurationSeconds(3600);
AssumeRoleResult assumeResult = stsClient.assumeRole(assumeRequest);
return new BasicSessionCredentials(
assumeResult.getCredentials()
.getAccessKeyId(), assumeResult.getCredentials().getSecretAccessKey(),
assumeResult.getCredentials().getSessionToken());
AWSSecurityTokenService stsClient = AWSSecurityTokenServiceClientBuilder.standard()
.withCredentials(baseProvider)
.withRegion(baseRegion)
.build();
return new STSAssumeRoleSessionCredentialsProvider.Builder(
getRoleArn(account, roleName), "pic-ro-" + account)
.withStsClient(stsClient)
.build();
}


/**
* Gets the base account credentials.
*
* @param roleName the role name
* @return the base account credentials
*/
private BasicSessionCredentials getBaseAccountCredentials(String roleName) {
private AWSCredentialsProvider getBaseAccountCredentials(String roleName) {
if (devMode) {
String accessKey = System.getProperty("ACCESS_KEY");
String secretKey = System.getProperty("SECRET_KEY");
BasicAWSCredentials awsCreds = new BasicAWSCredentials(accessKey, secretKey);
AWSSecurityTokenServiceClientBuilder stsBuilder = AWSSecurityTokenServiceClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(awsCreds)).withRegion(baseRegion);
AWSSecurityTokenService sts = stsBuilder.build();
AssumeRoleRequest assumeRequest = new AssumeRoleRequest().withRoleArn(getRoleArn(baseAccount, roleName)).withRoleSessionName("pic-base-ro").withDurationSeconds(3600);
AssumeRoleResult assumeResult = sts.assumeRole(assumeRequest);
return new BasicSessionCredentials(
assumeResult.getCredentials().getAccessKeyId(), assumeResult.getCredentials().getSecretAccessKey(),
assumeResult.getCredentials().getSessionToken());

AWSSecurityTokenService sts = AWSSecurityTokenServiceClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(
new BasicAWSCredentials(accessKey, secretKey)))
.withRegion(baseRegion)
.build();
return new STSAssumeRoleSessionCredentialsProvider.Builder(
getRoleArn(baseAccount, roleName), "pic-base-ro")
.withStsClient(sts)
.build();
} else {
AWSSecurityTokenService sts = AWSSecurityTokenServiceClientBuilder.defaultClient();

@coderabbitai coderabbitai Bot Aug 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect whether deployment and application configuration provide an AWS SDK
# default-region setting in addition to the base.region system property.
rg -n -C2 'base\.region|AWS_REGION|aws\.region|AWS_DEFAULT_REGION' .

Repository: PaladinCloud/CE

Length of output: 50371

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== files in jobs/pacman-data-shipper =="
git ls-files jobs/pacman-data-shipper | sed -n '1,200p'

echo
echo "== credential provider outline && contents =="
sed -n '1,220p' jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/config/CredentialProvider.java

echo
echo "== dependency/version references (focused) =="
rg -n -C2 'AWSSecurityTokenService|AWSSecurityTokenServiceClientBuilder|sts|base\.region|AWS_REGION|aws\.region|AWS_DEFAULT_REGION|SYSTEM_PROPERTY|Property' jobs/pacman-data-shipper --glob '*.java' --glob '*.yml' --glob '*.yaml' --glob '*properties'

echo
echo "== build config =="
for f in jobs/pacman-data-shipper/pom.xml jobs/pacman-data-shipper/build.gradle jobs/pacman-data-shipper/build.gradle.kts; do
  test -f "$f" && { echo "--- $f"; sed -n '1,220p' "$f"; }
done

echo
echo "== all AWS region env/system mapping in pacman-data-shipper =="
rg -n -C2 'base\.region|AWS_REGION|aws\.region|AWS_DEFAULT_REGION|System\.getProperty\("region"|System\.(getenv|getenv)|getenv\("region"|getenv\("aws' jobs/pacman-data-shipper

Repository: PaladinCloud/CE

Length of output: 50371

🌐 Web query:

AWS Java SDK 1 x AWSSecurityTokenServiceClientBuilder defaultClient region environment variable AWS_DEFAULT_REGION

💡 Result:

In the AWS SDK for Java 1.x, the AWSSecurityTokenServiceClientBuilder.defaultClient method uses the DefaultAwsRegionProviderChain to determine the AWS region [1][2][3]. This provider chain does not check the AWS_DEFAULT_REGION environment variable by default; it specifically prioritizes the AWS_REGION environment variable [4][5][6]. The standard region lookup process used by the SDK for Java 1.x builder is: 1. Explicit region configuration set via the builder (e.g.,.withRegion) [4][5]. 2. The AWS_REGION environment variable [4][5]. 3. The AWS shared configuration file (typically ~/.aws/config) [4][5]. 4. The Amazon EC2 instance metadata service [4][5]. Because AWS_DEFAULT_REGION is not part of this specific provider chain for the Java 1.x SDK, it will be ignored if AWS_REGION is not set. If you require the SDK to recognize AWS_DEFAULT_REGION, you would typically need to manually configure the client or implement a custom region provider, as the built-in default chain is explicitly defined to use AWS_REGION [5][7][6]. Note that the AWS SDK for Java 1.x reached end-of-support on December 31, 2025, and users are encouraged to migrate to the AWS SDK for Java 2.x [8].

Citations:

Set the STS client region from base.region.

AWSSecurityTokenServiceClientBuilder.defaultClient() does not read base.region and uses AWS_REGION instead. Since this job only applies base.region, the base-account credential refresh path can pick any other AWS region for STS and fail on AssumeRole.

Proposed fix 
-            AWSSecurityTokenService sts = AWSSecurityTokenServiceClientBuilder.defaultClient();
+            AWSSecurityTokenService sts = AWSSecurityTokenServiceClientBuilder.standard()
+                    .withRegion(baseRegion)
+                    .build();
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/config/CredentialProvider.java`
at line 72, Update the STS client creation in CredentialProvider to configure
its region from the existing base.region value instead of using
AWSSecurityTokenServiceClientBuilder.defaultClient(). Preserve the existing
credential refresh and AssumeRole flow while ensuring the client explicitly
targets base.region.

AssumeRoleRequest assumeRequest = new AssumeRoleRequest().withRoleArn(getRoleArn(baseAccount, roleName)).withRoleSessionName("pic-base-ro").withDurationSeconds(3600);
AssumeRoleResult assumeResult = sts.assumeRole(assumeRequest);
return new BasicSessionCredentials(
assumeResult.getCredentials().getAccessKeyId(), assumeResult.getCredentials().getSecretAccessKey(),
assumeResult.getCredentials().getSessionToken());
return new STSAssumeRoleSessionCredentialsProvider.Builder(
getRoleArn(baseAccount, roleName), "pic-base-ro")
.withStsClient(sts)
.build();
}
}

Expand Down
5 changes: 3 additions & 2 deletions ...-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/config/S3ClientConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@ private S3ClientConfig() {
String s3Role = System.getProperty("s3.role");
objectMapper = new ObjectMapper();
s3Client = AmazonS3ClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(new CredentialProvider()
.getCredentials(s3Account, s3Role))).withRegion(s3Region).build();
.withCredentials(new CredentialProvider()
.getCredentials(s3Account, s3Role)).withRegion(s3Region).build();

}

private static class InstanceHolder {
Expand Down
2 changes: 1 addition & 1 deletion .../main/java/com/tmobile/cso/pacman/datashipper/entity/VulnerabilityAssociationManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ public List<Map<String, String>> uploadVulnerabilityInfo(String dataSource) {
LOGGER.info("Started Vulnerability collection for - {}", dataSource);
List<Map<String, String>> errorList = new ArrayList<>();
AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withCredentials(
new AWSStaticCredentialsProvider(new CredentialProvider().getCredentials(S3_ACCOUNT, S3_ROLE))).withRegion(S3_REGION).build();
(new CredentialProvider().getCredentials(S3_ACCOUNT, S3_ROLE))).withRegion(S3_REGION).build();

@coderabbitai coderabbitai Bot Aug 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Locate refreshable provider construction and lifecycle handling.
rg -n -C3 \
  'STSAssumeRoleSessionCredentialsProvider|new CredentialProvider\(\)\.getCredentials|\.close\(\)|\.shutdown\(\)' \
  jobs/pacman-data-shipper/src/main/java

Repository: PaladinCloud/CE

Length of output: 8530

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== Candidate files =="
fd -a 'VulnerabilityAssociationManager.java|DynamoDBHelper.java|CredentialProvider.java|LambdaInvoker.java' jobs/pacman-data-shipper/src/main/java || true

echo

echo "== Build/config for AWS SDK version =="
fd -a 'pom.xml|build.gradle|build.gradle.kts|settings.gradle|settings.gradle.kts' jobs/pacman-data-sdk-shipper . | sed 's#^\./##' || true
rg -n -C3 'aws-java-sdk|software.amazon.awssdk|dependency: aws-java-sdk|sdk-bom|version\.aws|<aws' -S . || true

echo

echo "== File contents =="
for f in \
  jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/entity/VulnerabilityAssociationManager.java \
  jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/util/DynamoDBHelper.java \
  jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/config/CredentialProvider.java \
  jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman(datashipper/util/LambdaInvoker.java
do
  echo "--- $f ---"
  wc -l "$f"
  cat -n "$f" || true
done

Repository: PaladinCloud/CE

Length of output: 50371

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== pom relevant dependencies =="
sed -n '1,120p' jobs/pacman-data-shipper/pom.xml

echo
echo "== target files =="
for f in \
  jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/entity/VulnerabilityAssociationManager.java \
  jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/util/DynamoDBHelper.java \
  jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/config/CredentialProvider.java \
  jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/util/LambdaInvoker.java
do
  echo "--- $f ---"
  wc -l "$f"
  cat -n "$f"
done

echo
echo "== AWS SDK 1.11.490 STSAssumeRoleSessionCredentialsProvider definitions from jars/remote =="
python3 - <<'PY'
import json, urllib.request, tempfile, zipfile, os, re, pathlib
urls = [
    "https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.490/aws-java-sdk-sts-1.11.490.jar",
    "https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-parent/1.11.490/aws-java-sdk-parent-1.11.490.pom",
]
for url in urls:
    print(f"FETCH {url}")
    with urllib.request.urlopen(url, timeout=20) as r:
        data = r.read()
    if url.endswith(".jar"):
        with tempfile.NamedTemporaryFile(suffix=".jar") as tf:
            tf.write(data)
            tf.flush()
            with zipfile.ZipFile(tf.name) as z:
                names = [n for n in z.namelist() if "STSAssumeRoleSessionCredentialsProvider" in n or "CredentialsProviderUtil" in n or "Refreshable" in n]
                print("entries:", names)
                for name in names[:12]:
                    text = z.read(name).decode("utf-8", errors="replace")
                    lines = text.splitlines()
                    for i,l in enumerate(lines[:120], 1):
                        if any(kw in l for kw in ["close", "shutdown", "new STSSecurityTokenService", "AWSSecurityTokenService", "STSAssumeRoleSessionCredentialsProvider", "closeStsClient"]):
                            print(f"{name}:{i}: {l}")
PY

Repository: PaladinCloud/CE

Length of output: 26406

🌐 Web query:

AWS SDK 1.11 STSAssumeRoleSessionCredentialsProvider close STS client client.close does not close provider credentials provider

💡 Result:

In the AWS SDK for Java 1.11, calling the close method on an STSAssumeRoleSessionCredentialsProvider does not explicitly close the underlying STS client [1][2]. The close method is primarily responsible for shutting down the background thread used for asynchronous credential refreshing [1][3]. The STS client used by this provider is typically managed separately by the user, especially if it was passed into the provider's builder [2]. If you created an STS client specifically to use with the credentials provider, you must manage its lifecycle independently and call its close method when it is no longer required [1][2]. The SDK does not automatically close resources such as custom ExecutorServices or the STS client itself upon closing the credentials provider [1][2]. In summary: 1. The close method on STSAssumeRoleSessionCredentialsProvider terminates the background refresh thread [1][4]. 2. You must manually close the STS client instance used by the provider if it is no longer needed [2]. 3. Similarly, if you provided a custom ExecutorService to the provider, you are responsible for shutting it down [1][2].

Citations:

Close STS credential sessions after each request-scoped AWS client.

CredentialProvider returns STSAssumeRoleSessionCredentialsProvider instances backed by STS clients created per call. AWS service clients do not close these session providers/STS clients, so repeated calls can leave credential-refresh tasks or STS clients allocated.

  • VulnerabilityAssociationManager: hold the provider and STS client in request-scoped variables, process the S3 workload, then close the provider and shut down the local S3/STS resources.
  • DynamoDBHelper: keep the provider in scope and close it in finally with provider.close(); client.shutdown();.
  • LambdaInvoker: if this path remains requestscoped, apply the same provider/STS-resource lifecycle, or move to an application-scoped provider with controlled shutdown.
📍 Affects 2 files
  • jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/entity/VulnerabilityAssociationManager.java#L45-L45 (this comment)
  • jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/util/DynamoDBHelper.java#L37-L37
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/entity/VulnerabilityAssociationManager.java`
at line 45, Close request-scoped AWS credential and client resources after each
workload. In
jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/entity/VulnerabilityAssociationManager.java:45,
retain the STS provider and local S3/STS clients in scope, process the workload,
then close and shut them down. In
jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/util/DynamoDBHelper.java:37,
retain the provider and close it in finally alongside client.shutdown(); apply
equivalent lifecycle handling in LambdaInvoker if its provider remains
request-scoped.

ObjectMapper objectMapper = new ObjectMapper();

for (Map.Entry<String, String> entry : sourceFileToIndexMapping.entrySet()) {
Expand Down
3 changes: 1 addition & 2 deletions ...an-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/util/DynamoDBHelper.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,7 @@ public static Map<String, String> get(String region, String tableName,

LOGGER.info("Querying '{}' for item: {}", tableName, request);
AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(
new CredentialProvider().getCredentials(account, role)))
.withCredentials(new CredentialProvider().getCredentials(account, role))
.withRegion(region)
.build();
try {
Expand Down
2 changes: 1 addition & 1 deletion ...man-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/util/LambdaInvoker.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ private static AWSLambda getLambdaClient() {
String role = System.getProperty("s3.role");

lambdaClient = AWSLambdaClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(
.withCredentials((
new CredentialProvider().getCredentials(account, role)))
.withRegion(getRegion())
.build();
Expand Down
Loading