[JENKINS-75713] Using globs with depths causes errors in archiveartifact

[jenkins-infra-bot]

Hello,

 

Upgrading from 902 to 926 caused a weird issue on my end.

 

On a pipeline I'm using archiveartifacts like this:

 archiveArtifacts artifacts: 'my-client-build-eu/**/*.*, my-client-build-us/**/*.*', onlyIfSuccessful: true

After upgrading to 926, it started to return the following error out of the blue:

Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(sections=[])), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(sections=[])): Profile file contained no credentials for profile 'default': ProfileFile(sections=[]), ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Failed to load credentials from IMDS.]

For the context, I'm using instance profiles and that profile has s3 access already and just works. I also noticed that, if I specify a top level like (

/*

) version 926 still works. But if I provide a pattern like

directory/**/*.*

, it fails the error above.

Downgrading to 906 back fixes the issue.

Jenkins version: 2.504.1 (LTS, docker image)
 

---

Originally reported by sb1111, imported from: Using globs with depths causes errors in archiveartifact

• status: Open
• priority: Minor
• component(s): artifact-manager-s3-plugin
• resolution: Unresolved
• votes: 0
• watchers: 4
• imported: 2025-12-02

Raw content of original issue

Hello,

 

Upgrading from 902 to 926 caused a weird issue on my end.

 

On a pipeline I'm using archiveartifacts like this:


 archiveArtifacts artifacts: 'my-client-build-eu/**/*.*, my-client-build-us/**/*.*', onlyIfSuccessful: true


After upgrading to 926, it started to return the following error out of the blue:

Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(sections=[])), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(sections=[])): Profile file contained no credentials for profile 'default': ProfileFile(sections=[]), ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Failed to load credentials from IMDS.]


For the context, I'm using instance profiles and that profile has s3 access already and just works. I also noticed that, if I specify a top level like (

/*

) version 926 still works. But if I provide a pattern like 

directory/**/*.*

   , it fails the error above. 
Downgrading to 906 back fixes the issue.
Jenkins version: 2.504.1 (LTS, docker image)
 

[jenkins-infra-bot]

jglick:

• Original comment link
• Raw content of original comment:

  Presumably this is caused by the switch to v2 in 917, and I guess
  
  Failed to load credentials from IMDS.
  
  is the problem. This would not normally have anything to do with the glob pattern, though. Perhaps it works for a small number of artifacts and fails for a larger number uploaded in quick succession? Perhaps the v1 version of the plugin code was caching credentials between calls and so making only one outbound API call to IMDS at most per step call, whereas the v2 version is making one IMDS call per artifact and hitting some sort of rate limiting? If you know of any way to reproduce the problem from scratch that would be very helpful.
  

Presumably this is caused by the switch to v2 in 917, and I guess

Failed to load credentials from IMDS.

is the problem. This would not normally have anything to do with the glob pattern, though. Perhaps it works for a small number of artifacts and fails for a larger number uploaded in quick succession? Perhaps the v1 version of the plugin code was caching credentials between calls and so making only one outbound API call to IMDS at most per step call, whereas the v2 version is making one IMDS call per artifact and hitting some sort of rate limiting? If you know of any way to reproduce the problem from scratch that would be very helpful.

[jenkins-infra-bot]

sb1111:

• Original comment link
• Raw content of original comment:

  jglick yeah amount of the files could be the real issue here.
  
  Before the archiveartifact step, I'm generating lots of CSS, js (react bundles at all) and using them in another pipeline.
  
  
  A similar behavior could be reproduced with a oneliner that generates sub-dirs and CSS/js files
  
  for i in $(seq -w 1 10); do mkdir -p "dir$i"; for j in $(seq -w 1 10); do mkdir -p "dir$i/subdir$j"; for k in $(seq $((RANDOM % 5 + 1))); do touch "dir$i/subdir$j/file$k.js" "dir$i/subdir$j/file$k.css"; done; done; done
  
  
  then try to run archiveartifacts with the pattern like 
  
  directory/**/*.*
  
  .
  

jglick yeah amount of the files could be the real issue here.

Before the archiveartifact step, I'm generating lots of CSS, js (react bundles at all) and using them in another pipeline.

A similar behavior could be reproduced with a oneliner that generates sub-dirs and CSS/js files

for i in $(seq -w 1 10); do mkdir -p "dir$i"; for j in $(seq -w 1 10); do mkdir -p "dir$i/subdir$j"; for k in $(seq $((RANDOM % 5 + 1))); do touch "dir$i/subdir$j/file$k.js" "dir$i/subdir$j/file$k.css"; done; done; done

then try to run archiveartifacts with the pattern like

directory/**/*.*

.

[jenkins-infra-bot]

jglick:

• Original comment link
• Raw content of original comment:

  I can ask the people who worked on the v2 migration to check if this reproduces with a large artifact count when using an instance profile to authenticate. (I do not have an environment using instance profiles handy to check. Normally I would test either using personal credentials, or in EKS using either pod identity or IRSA, none of which exercise InstanceProfileCredentialsProvider.) In the meantime, a possible workaround would be to modify your upstream pipeline to create a tarball or ZIP of all the files and archive that as a single blob, and the downstream pipeline to retrieve the single file and unpack it. (This would be more efficient anyway. stash / unstash within a single Pipeline build already uses a single compressed tarball.)
  

I can ask the people who worked on the v2 migration to check if this reproduces with a large artifact count when using an instance profile to authenticate. (I do not have an environment using instance profiles handy to check. Normally I would test either using personal credentials, or in EKS using either pod identity or IRSA, none of which exercise InstanceProfileCredentialsProvider.) In the meantime, a possible workaround would be to modify your upstream pipeline to create a tarball or ZIP of all the files and archive that as a single blob, and the downstream pipeline to retrieve the single file and unpack it. (This would be more efficient anyway. stash / unstash within a single Pipeline build already uses a single compressed tarball.)

[jenkins-infra-bot]

sb1111:

• Original comment link
• Raw content of original comment:

  Thank you so much for your interest! In meanwhile, I'll spin a test instance that I can troubleshoot IAM furthermore. 
  
  I'll go with creating a single archive instead of trying to upload them in bulk (quite an old pipeline, haha). I just wanted to report this for awareness. 
  

Thank you so much for your interest! In meanwhile, I'll spin a test instance that I can troubleshoot IAM furthermore.

I'll go with creating a single archive instead of trying to upload them in bulk (quite an old pipeline, haha). I just wanted to report this for awareness.

[jenkins-infra-bot]

olamy:

• Original comment link
• Raw content of original comment:

  I wonder if by any chance you have more informations available in the stacktrace? because the message doesn't give us much information (not full stack)
  
  Maybe if you could add some debugging for package `software.amazon.awssdk.auth.credentials` (I doubt this will provide more information from aws stack but worth to try)
  
  Can you increase as the timeout for this via the system property `-Daws.ec2MetadataServiceTimeout=10`
  
  or env var: AWS_METADATA_SERVICE_TIMEOUT=10
  
  default is 1 so it woth try increase the value a bit.
  
  this should be setup for the node where job is running. 
  
   
  
  This has been introduced with this release of sdk https://github.com/aws/aws-sdk-java-v2/releases/tag/2.29.36 
  
   
  

I wonder if by any chance you have more informations available in the stacktrace? because the message doesn't give us much information (not full stack)

Maybe if you could add some debugging for package `software.amazon.awssdk.auth.credentials` (I doubt this will provide more information from aws stack but worth to try)

Can you increase as the timeout for this via the system property `-Daws.ec2MetadataServiceTimeout=10`

or env var: AWS_METADATA_SERVICE_TIMEOUT=10

default is 1 so it woth try increase the value a bit.

this should be setup for the node where job is running. 

 

This has been introduced with this release of sdk https://github.com/aws/aws-sdk-java-v2/releases/tag/2.29.36 

 

[jenkins-infra-bot]

olamy:

• Original comment link
• Raw content of original comment:

  sb1111 ping 
  

sb1111 ping

[jenkins-infra-bot]

olamy:

• Original comment link
• Raw content of original comment:

  probably better to have of http exchange see https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/logging-slf4j.html#sdk-java-logging-request-response
  
   
  
  software.amazon.awssdk.request with debug level
  

probably better to have of http exchange see https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/logging-slf4j.html#sdk-java-logging-request-response

 

software.amazon.awssdk.request with debug level

[jenkins-infra-bot]

sb1111:

• Original comment link
• Raw content of original comment:

  olamy I didn't get/see any stacktrace when the issue occurred but I'll definitely try enabling debug logging when I spin up a test instance.
  
  I'll let you know 
  

olamy I didn't get/see any stacktrace when the issue occurred but I'll definitely try enabling debug logging when I spin up a test instance.

I'll let you know

[jenkins-infra-bot]

olamy:

• Original comment link
• Raw content of original comment:

  sb1111 thanks. by chance do you have more logs in the Jenkins system logs? stacktrace should be around
  

sb1111 thanks. by chance do you have more logs in the Jenkins system logs? stacktrace should be around

[jenkins-infra-bot]

olamy:

• Original comment link
• Raw content of original comment:

  sb1111 could you please give us the full stacktrace you had because there are some parts missing here.
  
  I can reproduce BUT except some files are uploaded but not all.
  
  I'm really interested in a better stacktrace.
  

sb1111 could you please give us the full stacktrace you had because there are some parts missing here.

I can reproduce BUT except some files are uploaded but not all.

I'm really interested in a better stacktrace.

[jenkins-infra-bot]

olamy:

• Original comment link
• Raw content of original comment:

  I have this log which looks to be the cause of the problem 
  Ignoring non-fatal exception while attempting to load metadata token from instance profile.
  java.net.SocketTimeoutException: Connect timed out
    at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:546)
    at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:592)
    at java.base/java.net.Socket.connect(Socket.java:751)
    at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:178)
    at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:531)
    at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:636)
    at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:280)
    at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:386)
    at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:408)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1319)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1297)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1138)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1067)
    at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.regions.internal.util.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:54)
    at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.regions.internal.util.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:60)
    at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.regions.util.HttpResourcesUtils.readResource(HttpResourcesUtils.java:112)
    at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.getToken(InstanceProfileCredentialsProvider.java:243)
    at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.createEndpointProvider(InstanceProfileCredentialsProvider.java:212)
    at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.refreshCredentials(InstanceProfileCredentialsProvider.java:159)
    at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.utils.cache.CachedSupplier.lambda$jitteredPrefetchValueSupplier$8(CachedSupplier.java:300)
    at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.utils.cache.CachedSupplier$PrefetchStrategy.fetch(CachedSupplier.java:448)
    at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.utils.cache.CachedSupplier.refreshCache(CachedSupplier.java:208)
    at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.utils.cache.CachedSupplier.get(CachedSupplier.java:135)
    at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.resolveCredentials(InstanceProfileCredentialsProvider.java:150)
    at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.auth.credentials.AwsCredentialsProvider.resolveIdentity(AwsCredentialsProvider.java:54)
    at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.identity.spi.IdentityProvider.resolveIdentity(IdentityProvider.java:60)
    at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain.resolveCredentials(AwsCredentialsProviderChain.java:109)
   
  
  You can go use the groovy console and change dynamically the timeout with 
  
  System.setProperty("aws.ec2MetadataServiceTimeout", "20")
  
  I did and could not reproduce the problem. 
  
  there is another issue which I'm looking at now 
  Also: org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: 054033f6-2912-4f46-bd07-ea2da251e719 software.amazon.awssdk.core.exception.SdkClientException: Unable to load region from any of the providers in the chain software.amazon.awssdk.regions.providers.DefaultAwsRegionProviderChain@5c31e185: [software.amazon.awssdk.regions.providers.SystemSettingsRegionProvider@6ef3cb85: Unable to load region from system settings. Region must be specified either via environment variable (AWS_REGION) or system property (aws.region)., software.amazon.awssdk.regions.providers.AwsProfileRegionProvider@59110c0d: No region provided in profile: default, software.amazon.awssdk.regions.providers.InstanceProfileRegionProvider@372f8744: Unable to contact EC2 metadata service.]
   
  
  looks like very similar to Force region for client  aws-credentials-plugin#291
  
  which is another problem detected right after aws sdk v2 migration but fixed recently.
  
   
  

I have this log which looks to be the cause of the problem 

Ignoring non-fatal exception while attempting to load metadata token from instance profile.
java.net.SocketTimeoutException: Connect timed out
  at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:546)
  at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:592)
  at java.base/java.net.Socket.connect(Socket.java:751)
  at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:178)
  at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:531)
  at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:636)
  at java.base/sun.net.www.http.HttpClient.(HttpClient.java:280)
  at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:386)
  at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:408)
  at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1319)
  at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1297)
  at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1138)
  at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1067)
  at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.regions.internal.util.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:54)
  at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.regions.internal.util.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:60)
  at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.regions.util.HttpResourcesUtils.readResource(HttpResourcesUtils.java:112)
  at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.getToken(InstanceProfileCredentialsProvider.java:243)
  at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.createEndpointProvider(InstanceProfileCredentialsProvider.java:212)
  at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.refreshCredentials(InstanceProfileCredentialsProvider.java:159)
  at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.utils.cache.CachedSupplier.lambda$jitteredPrefetchValueSupplier$8(CachedSupplier.java:300)
  at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.utils.cache.CachedSupplier$PrefetchStrategy.fetch(CachedSupplier.java:448)
  at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.utils.cache.CachedSupplier.refreshCache(CachedSupplier.java:208)
  at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.utils.cache.CachedSupplier.get(CachedSupplier.java:135)
  at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.resolveCredentials(InstanceProfileCredentialsProvider.java:150)
  at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.auth.credentials.AwsCredentialsProvider.resolveIdentity(AwsCredentialsProvider.java:54)
  at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.identity.spi.IdentityProvider.resolveIdentity(IdentityProvider.java:60)
  at PluginClassLoader for aws-java-sdk2-core//software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain.resolveCredentials(AwsCredentialsProviderChain.java:109)

 

You can go use the groovy console and change dynamically the timeout with 

System.setProperty("aws.ec2MetadataServiceTimeout", "20")

I did and could not reproduce the problem. 

there is another issue which I'm looking at now 

Also: org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: 054033f6-2912-4f46-bd07-ea2da251e719 software.amazon.awssdk.core.exception.SdkClientException: Unable to load region from any of the providers in the chain software.amazon.awssdk.regions.providers.DefaultAwsRegionProviderChain@​5c31e185: [software.amazon.awssdk.regions.providers.SystemSettingsRegionProvider@​6ef3cb85: Unable to load region from system settings. Region must be specified either via environment variable (AWS_REGION) or system property (aws.region)., software.amazon.awssdk.regions.providers.AwsProfileRegionProvider@​59110c0d: No region provided in profile: default, software.amazon.awssdk.regions.providers.InstanceProfileRegionProvider@​372f8744: Unable to contact EC2 metadata service.]

 

looks like very similar to jenkinsci/aws-credentials-plugin#291

which is another problem detected right after aws sdk v2 migration but fixed recently.

 

[jenkins-infra-bot]

olamy:

• Original comment link
• Raw content of original comment:

  regarding log in failing intermetently when using instance profile there is not much to do expect changing the timeout via sys property.
  
  The other problem regarding region is fixed by Enforce region for aws sdk calls #633
  

regarding log in failing intermetently when using instance profile there is not much to do expect changing the timeout via sys property.

The other problem regarding region is fixed by #633

[jenkins-infra-bot]

[Original is_duplicated_by from Jira: JENKINS-75777]