Failed to send telemetry 6 record(s). Re-queue records. NoCredentialProviders: no valid providers in chain. Deprecated.

[nazarsa]

2020-05-10T05:05:13Z [Warn] Delaying sending of additional batches by 0 seconds
2020-05-10T05:05:14Z [Debug] processor: sending partial batch
2020-05-10T05:05:14Z [Debug] processor: segment batch size: 1. capacity: 50
2020-05-10T05:05:14Z [Error] Sending segment batch failed with: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2020-05-10T05:05:14Z [Warn] Delaying sending of additional batches by 0 seconds
2020-05-10T05:05:30Z [Debug] Failed to send telemetry 3 record(s). Re-queue records. NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2020-05-10T05:06:30Z [Debug] Failed to send telemetry 4 record(s). Re-queue records. NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2020-05-10T05:07:30Z [Debug] Failed to send telemetry 5 record(s). Re-queue records. NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2020-05-10T05:07:57Z [Debug] Received request on HTTP Proxy server : /GetSamplingRules
2020-05-10T05:07:57Z [Error] Unable to sign request: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2020-05-10T05:07:58Z [Debug] Received request on HTTP Proxy server : /GetSamplingRules
2020-05-10T05:07:58Z [Error] Unable to sign request: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2020-05-10T05:07:58Z [Debug] Received request on HTTP Proxy server : /GetSamplingRules
2020-05-10T05:07:58Z [Error] Unable to sign request: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2020-05-10T05:07:58Z [Debug] Received request on HTTP Proxy server : /GetSamplingRules
2020-05-10T05:07:58Z [Error] Unable to sign request: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2020-05-10T05:08:30Z [Debug] Failed to send telemetry 6 record(s). Re-queue records. NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors

[nazarsa]

I have the ~/.aws/credentials configured with access key and secret key, xray daemon running and AWSXrayFullAccess

[nazarsa]

Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty

[bhautikpip]

Hi @nazarsa ,

I would recommend you to check again your ~/.aws/credentials (especially access and secret key) and ~/.aws/config file for any misconfiguration and typing errors. This looks more like daemon is not able to fetch credentials. Also, are you running daemon locally ?

References (might help to resolve):
https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html
https://github.com/apex/apex/issues/426

[nazarsa]

nohup xray -o -n us-east-1 &
works with export AWS_ACCESS_KEY_ID and export AWS_SECRET_ACCESS_KEY, my credentials file have no error, this looks like a bug.

thanks @bhautikpip

[bhautikpip]

Would you mind providing how did you configure daemon and repro steps in order to correctly reproduce this issue ? Are you running daemon locally ?

[bencehornak]

I experienced the same, when I followed the AWS guide here:
https://docs.aws.amazon.com/xray/latest/devguide/xray-daemon-local.html#xray-daemon-local-docker

The article wrongly suggests that you should mount your ~/.aws directory to the /root/.aws/ container directory. The correct path is /home/xray/.aws, because the daemon is running as the xray user.

[bhautikpip]

Basically you should mount your local aws credentials path which normally should be found at ~/.aws but for some reason your local aws credentials path is different then you should use that path.