Skip to content

Document connection issue due to out of sync clock #451

Open
@PGijsbers

Description

@PGijsbers

I experienced connection issues after the server clock got out of sync:

An error occurred (403) when calling the HeadBucket operation: Forbidden
Traceback (most recent call last):
  File "runbenchmark.py", line 181, in <module>
    bench.setup(amlb.SetupMode[args.setup])
  File ".../automlbenchmark/amlb/runners/aws.py", line 164, in setup
    self.bucket = self._create_s3_bucket()
  File ".../automlbenchmark/amlb/runners/aws.py", line 809, in _create_s3_bucket
    raise e
  File ".../automlbenchmark/amlb/runners/aws.py", line 788, in _create_s3_bucket
    self.s3.meta.client.head_bucket(Bucket=bucket_name)
  File ".../automlbenchmark/venv/lib/python3.8/site-packages/botocore/client.py", line 386, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File ".../automlbenchmark/venv/lib/python3.8/site-packages/botocore/client.py", line 705, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadBucket operation: Forbidden

It comes with this error. In my case the issue were not the credentials, but the clock being out of sync, which was identified through testing with the aws cli:

>aws s3api list-buckets

An error occurred (RequestTimeTooSkewed) when calling the ListBuckets operation: The difference between the request time and the current time is too large.

We need to check if we extract this RequestTimeTooSkewed from the thrown exception. If so, we need to ensure it is shown to the user. If not, we need to mention this possible cause (and/or maybe we can set up a separate test that does throw that error if it's a clock issue, so we can automate the diagnostics).

Metadata

Metadata

Assignees

No one assigned

    Labels

    awsAWS support

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions