Bug: [Errno 8] nodename nor servname provided, or not known

[danny-su]

Description:

docker ps
/opt/homebrew/bin/sam build Function --template /Users/danny/Downloads/demo/hello/.aws-sam/temp-template.yaml --build-dir /Users/danny/Downloads/demo/hello/.aws-sam/build --skip-pull-image
Building codeuri: /Users/danny/Downloads/demo/hello runtime: go1.x metadata: {} architecture: x86_64 functions: Function
 Running GoModulesBuilder:Build

Build Succeeded

Built Artifacts  : build
Built Template   : build/template.yaml

Commands you can use next
=========================
[*] Validate SAM template: sam validate
[*] Invoke Function: sam local invoke -t build/template.yaml
[*] Test Function in the Cloud: sam sync --stack-name {{stack-name}} --watch
[*] Deploy: sam deploy --guided --template-file build/template.yaml
/opt/homebrew/bin/sam local invoke --template /Users/danny/Downloads/demo/hello/.aws-sam/build/template.yaml --event "/private/var/folders/vr/4przq03n0_q45ny09z79qh_w0000gn/T/[Local] main-event6.json" --skip-pull-image
Invoking main (go1.x)
Requested to skip pulling images ...

Mounting /Users/danny/Downloads/demo/hello/.aws-sam/build/Function as /var/task:ro,delegated, inside runtime container
Exception in thread Thread-2 (wait_for_logs):
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.35/containers/302c193b61bbf8d4c0591f7585c08b7b73b972cdd0cfb0aa04259058a8c50863/attach?logs=1&stdout=1&stderr=1&stream=1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/opt/homebrew/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/local/docker/container.py", line 379, in wait_for_logs
    logs_itr = real_container.attach(stream=True, logs=True, demux=True)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/models/containers.py", line 96, in attach
    return self.client.api.attach(self.id, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/api/container.py", line 60, in attach
    output = self._read_from_socket(
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/api/client.py", line 413, in _read_from_socket
    socket = self._get_raw_response_socket(response)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/api/client.py", line 318, in _get_raw_response_socket
    self._raise_for_status(response)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e) from e
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/errors.py", line 39, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation) from e
docker.errors.APIError: 500 Server Error for http+docker://localhost/v1.35/containers/302c193b61bbf8d4c0591f7585c08b7b73b972cdd0cfb0aa04259058a8c50863/attach?logs=1&stdout=1&stderr=1&stream=1: Internal Server Error ("can only attach to created or running containers - currently in state stopping: container state improper")

Error: [Errno 8] nodename nor servname provided, or not known
Traceback:
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/lib/telemetry/metric.py", line 184, in wrapped
    raise exception  # pylint: disable=raising-bad-type
    ^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/lib/telemetry/metric.py", line 149, in wrapped
    return_value = func(*args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/lib/utils/version_checker.py", line 42, in wrapped
    actual_result = func(*args, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/cli/main.py", line 95, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/commands/local/invoke/cli.py", line 101, in cli
    do_cli(
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/commands/local/invoke/cli.py", line 199, in do_cli
    context.local_lambda_runner.invoke(
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/commands/local/lib/local_lambda.py", line 144, in invoke
    self.local_runtime.invoke(
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/lib/telemetry/metric.py", line 324, in wrapped_func
    return_value = func(*args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/local/lambdafn/runtime.py", line 201, in invoke
    container.wait_for_result(
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/local/docker/container.py", line 349, in wait_for_result
    self._wait_for_socket_connection()
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/local/docker/container.py", line 388, in _wait_for_socket_connection
    while not self._can_connect_to_socket():
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/local/docker/container.py", line 406, in _can_connect_to_socket
    connection_succeeded = not a_socket.connect_ex(location)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

An unexpected error was encountered while executing "sam local invoke".
Search for an existing issue:
https://github.com/aws/aws-sam-cli/issues?q=is%3Aissue+is%3Aopen+Bug%3A%20sam%20local%20invoke%20-%20gaierror
Or create a bug report:
https://github.com/aws/aws-sam-cli/issues/new?template=Bug_report.md&title=Bug%3A%20sam%20local%20invoke%20-%20gaierror
Running SAM local invoke has failed: Command did not exit successfully, exit code: 1
10 has failed: Command did not exit successfully, exit code: 1

Steps to reproduce:

Observed result:

Expected result:

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

{
"version": "1.97.0",
"system": {
"python": "3.11.5",
"os": "macOS-13.5.2-arm64-arm-64bit"
},
"additional_dependencies": {
"docker_engine": "4.6.2",
"aws_cdk": "Not available",
"terraform": "1.5.7"
},
"available_beta_feature_env_vars": [
"SAM_CLI_BETA_FEATURES",
"SAM_CLI_BETA_BUILD_PERFORMANCE",
"SAM_CLI_BETA_TERRAFORM_SUPPORT",
"SAM_CLI_BETA_RUST_CARGO_LAMBDA"
]
}

1. OS: macOS 13.5.2
2. sam --version: SAM CLI, version 1.97.0
3. AWS region: HongKong (ap-east-1)

# Paste the output of `sam --info` here

Add --debug flag to command you are running

[mndeveci]

Hi there,

It seems like sam local invoke execution fails when trying to read the logs from a stopped container. I've tried to re-produce this with a GoLang Hello World example (I saw from the logs that function is using go1.x) but I wasn't able to re-produce this issue.

Can you provide more details, preferably a re-producable example so that we can deep dive into the root cause?

Thanks!

[danny-su]

Here is my demo project.
hello.zip
Today, I upgrade docker to 4.6.2, the following is the output of 'sam --info':

{
  "version": "1.97.0",
  "system": {
    "python": "3.11.5",
    "os": "macOS-13.5.2-arm64-arm-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "4.6.2",
    "aws_cdk": "Not available",
    "terraform": "1.5.7"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

But, I got another error 'Error: a bytes-like object is required, not 'str'':

docker ps
/opt/homebrew/bin/sam build Function --template /Users/danny/Downloads/demo/hello/.aws-sam/temp-template.yaml --build-dir /Users/danny/Downloads/demo/hello/.aws-sam/build
Building codeuri: /Users/danny/Downloads/demo/hello runtime: go1.x metadata: {} architecture: x86_64 functions: Function
 Running GoModulesBuilder:Build

Build Succeeded

Built Artifacts  : build
Built Template   : build/template.yaml

Commands you can use next
=========================
[*] Validate SAM template: sam validate
[*] Invoke Function: sam local invoke -t build/template.yaml
[*] Test Function in the Cloud: sam sync --stack-name {{stack-name}} --watch
[*] Deploy: sam deploy --guided --template-file build/template.yaml
/opt/homebrew/bin/sam local invoke --template /Users/danny/Downloads/demo/hello/.aws-sam/build/template.yaml --event "/private/var/folders/vr/4przq03n0_q45ny09z79qh_w0000gn/T/[Local] main-event4.json"
Invoking main (go1.x)

Error: a bytes-like object is required, not 'str'
Traceback:
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/lib/telemetry/metric.py", line 184, in wrapped
    raise exception  # pylint: disable=raising-bad-type
    ^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/lib/telemetry/metric.py", line 149, in wrapped
    return_value = func(*args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/lib/utils/version_checker.py", line 42, in wrapped
    actual_result = func(*args, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/cli/main.py", line 95, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/commands/local/invoke/cli.py", line 101, in cli
    do_cli(
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/commands/local/invoke/cli.py", line 199, in do_cli
    context.local_lambda_runner.invoke(
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/commands/local/lib/local_lambda.py", line 144, in invoke
    self.local_runtime.invoke(
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/lib/telemetry/metric.py", line 324, in wrapped_func
    return_value = func(*args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/local/lambdafn/runtime.py", line 189, in invoke
    container = self.create(function_config, debug_context, container_host, container_host_interface)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/local/lambdafn/runtime.py", line 85, in create
    container = LambdaContainer(
                ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/local/docker/lambda_container.py", line 94, in __init__
    image = LambdaContainer._get_image(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/local/docker/lambda_container.py", line 237, in _get_image
    return lambda_image.build(runtime, packagetype, image, layers, architecture, function_name=function_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/local/docker/lambda_image.py", line 202, in build
    self._check_base_image_is_current(base_image)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/local/docker/lambda_image.py", line 476, in _check_base_image_is_current
    if self.is_base_image_current(image_name):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/local/docker/lambda_image.py", line 500, in is_base_image_current
    return self.get_local_image_digest(image_name) == self.get_remote_image_digest(image_name)
                                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/samcli/local/docker/lambda_image.py", line 516, in get_remote_image_digest
    remote_info = self.docker_client.images.get_registry_data(image_name)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/models/images.py", line 356, in get_registry_data
    attrs=self.client.api.inspect_distribution(name, auth_config),
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/utils/decorators.py", line 34, in wrapper
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/api/image.py", line 287, in inspect_distribution
    return self._result(
           ^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e) from e
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/errors.py", line 34, in create_api_error_from_http_exception
    if any(fragment in explanation_msg
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/aws-sam-cli/1.97.0/libexec/lib/python3.11/site-packages/docker/errors.py", line 34, in <genexpr>
    if any(fragment in explanation_msg
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^

An unexpected error was encountered while executing "sam local invoke".
Search for an existing issue:
https://github.com/aws/aws-sam-cli/issues?q=is%3Aissue+is%3Aopen+Bug%3A%20sam%20local%20invoke%20-%20TypeError
Or create a bug report:
https://github.com/aws/aws-sam-cli/issues/new?template=Bug_report.md&title=Bug%3A%20sam%20local%20invoke%20-%20TypeError
Running SAM local invoke has failed: Command did not exit successfully, exit code: 1
13 has failed: Command did not exit successfully, exit code: 1

I installed both docker and podman.

[abcdrfg777777777]

sam build --use-container
AND
sam local invoke hello-lambda function --container-host 127.0.0.1