method of ec2 service `ec2.DescribeInstances` timedout in IPV6 only environment.

[srahul3]

Acknowledgements

• I have searched (https://github.com/aws/aws-sdk/issues?q=is%3Aissue) for past instances of this issue
• I have verified all of my SDK modules are up-to-date (you can perform a bulk update with go get -u github.com/aws/aws-sdk-go-v2/...)

Describe the bug

Problem

The method ec2.DescribeInstances timeout in ipv6 only environment.

Description

In a private VPC with multiple EC2's running in IPV6 only(no ipv4 addresses) environment. The EC2s detect each other by calling the method ec2.DescribeInstances of this SDK. The code is failing with i/o timeout error.

Note: All EC2s are granted with following permission

"ec2:DescribeInstances",
 "ec2:DescribeTags",
 "autoscaling:DescribeAutoScalingGroups"

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

The method shall respond with *ec2.DescribeInstancesOutput

Current Behavior

$ DEBUG=true go run main.go
2025/01/11 05:15:08 unable to describe instances, operation error EC2: DescribeInstances, exceeded maximum number of attempts, 3, https response error StatusCode: 0, RequestID: , request send failed, Post "https://ec2.us-east-1.amazonaws.com/": dial tcp [64:ff9b::342e:92c6]:443: i/o timeout
exit status 1

Reproduction Steps

Steps to reproduce

• Spin a private VPC with support for ipv6 CIDR aswell and Configure respective 1 or more public subnets with InternetGateway.
• Create a role with permissions mentioned in description of this bug.
• Create more than one ec2 with assumed role mentioned in above step.
• Ensure the ec2s have only ipv6 address.
• SSH into one of the instance and run below mentioned go code.

Expected response:

Tets code

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/ec2"
	"github.com/joho/godotenv"
)

func main() {
	cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-east-1"))
	if err != nil {
		log.Fatalf("unable to load SDK config, %v", err)
	}

	svc := ec2.NewFromConfig(cfg)
	resp, err := svc.DescribeInstances(context.TODO(), &ec2.DescribeInstancesInput{})
	if err != nil {
		log.Fatalf("unable to describe instances, %v", err)
	}

	var addrs []string
	for _, r := range resp.Reservations {
		fmt.Printf("\n[DEBUG] discover-aws: Reservation %s has %d instances", *r.ReservationId, len(r.Instances))
		for _, inst := range r.Instances {
			id := *inst.InstanceId
			fmt.Printf("\n[DEBUG] discover-aws: Found instance %s", id)

			for _, networkinterface := range inst.NetworkInterfaces {
				fmt.Printf("\n[DEBUG] discover-aws: Checking NetworInterfaceId %s on Instance %s", *networkinterface.NetworkInterfaceId, id)
				// Check if instance got any ipv6
				if networkinterface.Ipv6Addresses == nil {
					fmt.Printf("\n[DEBUG] discover-aws: Instance %s has no IPv6 on NetworkInterfaceId %s", id, *networkinterface.NetworkInterfaceId)
					continue
				}
				for _, ipv6address := range networkinterface.Ipv6Addresses {
					fmt.Printf("\n[INFO] discover-aws: Instance %s has IPv6 %s on NetworkInterfaceId %s", id, *ipv6address.Ipv6Address, *networkinterface.NetworkInterfaceId)
					addrs = append(addrs, *ipv6address.Ipv6Address)
				}
			}
		}
	}

	fmt.Printf("[DEBUG] discover-aws: Found ip addresses: %v", addrs)

}

Possible Solution

No response

Additional Information/Context

No response

AWS Go SDK V2 Module Versions Used

	github.com/aws/aws-sdk-go-v2 v1.32.8 // indirect
	github.com/aws/aws-sdk-go-v2/config v1.28.10 // indirect
	github.com/aws/aws-sdk-go-v2/credentials v1.17.51 // indirect
	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.23 // indirect
	github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.27 // indirect
	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.27 // indirect
	github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 // indirect
	github.com/aws/aws-sdk-go-v2/service/ec2 v1.198.3 // indirect
	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.1 // indirect
	github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery v1.10.8 // indirect
	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.8 // indirect
	github.com/aws/aws-sdk-go-v2/service/sso v1.24.9 // indirect
	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.8 // indirect
	github.com/aws/aws-sdk-go-v2/service/sts v1.33.6 // indirect
	github.com/aws/smithy-go v1.22.1 // indirect

Compiler and Version used

go version go1.23.2 linux/amd64

Operating System and version

Ubuntu 22.04.5 LTS

[srahul3]

I went through some of the docs and realised if I set the environment variable export AWS_ENDPOINT_URL=https://ec2.us-east-1.api.aws, I should work. It worked!

Alternatively I also tried setting the env export AWS_USE_DUALSTACK_ENDPOINT=true, this also worked

or

I could also do something like

sess := session.Must(session.NewSession())
svc := ec2.New(sess, &aws.Config{
    Region: aws.String(endpoints.UsEast1RegionID),
    Endpoint: aws.String("https://ec2.us-east-1.api.aws")
})

I wonder if this needed to be handled by SDK implicitly?

[lucix-aws]

This is working as intended. There's no reliable way for us as the SDK to know the constraints of your networking environment (e.g. IPv6-only).

The solution is to enable dualstack through AWS_USE_DUALSTACK_ENDPOINT as you've done. FYI I wouldn't use the ENDPOINT_URL envs for this, that has other implications. The former env is enough.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.