Service endpoint_url from config file ignored

[fkjaekel]

Describe the bug

In order to use the least environment variables possible with localstack, I created the following profile:

[services localstack-specific]
s3 =
  endpoint_url = http://s3.localhost.localstack.cloud:4566

[profile localstack]
region = us-east-1
output = json
endpoint_url=http://localhost:4566
services = localstack-specific

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

Be able to use the endpoint_url from config file instead of the AWS_ENDPOINT_URL_S3 environment variable

Current Behavior

Since the endpoint_url from config is not being used, if I don't set AWS_ENDPOINT_URL_S3 I get this at localstack:

File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/aws/protocol/parser.py", line 779, in _parse_xml_string_to_dom
raise ProtocolParserError(
localstack.aws.protocol.parser.ProtocolParserError: Unable to parse request (not well-formed (invalid token): line 1, column 0), invalid XML received:
b'{"sourceId":"1","eventDateTime":"2024-11-14T12:13:18.6870-03:00","code":"1 ","sourceApplicationId":1,"display":"TESTE","active":"true "}'
(Service: S3, Status Code: 500, Request ID: 4d0a9fbd-6893-4f2d-9f55-138ddb31fbd6, Extended Request ID: s9lzHYrFp76ZVxRcpX9+5cjAnEH2ROuNkd2BHfIa6UkFVdtjf5mKR3/eTPFvsiP/XV/VLi31234=)
Suppressed: software.amazon.awssdk.core.exception.SdkClientException: Request attempt 2 failure: exception while calling s3 with unknown operation: Traceback (most recent call last):
File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack/aws/protocol/parser.py", line 776, in _parse_xml_string_to_dom
parser.feed(xml_string)
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, column 0

Reproduction Steps

set AWS_PROFILE=localstack and use S3AsyncClient

Possible Solution

No response

Additional Information/Context

localstack version: 4

AWS Java SDK version used

2.29.47

JDK version used

openjdk version "21.0.2" 2024-01-16 OpenJDK Runtime Environment GraalVM CE 21.0.2+13.1 (build 21.0.2+13-jvmci-23.1-b30) OpenJDK 64-Bit Server VM GraalVM CE 21.0.2+13.1 (build 21.0.2+13-jvmci-23.1-b30, mixed mode, sharing)

Operating System and version

Linux Mint 22

[bhoradc]

Hi @fkjaekel,

Thank you for bringing this issue to our attention. I am able to reproduce the scenario with AWS S3.

Support for service-specific endpoints was introduced in Java SDK version 2.28.1. I have observed that the service-specific endpoint configuration under the services section has not been working correctly since then. I have validated this behavior with the latest SDK version (2.29.48) as well.

• Reproducible code sample:

package org.example;

import software.amazon.awssdk.core.async.AsyncRequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3AsyncClient;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.model.PutObjectResponse;
import java.nio.file.Paths;
import java.util.concurrent.CompletableFuture;

public class Main {
    public static void main(String[] args) {
        S3AsyncClient s3AsyncClient = S3AsyncClient.builder()
                .region(Region.US_EAST_1)
                .build();
        String bucketName = "***";
        String key = "***.txt";
        String filePath = "/Users/***/Desktop/sampleobject.txt";
        PutObjectRequest putObjectRequest = PutObjectRequest.builder()
                .bucket(bucketName)
                .key(key)
                .build();
        CompletableFuture<PutObjectResponse> future = s3AsyncClient.putObject(
                putObjectRequest,
                AsyncRequestBody.fromFile(Paths.get(filePath))
        );
        future.whenComplete((response, error) -> {
            if (error != null) {
                System.err.println("Error uploading file: " + error.getMessage());
            } else {
                System.out.println("File uploaded successfully! ETag: " + response.eTag());
            }
            s3AsyncClient.close();
        });
        future.join();
    }
}

• Config file:

[services s3-specific]
s3 = 
  endpoint_url = https://s3.us-east-2.amazonaws.com
[profile localstack]
region = us-east-1
output = json
services = s3-specific

I have tested and confirmed that the service-specific endpoint can be set correctly using the AWS_ENDPOINT_URL_S3 environment variable or the aws.endpointUrlS3 JVM system property. You can continue using these methods as a workaround until the issue is resolved.

I will review this issue with the team to further investigate on it.

Regards,
Chaitanya

[RanVaknin]

Fixed with #5944 Thanks.
Ran~