How to set s3 region?

[deluxghost]

I'm using a local minio server with a custom region, while Send is using the default us-east-1 value. Therefore the s3 authorization header is wrong and denied by minio server

Is there any way to set the region? If not, please add a config.

[timvisee]

You should be able to set this using these variables:

send/server/config.js

Lines 27 to 41 in b9faece

	s3_bucket: {
	format: String,
	default: '',
	env: 'S3_BUCKET'
	},
	s3_endpoint: {
	format: String,
	default: '',
	env: 'S3_ENDPOINT'
	},
	s3_use_path_style_endpoint: {
	format: Boolean,
	default: false,
	env: 'S3_USE_PATH_STYLE_ENDPOINT'
	},

Does that help?

[deluxghost]

You should be able to set this using these variables:

send/server/config.js

Lines 27 to 41 in b9faece

	s3_bucket: {
	format: String,
	default: '',
	env: 'S3_BUCKET'
	},
	s3_endpoint: {
	format: String,
	default: '',
	env: 'S3_ENDPOINT'
	},
	s3_use_path_style_endpoint: {
	format: Boolean,
	default: false,
	env: 'S3_USE_PATH_STYLE_ENDPOINT'
	},

Does that help?

hmm, i dont see a region config, only endpoint, bucket and path style.

[timvisee]

I must admit that I'm not too familiar with S3. Doesn't setting a custom endpoint allow you to set the region as well (URL prefix)?

If not, I'll probably be able to take a look into this after the weekend.

[deluxghost]

I must admit that I'm not too familiar with S3. Doesn't setting a custom endpoint allow you to set the region as well (URL prefix)?

If not, I'll probably be able to take a look into this after the weekend.

Im not very familiar with that too, looks like there is no standard for s3 url yet

other apps like gitlab support s3 region config. If you are using a 3rdparty lib, it might be easy to implement

[ihatemyisp]

I'll be the first to admit my S3 knowledge is severely lacking but I use Send (docker compose) with Minio. The S3 Region I have set in Minio is not the one Send uses by default, but it works.

Do you have S3_USE_PATH_STYLE_ENDPOINT set to "true"?

I did have to change one section of the IAM policy that @timvisee has on GitLab to:

...
"Statement": [
        {
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": [
                "arn:aws:s3:::<s3_bucket_name>"
            ],
            "Effect": "Allow"
        },
...

[NamPNQ]

Use can set environment variable S3_USE_PATH_STYLE_ENDPOINT=true to fix your problems, this is not related region

[boomam]

Having a similar issue, the path style variable however does not solve the issue.
Are there any known workarounds?

[lilas-dev]

Having a similar issue, the path style variable however does not solve the issue. Are there any known workarounds?

I think you have make some mistake on your config, I just try a simple setup, its work like a charm

version: "3"

services:
  send:
    image: registry.gitlab.com/timvisee/send:latest
    ports:
      - '1234:1234'
    environment:
      - NODE_ENV=production
      - BASE_URL=http://localhost:1234
      - PORT=1234
      - REDIS_HOST=redis
      - AWS_ACCESS_KEY_ID=iloveyou
      - AWS_SECRET_ACCESS_KEY=andyoudontloveme
      - S3_BUCKET=send
      - S3_ENDPOINT=http://minio:9000
      - S3_USE_PATH_STYLE_ENDPOINT=true
    depends_on:
      minio:
        condition: service_healthy
  redis:
    image: redis
  minio:
    image: minio/minio
    command: ['server', '/data']
    environment:
      - MINIO_ROOT_USER=iloveyou
      - MINIO_ROOT_PASSWORD=andyoudontloveme
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
  mc:
    image: minio/mc
    depends_on:
      minio:
        condition: service_healthy
    entrypoint: >
      /bin/sh -c "
      /usr/bin/mc alias set myminio http://minio:9000 iloveyou andyoudontloveme;
      /usr/bin/mc mb myminio/send;
      "

[boomam]

Having a similar issue, the path style variable however does not solve the issue. Are there any known workarounds?

I think you have make some mistake on your config, I just try a simple setup, its work like a charm

Can you confirm, what region do you have your Minio set too?
To be clear, the issue is not connecting it to Minio, but connecting to Minio where the region is not the default - as there is no variable in 'send' right now to set that.

[lilas-dev]

@boomam I dont see Minio support region https://min.io/docs/minio/linux/reference/minio-server/minio-server.html#environment-variables

Can you post your config and the log of error? Its hard to understand what happened without this.

[boomam]

@boomam I dont see Minio support region https://min.io/docs/minio/linux/reference/minio-server/minio-server.html#environment-variables

Correct, but you still have to specify it on the connection.
It's the same with other apps that can use S3, like Pydio or NextCloud.

Can you post your config and the log of error? Its hard to understand what happened without this.

I'd have to loop back in a bit to generate the error again, I ended up giving up with Send and deployed something else.

Regardless, other than the URL the config for S3 is the same as yours - just not specifying port 9000 being the only difference, as I have all my traffic go through a load balancer on 443/tcp. So the endpoint instead reads as 'https://minio.domain.com'.

To be clear on that aspect, it works with other apps - where I can specify the region in use.
So its not a LB issue or a Minio config issue. ;-)

Your other config examples, such as how you are deploying Minio, are not valid for me as they are on separate systems.

[boomam]

Had to tidy it up, it dumps it all on one line for some reason in STOUT -

Timestamp=1685717940242000000 
Logger=FirefoxSend 
Type=send.upload.upload 
Severity=2 
Pid=1 
EnvVersion=2.0 
Fields={"error":"UnknownEndpoint: Inaccessible 
host: `' at port `443'. This service may not be available in the `us-east-1' region.","stack":"\n    
at Request.ENOTFOUND_ERROR (/app/node_modules/aws-sdk/lib/event_listeners.js:611:46)\n    
at Request.callListeners (/app/node_modules/aws-sdk/lib/sequential_executor.js:106:20)\n    
at Request.emit (/app/node_modules/aws-sdk/lib/sequential_executor.js:78:10)\n    
at Request.emit (/app/node_modules/aws-sdk/lib/request.js:686:14)\n    
at error (/app/node_modules/aws-sdk/lib/event_listeners.js:443:22)\n    
at ClientRequest.<anonymous> (/app/node_modules/aws-sdk/lib/http/node.js:99:9)\n    
at ClientRequest.emit (node:events:390:28)\n    
at ClientRequest.emit (node:domain:475:12)\n    
at TLSSocket.socketErrorListener (node:_http_client:447:9)\n    at TLSSocket.emit (node:events:390:28)"}

Connectivity between container and service is confirmed, it can get to Minio/S3.
It's the region its tripping up on.

[lilas-dev]

@boomam From the logs, I think the problem maybe about your config of ` S3_ENDPOINT` Le ven. 2 juin 2023 à 17:03, boomam ***@***.***> a écrit :

…

Had to tidy it up, it dumps it all on one line for some reason in STOUT - Timestamp=1685717940242000000 Logger=FirefoxSend Type=send.upload.upload Severity=2 Pid=1 EnvVersion=2.0 Fields={"error":"UnknownEndpoint: Inaccessible host: `' at port `443'. This service may not be available in the `us-east-1' region.","stack":"\n at Request.ENOTFOUND_ERROR (/app/node_modules/aws-sdk/lib/event_listeners.js:611:46)\n at Request.callListeners (/app/node_modules/aws-sdk/lib/sequential_executor.js:106:20)\n at Request.emit (/app/node_modules/aws-sdk/lib/sequential_executor.js:78:10)\n at Request.emit (/app/node_modules/aws-sdk/lib/request.js:686:14)\n at error (/app/node_modules/aws-sdk/lib/event_listeners.js:443:22)\n at ClientRequest.<anonymous> (/app/node_modules/aws-sdk/lib/http/node.js:99:9)\n at ClientRequest.emit (node:events:390:28)\n at ClientRequest.emit (node:domain:475:12)\n at TLSSocket.socketErrorListener (node:_http_client:447:9)\n at TLSSocket.emit (node:events:390:28)"} Connectivity between container and service is confirmed, it can get to Minio/S3. It's the region its tripping up on. — Reply to this email directly, view it on GitHub <#82 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQ34G6OJGQPMLMXMQVDTQZLXJH6DNANCNFSM5TZ3FNNQ> . You are receiving this because you commented.Message ID: ***@***.***>