[Bug]: AWS (Amazon Web Services) connection is failing in OpenRAG version 0.4.0-dev5.

[SakshiDhoni-ibmer]

OpenRAG Version

0.4.0 dev5

Deployment Method

uvx (uvx openrag)

Operating System

26.3.1

Python Version

3.13

Affected Area

Ingestion (document processing, upload, Docling)

Bug Description

AWS (Amazon Web Services) connection test is failing in OpenRAG version 0.4.0-dev5, while the same configuration was working correctly in 0.4.0-dev2.
The system is unable to establish or verify a successful AWS connection despite valid credentials and proper configuration.

Steps to Reproduce

1. Navigate to the OpenRAG project directory
2. Add AWS credentials in the .env file:
   AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY
   AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY
   AWS_DEFAULT_REGION=us-east-1
3. Restart all services.
4. Ensure the same credentials are available in the in AWS configuration Advance setup as well.
5. Attempt to test/verify AWS connection.

Expected Behavior

The AWS connection should be successfully established and verified, as it was in version 0.4.0-dev2.

Actual Behavior

Unable to verify successful connection, AWS option is not shown in add knowledge list to add document.

Relevant Logs

Screenshots

No response

Additional Context

No response

Checklist

• I have searched existing issues to ensure this bug hasn't been reported before.
• I have provided all the requested information.

[prasanthcaibmcom]

@ricofurtado Can you help take a look ?

[edwinjosechittilappilly]

AWS S3 and COS could only be activated in cloud SaaS environment wrt to product roadmap

[ricofurtado]

The AWS connector code has not changed between dev2 and dev5. I suspect the bug already existed in dev2.

[ricofurtado]

The bug was present from day one in PR #1168 and was never caught because testing was presumably done with admin/full-S3 credentials that do have s3:ListAllMyBuckets. It carried unchanged through dev2 → dev5.

@edwinjosechittilappilly do you have another another take in this issue?

[ricofurtado]

Pull request created in case we include it in the release 0.4.0
#1231

[ricofurtado]

There is a behaviour question(maybe product can anwer it?):

• If the credentials are valid but don’t have s3:ListAllMyBuckets, “Test connection” should still succeed?
  In this case, the UI should not block saving just because bucket listing failed.

• If bucket listing is unavailable, do we need to explain why in plain terms? E.g. “Credentials are valid, but this IAM user can’t list all buckets. Enter/select bucket names manually.”

-After saving, the connection should only be treated as usable if it has either:
ListBuckets permission, so auto-discovery can work, or
explicit bucket_names, so the connector can operate without bucket discovery.

So the practical fix is to split today’s combined step into two states:

• validate credentials
• try to discover buckets
• Bucket discovery should be best-effort, not a prerequisite for saving?
  If discovery fails with AccessDenied, we should still save the connection and guide the user into a manual bucket-name path instead of surfacing a generic failure?