Commit a69d6d8
authored
fix: enable s3fs cache_regions for bucket region detection (#582)
## Problem
New S3 buckets in different regions fail with `UserAuthError: Access
Denied` until AWS DNS/routing converges (2-3 hours) when you try to
connect with the wrong region (i.e. use `us-east-1` for a bucket in
`us-east-2`).
## Solution
Enable `cache_regions=True` in s3fs configuration to handle AWS region
redirects automatically.
**Before:** Bucket access fails immediately, works after hours of
waiting
**After:** Bucket access works immediately regardless of region1 parent 7c6afe0 commit a69d6d8
File tree
3 files changed
+8
-1
lines changed- unstructured_ingest
- processes/connectors/fsspec
3 files changed
+8
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
1 | 5 | | |
2 | 6 | | |
3 | 7 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
118 | 118 | | |
119 | 119 | | |
120 | 120 | | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
121 | 124 | | |
122 | 125 | | |
123 | 126 | | |
| |||
0 commit comments