0.37.0

CLAUDE.md

@@ -0,0 +1,106 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Repository Overview
+
+rust-s3 is a Rust library for working with Amazon S3 and S3-compatible object storage APIs (Minio, Wasabi, GCS, R2, etc.). It's a workspace project with three main crates:
+
+- **s3** - Main library implementation in `/s3`
+- **aws-region** - AWS region handling in `/aws-region`
+- **aws-creds** - AWS credentials management in `/aws-creds`
+
+## Development Commands
+
+### Building and Testing
+
+```bash
+# Run CI tests (recommended first step)
+make ci
+
+# Run all tests including ignored ones
+make ci-all
+
+# Format code
+make fmt
+
+# Run clippy lints
+make clippy
+
+# Test specific runtime configurations
+cd s3
+make tokio           # Test with tokio runtime
+make async-std       # Test with async-std runtime
+make sync-nativetls  # Test sync implementation
+
+# Run a single test
+cargo test test_name
+
+# Run tests with specific features
+cargo test --no-default-features --features sync-native-tls
+```
+
+### Running Examples
+
+```bash
+# Run examples (requires AWS credentials)
+cargo run --example tokio
+cargo run --example async-std --no-default-features --features async-std-native-tls
+cargo run --example sync --no-default-features --features sync-native-tls
+cargo run --example minio
+cargo run --example r2
+cargo run --example google-cloud
+```
+
+## Architecture and Key Components
+
+### Core Structure
+
+The main `Bucket` struct (s3/src/bucket.rs) represents an S3 bucket and provides all S3 operations. Key architectural decisions:
+
+1. **Multiple Runtime Support**: The library uses `maybe-async` to support tokio, async-std, and sync runtimes through feature flags
+2. **Backend Abstraction**: HTTP requests are abstracted through backend modules:
+   - `request/tokio_backend.rs` - Tokio + reqwest
+   - `request/async_std_backend.rs` - async-std + surf
+   - `request/blocking.rs` - Sync implementation with attohttpc
+
+3. **Request Signing**: AWS Signature V4 implementation in `s3/src/signing.rs`
+4. **Streaming Support**: Large file operations support streaming to avoid memory issues
+
+### Feature Flags
+
+The library uses extensive feature flags to control dependencies:
+
+- **default**: `tokio-native-tls` runtime with native TLS
+- **sync**: Synchronous implementation without async runtime
+- **blocking**: Generates `*_blocking` variants of all async methods
+- **fail-on-err**: Return Result::Err for HTTP errors
+- **tags**: Support for S3 object tagging operations
+
+### Testing Approach
+
+Tests are primarily integration tests marked with `#[ignore]` that require actual S3 credentials. They're located inline within source files using `#[cfg(test)]` modules. Run ignored tests with:
+
+```bash
+cargo test -- --ignored
+```
+
+## Important Implementation Notes
+
+1. **Request Retries**: All requests are automatically retried once on failure. Additional retries can be configured with `bucket.set_retries()`
+
+2. **Path vs Subdomain Style**: The library supports both path-style and subdomain-style bucket URLs. Subdomain style is default.
+
+3. **Presigned URLs**: The library supports generating presigned URLs for GET, PUT, POST, and DELETE operations without requiring credentials at request time.
+
+4. **Error Handling**: With `fail-on-err` feature (default), HTTP errors return `Result::Err`. Without it, errors are embedded in the response.
+
+5. **Streaming**: Use `get_object_stream` and `put_object_stream` methods for large files to avoid loading entire content in memory.
+
+## Code Conventions
+
+- Use existing error types from `s3/src/error.rs`
+- Follow the async/sync abstraction pattern using `maybe_async` macros
+- Integration tests should be marked with `#[ignore]` if they require credentials
+- All public APIs should have documentation examples
+- Maintain compatibility with multiple S3-compatible services (not just AWS)

Makefile

@@ -1,24 +1,33 @@
 all: ci-all
 
-ci: s3-ci region-ci creds-ci
+# Main CI targets - fmt and clippy first, then tests
+ci: fmt clippy test
 
-ci-all: s3-all region-ci creds-ci
+ci-all: fmt clippy test-all
 
+# Formatting targets
 fmt: s3-fmt region-fmt creds-fmt
 
+# Clippy targets for all features
 clippy: s3-clippy region-clippy creds-clippy
 
-s3-all:
-	cd s3; make test-all
+# Test targets (run after fmt and clippy)
+test: s3-test region-test creds-test
+
+test-all: s3-test-all region-test creds-test
 
-s3-ci:
-	cd s3; make ci
+# Test targets for individual crates
+s3-test:
+	cd s3; make test-not-ignored
+
+s3-test-all:
+	cd s3; make test-all
 
-region-ci:
-	cd aws-region; make ci
+region-test:
+	cd aws-region; cargo test
 
-creds-ci:
-	cd aws-creds; make ci
+creds-test:
+	cd aws-creds; cargo test
 
 s3-fmt:
 	cd s3; cargo fmt --all

README.md

@@ -71,6 +71,18 @@ All runtimes support either `native-tls` or `rustls-tls`, there are features for
 
 `Bucket` struct provides constructors for `path-style` paths, `subdomain` style is the default. `Bucket` exposes methods for configuring and accessing `path-style` configuration. `blocking` feature will generate a `*_blocking` variant of all the methods listed below.
 
+#### LocalStack Compatibility
+
+When using LocalStack, you may need to skip sending the location constraint in bucket creation requests. LocalStack doesn't support location constraints in the request body and will return `InvalidLocationConstraint` errors. Set this environment variable to skip the constraint:
+
+```bash
+export RUST_S3_SKIP_LOCATION_CONSTRAINT=true
+# or
+export RUST_S3_SKIP_LOCATION_CONSTRAINT=1
+```
+
+This may also be needed for other S3-compatible services that don't support AWS-style location constraints.
+
 #### Buckets
 
 |          |                                                                                          |

aws-creds/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "aws-creds"
-version = "0.38.0"
+version = "0.39.0"
 authors = ["Drazen Urch"]
 description = "Rust library for working with Amazon IAM credential,s, supports `s3` crate"
 repository = "https://github.com/durch/rust-s3"
@@ -15,14 +15,14 @@ name = "awscreds"
 path = "src/lib.rs"
 
 [dependencies]
-thiserror = "1"
+thiserror = "2"
 home = "0.5"
 rust-ini = "0.21"
-attohttpc = { version = "0.28", default-features = false, features = [
+attohttpc = { version = "0.30", default-features = false, features = [
     "json",
 ], optional = true }
 url = "2"
-quick-xml = { version = "0.32", features = ["serialize"] }
+quick-xml = { version = "0.38", features = ["serialize"] }
 serde = { version = "1", features = ["derive"] }
 time = { version = "^0.3.6", features = ["serde", "serde-well-known"] }
 log = "0.4"

aws-creds/src/credentials.rs

@@ -428,7 +428,10 @@ impl Credentials {
     ///     Some("production")
     /// ).unwrap();
     /// ```
-    pub fn from_credentials_file<P: AsRef<Path>>(file: P, section: Option<&str>) -> Result<Credentials, CredentialsError> {
+    pub fn from_credentials_file<P: AsRef<Path>>(
+        file: P,
+        section: Option<&str>,
+    ) -> Result<Credentials, CredentialsError> {
         let conf = Ini::load_from_file(file.as_ref())?;
         let section = section.unwrap_or("default");
         let data = conf
@@ -561,7 +564,10 @@ aws_secret_access_key = SECRET
 "#;
         let file = create_test_credentials_file(content);
         let result = Credentials::from_credentials_file(file.path(), Some("nonexistent"));
-        assert!(matches!(result.unwrap_err(), CredentialsError::ConfigNotFound));
+        assert!(matches!(
+            result.unwrap_err(),
+            CredentialsError::ConfigNotFound
+        ));
     }
 }
 

aws-region/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "aws-region"
-version = "0.27.0"
+version = "0.28.0"
 authors = ["Drazen Urch"]
 description = "Tiny Rust library for working with Amazon AWS regions, supports `s3` crate"
 repository = "https://github.com/durch/rust-s3"
@@ -15,5 +15,5 @@ name = "awsregion"
 path = "src/lib.rs"
 
 [dependencies]
-thiserror = "1"
+thiserror = "2"
 serde = { version = "1", features = ["derive"], optional = true }

aws-region/src/region.rs

@@ -133,14 +133,12 @@ pub enum Region {
     WaApSoutheast1,
     /// Wasabi ap-southeast-2
     WaApSoutheast2,
+    /// Cloudflare R2 (global)
+    R2 { account_id: String },
+    /// Cloudflare R2 EU jurisdiction
+    R2Eu { account_id: String },
     /// Custom region
-    R2 {
-        account_id: String,
-    },
-    Custom {
-        region: String,
-        endpoint: String,
-    },
+    Custom { region: String, endpoint: String },
 }
 
 impl fmt::Display for Region {
@@ -199,6 +197,7 @@ impl fmt::Display for Region {
             OvhCaEastTor => write!(f, "ca-east-tor"),
             OvhSgp => write!(f, "sgp"),
             R2 { .. } => write!(f, "auto"),
+            R2Eu { .. } => write!(f, "auto"),
             Custom { ref region, .. } => write!(f, "{}", region),
         }
     }
@@ -319,6 +318,7 @@ impl Region {
             OvhCaEastTor => String::from("s3.ca-east-tor.io.cloud.ovh.net"),
             OvhSgp => String::from("s3.sgp.io.cloud.ovh.net"),
             R2 { ref account_id } => format!("{}.r2.cloudflarestorage.com", account_id),
+            R2Eu { ref account_id } => format!("{}.eu.r2.cloudflarestorage.com", account_id),
             Custom { ref endpoint, .. } => endpoint.to_string(),
         }
     }
@@ -335,10 +335,15 @@ impl Region {
 
     pub fn host(&self) -> String {
         match *self {
-            Region::Custom { ref endpoint, .. } => match endpoint.find("://") {
-                Some(pos) => endpoint[pos + 3..].to_string(),
-                None => endpoint.to_string(),
-            },
+            Region::Custom { ref endpoint, .. } => {
+                let host = match endpoint.find("://") {
+                    Some(pos) => endpoint[pos + 3..].to_string(),
+                    None => endpoint.to_string(),
+                };
+                // Remove trailing slashes to avoid signature mismatches
+                // AWS CLI and other SDKs handle this similarly
+                host.trim_end_matches('/').to_string()
+            }
             _ => self.endpoint(),
         }
     }
@@ -386,3 +391,34 @@ fn test_region_eu_central_2() {
     let region = "eu-central-2".parse::<Region>().unwrap();
     assert_eq!(region.endpoint(), "s3.eu-central-2.amazonaws.com");
 }
+
+#[test]
+fn test_custom_endpoint_trailing_slash() {
+    // Test that trailing slashes are removed from custom endpoints
+    let region_with_slash = Region::Custom {
+        region: "eu-central-1".to_owned(),
+        endpoint: "https://s3.gra.io.cloud.ovh.net/".to_owned(),
+    };
+    assert_eq!(region_with_slash.host(), "s3.gra.io.cloud.ovh.net");
+
+    // Test without trailing slash
+    let region_without_slash = Region::Custom {
+        region: "eu-central-1".to_owned(),
+        endpoint: "https://s3.gra.io.cloud.ovh.net".to_owned(),
+    };
+    assert_eq!(region_without_slash.host(), "s3.gra.io.cloud.ovh.net");
+
+    // Test multiple trailing slashes
+    let region_multiple_slashes = Region::Custom {
+        region: "eu-central-1".to_owned(),
+        endpoint: "https://s3.example.com///".to_owned(),
+    };
+    assert_eq!(region_multiple_slashes.host(), "s3.example.com");
+
+    // Test with port and trailing slash
+    let region_with_port = Region::Custom {
+        region: "eu-central-1".to_owned(),
+        endpoint: "http://localhost:9000/".to_owned(),
+    };
+    assert_eq!(region_with_port.host(), "localhost:9000");
+}

examples/async-std-backend.rs

@@ -3,9 +3,9 @@
 #[cfg(feature = "async-std")]
 use awscreds::Credentials;
 #[cfg(feature = "async-std")]
-use s3::error::S3Error;
-#[cfg(feature = "async-std")]
 use s3::Bucket;
+#[cfg(feature = "async-std")]
+use s3::error::S3Error;
 
 #[cfg(not(feature = "async-std"))]
 fn main() {}

examples/gcs-tokio.rs

@@ -5,9 +5,9 @@ use awscreds::Credentials;
 #[cfg(feature = "tokio")]
 use awsregion::Region;
 #[cfg(feature = "tokio")]
-use s3::error::S3Error;
-#[cfg(feature = "tokio")]
 use s3::Bucket;
+#[cfg(feature = "tokio")]
+use s3::error::S3Error;
 
 #[cfg(not(feature = "tokio"))]
 fn main() {}

examples/r2-tokio.rs

@@ -5,9 +5,9 @@ use awscreds::Credentials;
 #[cfg(feature = "tokio")]
 use awsregion::Region;
 #[cfg(feature = "tokio")]
-use s3::error::S3Error;
-#[cfg(feature = "tokio")]
 use s3::Bucket;
+#[cfg(feature = "tokio")]
+use s3::error::S3Error;
 
 #[cfg(not(feature = "tokio"))]
 fn main() {}

examples/sync-backend.rs

@@ -3,9 +3,9 @@
 #[cfg(feature = "sync")]
 use awscreds::Credentials;
 #[cfg(feature = "sync")]
-use s3::error::S3Error;
-#[cfg(feature = "sync")]
 use s3::Bucket;
+#[cfg(feature = "sync")]
+use s3::error::S3Error;
 
 #[cfg(feature = "sync")]
 fn main() -> Result<(), S3Error> {

examples/tokio-backend.rs

@@ -3,9 +3,9 @@
 #[cfg(feature = "tokio")]
 use awscreds::Credentials;
 #[cfg(feature = "tokio")]
-use s3::error::S3Error;
-#[cfg(feature = "tokio")]
 use s3::Bucket;
+#[cfg(feature = "tokio")]
+use s3::error::S3Error;
 
 #[cfg(not(feature = "tokio"))]
 fn main() {}