Sync trunk with main at v0.3.0

Brings trunk up to date with all work on main through v0.3.0:
- S3 streaming transfers and sccache integration
- SMB2 compounding, small-file fast paths, benchmarks (v0.2.0)
- Non-blocking file logging with slog!/serr! macros, ISO-8601 timestamps
- Version bump to 0.3.0, README updates

Author: lukekim

.github/workflows/ci.yml

@@ -68,6 +68,22 @@ jobs:
             brew install awscli
           fi
 
+      - name: Verify spiceio binary version
+        run: |
+          BUILT_VERSION=$(./target/debug/spiceio --version)
+          echo "Built: ${BUILT_VERSION}"
+          echo "SPICEIO_BUILT_VERSION=${BUILT_VERSION}" >> "$GITHUB_ENV"
+
+          # If an older spiceio is already listening (e.g. from a previous run),
+          # kill it so the test starts the freshly-built binary.
+          for PORT in 18333 18334; do
+            STALE_PID=$(lsof -i ":${PORT}" -sTCP:LISTEN -t 2>/dev/null || true)
+            if [[ -n "$STALE_PID" ]]; then
+              echo "Stale process on :${PORT} (pid ${STALE_PID}), killing..."
+              kill "$STALE_PID" 2>/dev/null || true
+            fi
+          done
+
       - name: Run sccache integration test
         # Skipped when UNAS_SMB_PASS secret is not configured (e.g. fork PRs)
         if: ${{ env.HAS_SMB_PASS == 'true' }}
@@ -80,9 +96,6 @@ jobs:
           SPICEIO_REGION: ${{ vars.SPICEIO_REGION || 'us-west-1' }}
         run: ./scripts/test-sccache.sh
 
-      - name: Compile benchmarks
-        run: cargo bench --locked --no-run
-
       - name: Build release artifact
         run: cargo build --release --locked --bin spiceio
 

.gitignore

@@ -1,21 +1 @@
-# Generated by Cargo
-# will have compiled files and executables
-debug
-target
-
-# These are backup files generated by rustfmt
-**/*.rs.bk
-
-# MSVC Windows builds of rustc generate these, which store debugging information
-*.pdb
-
-# Generated by cargo mutants
-# Contains mutation testing data
-**/mutants.out*/
-
-# RustRover
-#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
-#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
-#  and can be added to the global gitignore or merged into this file.  For a more nuclear
-#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
-#.idea/
+/target

CLAUDE.md

@@ -21,27 +21,9 @@ make release                   # optimized release build
 make lint                      # fmt-check + check + strict clippy + rustdoc warnings
 make test                      # sccache integration test (requires SPICEIO_SMB_USER/PASS)
 make fmt                       # auto-format
-make bench                     # criterion benchmarks (crypto + protocol)
-make bench-live                # live throughput benchmarks against NAS
 make clean                     # cargo clean
 ```
 
-## Pre-PR checklist
-
-Always run these before creating a PR — they mirror what CI checks:
-
-```bash
-cargo fmt --all                # auto-format first
-cargo fmt --all --check        # verify no formatting diff remains
-cargo clippy --locked --all-targets --all-features -- -D warnings -D clippy::all -D clippy::cargo -A clippy::cargo-common-metadata
-RUSTDOCFLAGS="-D warnings" cargo doc --locked --workspace --no-deps --document-private-items
-cargo test --locked            # unit tests must pass
-```
-
-Or simply: `make lint && cargo test --locked`
-
-The CI also runs `./scripts/test-sccache.sh` (sccache integration test) when SMB credentials are available.
-
 The binary requires these environment variables:
 - `SPICEIO_SMB_SERVER` (required) — SMB server hostname or IP
 - `SPICEIO_SMB_USER` (required) — SMB username
@@ -52,6 +34,7 @@ The binary requires these environment variables:
 - `SPICEIO_SMB_DOMAIN` — SMB domain (default empty)
 - `SPICEIO_BUCKET` — virtual S3 bucket name (defaults to `SPICEIO_SMB_SHARE`)
 - `SPICEIO_REGION` — AWS region to advertise (default `us-east-1`)
+- `SPICEIO_LOG_FILE` — append logs to this file in addition to stderr (optional; non-blocking, never stalls the proxy)
 
 ## Architecture
 

Cargo.lock

@@ -1,6 +1,6 @@
 [package]
 name = "spiceio"
-version = "0.1.0"
+version = "0.3.0"
 edition = "2024"
 description = "S3-compatible API proxy to SMB file shares"
 license = "Apache-2.0"
@@ -35,6 +35,10 @@ harness = false
 name = "protocol_bench"
 harness = false
 
+[[bench]]
+name = "s3_bench"
+harness = false
+
 [lints.rust]
 warnings = "deny"
 unsafe_op_in_unsafe_fn = "deny"

Cargo.toml

@@ -1,4 +1,4 @@
-.PHONY: build release check fmt fmt-check clippy doc lint test bench bench-live clean all
+.PHONY: build release check fmt fmt-check clippy doc lint test test-extended clean all
 
 all: fmt lint test build
 
@@ -28,11 +28,8 @@ lint: fmt-check check clippy doc
 test: build
 	./scripts/test-sccache.sh
 
-bench:
-	cargo bench --locked
-
-bench-live: release
-	./scripts/bench-live.sh
+test-extended: build test
+	./scripts/test-sccache-spiceai.sh
 
 clean:
 	cargo clean

Makefile

@@ -1,24 +1,71 @@
 # spiceio
 
-S3-compatible API proxy to SMB file shares. Translates S3 HTTP requests into SMB 3.1.x wire protocol operations over TCP.
+**S3-compatible API proxy that turns any SMB share into an S3 endpoint** -- no mounting, no `libsmbclient`, no FUSE. Translates S3 HTTP requests directly into SMB 3.1.x wire-protocol operations over TCP.
 
-- Speaks SMB directly — no mount, no libsmbclient
-- NTLMv2 authentication via macOS CommonCrypto (no external crypto crates)
-- macOS 26+ only
+## Why spiceio
 
-## Supported S3 operations
+Most tools that bridge SMB and S3 (MinIO, s3proxy, VersityGW) require mounting the share to the local filesystem first. spiceio skips that entirely -- it speaks the SMB wire protocol directly over TCP (port 445), so there's no mount, no kernel driver, and no FUSE layer in the way.
+
+This makes it the simplest path from "I have an SMB share" to "any S3 client can use it":
+
+```
+S3 client  --->  spiceio (HTTP :8333)  --->  SMB server (TCP :445)
+              S3 API translation            SMB 3.1.x wire protocol
+```
+
+### Key highlights
+
+- **Zero-mount design** -- speaks SMB 3.1.x natively over TCP, never touches the local filesystem
+- **Full S3 compatibility** for common operations: Get/Put/Copy/Delete/Head Object, ListObjects (v1 & v2), ListBuckets, multipart uploads, range + conditional requests
+- **SMB2 compounding** -- batches Create+Read+Close or Create+Write+Close into single round trips for small file performance
+- **Streaming I/O** -- GetObject and PutObject stream directly between HTTP and SMB without buffering entire files
+- **Non-blocking logging** -- timestamped stdout/stderr with optional file tee via `SPICEIO_LOG_FILE`; dedicated writer thread, never stalls the proxy
+- **Simple config** -- everything via environment variables, single binary, `--version` flag
+- **Zero external crypto** -- NTLMv2 auth and AES-CMAC signing via macOS CommonCrypto FFI
+
+### Use cases
+
+- **sccache remote cache** -- point sccache at spiceio to store build cache on a NAS without cloud storage
+- **CI artifact storage** -- use `aws s3 cp` to push/pull build artifacts from any SMB share
+- **NAS integration** -- give S3-native tools access to existing file shares
+
+## Quick start
+
+Requires macOS 26+ and Rust (edition 2024).
+
+```bash
+make release
+```
+
+```bash
+export SPICEIO_SMB_SERVER=nas.local
+export SPICEIO_SMB_USER=admin
+export SPICEIO_SMB_PASS=secret
+export SPICEIO_SMB_SHARE=files
+./target/release/spiceio
+```
 
-GetObject (range + conditional), PutObject, CopyObject, DeleteObject, HeadObject, ListObjectsV1/V2, ListBuckets, multipart uploads (create/upload-part/complete/abort/list-parts/list-uploads), HeadBucket, GetBucketLocation, and stubs for ACL, tagging, and versioning.
+Now any S3 client works:
 
-## Build
+```bash
+aws s3 ls s3://files/ --endpoint-url http://localhost:8333
+aws s3 cp myfile.txt s3://files/remote.txt --endpoint-url http://localhost:8333
+```
 
-Requires Rust (edition 2024) and macOS 26+.
+### sccache example
 
 ```bash
-make                # fmt + lint + test + build
-make release        # optimized release build
-make test           # run tests
-make lint           # fmt-check + check + strict clippy + rustdoc warnings
+export SCCACHE_BUCKET=files
+export SCCACHE_ENDPOINT=http://localhost:8333
+export SCCACHE_REGION=us-east-1
+export SCCACHE_S3_USE_SSL=false
+export SCCACHE_S3_KEY_PREFIX=sccache
+export AWS_ACCESS_KEY_ID=test
+export AWS_SECRET_ACCESS_KEY=test
+export RUSTC_WRAPPER=sccache
+export CARGO_INCREMENTAL=0
+
+cargo build   # artifacts cached on your NAS via spiceio
 ```
 
 ## Configuration
@@ -36,24 +83,56 @@ All configuration is via environment variables:
 | `SPICEIO_SMB_DOMAIN` | no       | *(empty)*           | SMB domain                |
 | `SPICEIO_BUCKET`     | no       | `SPICEIO_SMB_SHARE` | Virtual S3 bucket name    |
 | `SPICEIO_REGION`     | no       | `us-east-1`         | AWS region to advertise   |
+| `SPICEIO_LOG_FILE`   | no       | *(none)*            | Append logs to file (non-blocking) |
 
-## Usage
+## Supported S3 operations
+
+- **Objects**: GetObject (range + conditional), PutObject (conditional-write), CopyObject, DeleteObject, HeadObject
+- **Listing**: ListObjectsV1, ListObjectsV2, ListBuckets
+- **Multipart**: CreateMultipartUpload, UploadPart, CompleteMultipartUpload, AbortMultipartUpload, ListParts, ListMultipartUploads
+- **Bucket**: HeadBucket, GetBucketLocation, CreateBucket, DeleteBucket
+- **Stubs**: ACL, tagging, versioning, encryption, lifecycle, CORS (returns valid empty responses)
+
+Path-style addressing only (no virtual-hosted-style).
+
+## Architecture
+
+Three modules:
+
+- **`s3`** -- HTTP layer. Parses S3 requests, produces XML responses. Router dispatches to the appropriate handler. Small files (<64KB) use compound fast paths; large files stream.
+- **`smb`** -- Wire protocol client. Manages TCP connection, negotiate/session-setup handshake, and file operations. Supports SMB2 compounding for batching multiple operations in a single round trip.
+- **`crypto`** -- FFI bindings to macOS CommonCrypto. MD4, SHA-256, SHA-512, HMAC-MD5, HMAC-SHA256, AES-128-CMAC. No Rust crypto crates.
 
-```bash
-export SPICEIO_SMB_SERVER=nas.local
-export SPICEIO_SMB_USER=admin
-export SPICEIO_SMB_PASS=secret
-export SPICEIO_SMB_SHARE=files
-./target/release/spiceio
+```
+HTTP request
+  -> s3::router::handle_request
+    -> smb::ops::ShareSession method
+      -> smb::client::SmbClient wire operations
+        -> TCP to SMB server
 ```
 
-Then use any S3 client pointed at `http://localhost:8333`:
+## Development
 
 ```bash
-aws s3 ls s3://files/ --endpoint-url http://localhost:8333
-aws s3 cp local.txt s3://files/remote.txt --endpoint-url http://localhost:8333
+make                   # fmt + lint + test + build
+make release           # optimized release build
+make lint              # fmt-check + check + strict clippy + rustdoc warnings
+make test              # S3 API tests + sccache integration test
+make test-extended     # above + builds spiceai repo through sccache/spiceio
+make clean             # cargo clean
 ```
 
+Tests require `SPICEIO_SMB_USER` and `SPICEIO_SMB_PASS` environment variables and access to an SMB server.
+
+## How it compares
+
+| Tool | Needs local mount? | SMB access method | Cross-platform? | Best for |
+|---|---|---|---|---|
+| **spiceio** | No | Direct SMB 3.1.x wire | macOS 26+ only | Cleanest wire-level proxy, zero dependencies |
+| **rclone** | No | rclone's SMB backend | Yes | Cross-platform, battle-tested |
+| MinIO + mount | Yes | CIFS/FUSE mount | Yes | Production-grade S3 features |
+| s3proxy / VersityGW | Yes | CIFS/FUSE mount | Yes | Lightweight or high-perf FS backends |
+
 ## License
 
 Apache 2.0