Skip to content

Commit 8796333

Browse files
authored
S3-compatible API proxy over SMB 3.1.1 (#1)
* Add CLAUDE.md with project guidance for Claude Code * Add design principles: reliability, performance, macOS 26+, SMB 3.1.x * Add README with usage, configuration, and build instructions * Initial commit: S3-compatible API proxy over SMB 3.1.1 - S3 layer: full API router with SigV4 auth, multipart uploads, conditional writes, range reads, ListObjectsV1/V2, CopyObject, multi-object Delete, and stub endpoints for ACL/tagging/versioning - SMB layer: wire protocol client (negotiate, session setup, tree connect, create, read, write, close, query directory, query info) with NTLMv2/SPNEGO authentication - Crypto: macOS CommonCrypto FFI (MD4, MD5, SHA-256, HMAC-MD5, HMAC-SHA256) — zero external crypto dependencies - Makefile with fmt, lint, clippy, test, build targets - Targets macOS 26+, Rust 2024 edition * Add streaming S3 transfers and sccache test flow - add SpioBody for streaming HTTP responses - stream GetObject reads from SMB to HTTP - stream PutObject writes from HTTP to SMB - add sccache integration test script and Makefile target - update docs and dependency metadata * Add streaming S3 transfers and sccache test flow SMB protocol: - Fix SPNEGO DER encoding for payloads >127 bytes - Fix Create/Write request offset calculations (off-by-one) - Fix Create/Write response minimum size checks - Handle STATUS_PENDING by looping for final response - Offer SMB 3.0.0/3.0.2 dialect fallbacks in negotiate - Add NTLMSSP Version field and echo server negotiate flags - Cap max read/write to 64KB to avoid oversized messages SMB3 message signing: - SHA-512 and AES-128-CMAC via CommonCrypto FFI - SP800-108 KDF signing key derivation from session base key - Preauth integrity hash tracking through handshake - Sign all post-auth messages with AES-128-CMAC S3 layer: - Map SMB NotFound → S3 NoSuchKey (404), PermissionDenied → 403 - Streaming PutObject via channel-backed body Test harness: - make test runs sccache integration against SMB share - Cold build populates cache, warm build verifies 100% hit rate * Rename spio to spiceio, add AWS CLI S3 API tests - Rename project, binary, env vars (SPIO_ → SPICEIO_), types (SpioBody → SpiceioBody) - Add 14 AWS CLI tests: ListBuckets, HeadBucket, Put/Get (small/64KB/1MB with integrity), HeadObject, ListObjectsV2, CopyObject, DeleteObject, nested paths, overwrite - Fix QueryDirectory request NameOffset (off-by-one, same as Create/Write) - Test cleanup removes test objects from SMB share * Address all 18 PR review comments - Replace all get_u16_le/get_u32_le/get_u64_le on immutable slices with from_le_bytes across protocol.rs and client.rs - NtStatus: add Unknown(u32) variant to preserve raw status codes instead of mapping unknowns to AccessDenied - ListObjects: sort objects by key before marker/max-keys filtering - RangeSpec::resolve: return Option, guard against underflow and empty objects; return 416 for unsatisfiable ranges - Unify ETag generation: re-read metadata after write so ETags always derive from last_write_time - Multipart completion: validate all parts exist before assembly, defer state removal until success - decode_read_response: parse DataOffset as u16 not u8 - Set RequestedOplockLevel to 0x00 (NONE) not 0x02 - Cap NetBIOS length to 0x00FFFFFF (3-byte max)
1 parent 9d02d4f commit 8796333

20 files changed

Lines changed: 5710 additions & 0 deletions

CLAUDE.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## What is spiceio
6+
7+
spiceio is an S3-compatible API proxy that translates S3 HTTP requests into SMB 3.1.x file operations. It speaks the SMB wire protocol directly over TCP (no mount, no libsmbclient) and uses macOS CommonCrypto via FFI for all cryptographic primitives (NTLMv2 auth, SHA-256, HMAC). Targets macOS 26+ only.
8+
9+
## Design principles
10+
11+
- **Reliability and resilience** — handle errors gracefully, recover from transient failures, never corrupt data. Correctness comes first.
12+
- **High performance** — minimize allocations, avoid unnecessary copies, use efficient I/O patterns. The proxy should not be the bottleneck.
13+
- **macOS 26+ only** — leverage core OS APIs (CommonCrypto, Security.framework, system libraries) wherever possible instead of pulling in external crates. Keep the dependency tree minimal.
14+
- **SMB 3.1.x** — implement the SMB 3.1.x dialect family. Stay current with protocol capabilities.
15+
16+
## Build & Run
17+
18+
```bash
19+
make # fmt + lint + test + build (default target)
20+
make release # optimized release build
21+
make lint # fmt-check + check + strict clippy + rustdoc warnings
22+
make test # sccache integration test (requires SPICEIO_SMB_USER/PASS)
23+
make fmt # auto-format
24+
make clean # cargo clean
25+
```
26+
27+
The binary requires these environment variables:
28+
- `SPICEIO_SMB_SERVER` (required) — SMB server hostname or IP
29+
- `SPICEIO_SMB_USER` (required) — SMB username
30+
- `SPICEIO_SMB_PASS` (required) — SMB password
31+
- `SPICEIO_SMB_SHARE` (required) — SMB share name
32+
- `SPICEIO_BIND` — listen address (default `0.0.0.0:8333`)
33+
- `SPICEIO_SMB_PORT` — SMB port (default `445`)
34+
- `SPICEIO_SMB_DOMAIN` — SMB domain (default empty)
35+
- `SPICEIO_BUCKET` — virtual S3 bucket name (defaults to `SPICEIO_SMB_SHARE`)
36+
- `SPICEIO_REGION` — AWS region to advertise (default `us-east-1`)
37+
38+
## Architecture
39+
40+
The codebase has three modules:
41+
42+
- **`s3`** — HTTP layer. Parses incoming S3 API requests and produces XML responses. `router.rs` is the central dispatch (path-style bucket routing). Covers GetObject, PutObject, CopyObject, DeleteObject, HeadObject, ListObjectsV1/V2, multipart uploads, and stub endpoints for ACL/tagging/versioning. `xml.rs` is a hand-rolled XML builder. `multipart.rs` manages upload state in-memory, with parts stored as temp files under `.spiceio-uploads/` on the SMB share. `body.rs` implements `SpiceioBody`, a zero-copy streaming response body (channel-backed for large reads, inline for XML/errors).
43+
44+
- **`smb`** — Wire protocol client. `protocol.rs` defines SMB 3.1.x packet structures (little-endian). `client.rs` manages the TCP connection, negotiate/session-setup handshake, and exposes operations (tree connect, create, read, write, close, query directory). `auth.rs` implements NTLMv2 challenge-response. `ops.rs` provides the high-level `ShareSession` abstraction the S3 layer consumes (list, read, write, delete, stat, copy).
45+
46+
- **`crypto`** — FFI bindings to macOS CommonCrypto (`Security.framework`/`libcommonCrypto`). Exposes MD4, SHA-256, and HMAC-MD5. No Rust crypto crates.
47+
48+
**Request flow:** HTTP request → `s3::router::handle_request` → S3 operation → `smb::ops::ShareSession` method → `smb::client::SmbClient` wire operations → TCP to SMB server.
49+
50+
## Key design decisions
51+
52+
- Zero external crypto dependencies — all crypto goes through `crypto::ffi` to CommonCrypto.
53+
- No `async-trait` — the SMB client uses `tokio::sync::Mutex` around the TCP stream with manual `async` methods.
54+
- GetObject streams SMB read chunks directly to the HTTP response via `SpiceioBody::channel` — no full-file buffering.
55+
- PutObject streams HTTP request body chunks directly to SMB write calls — no full-body collection.
56+
- Body is collected into `Bytes` only for operations that require the full payload (multi-delete, multipart complete, upload-part for ETag hashing).
57+
- S3 path-style addressing only (no virtual-hosted-style).
58+
- Multipart upload parts are stored as temporary SMB files, not in memory.

0 commit comments

Comments
 (0)