Skip to content

Commit 67dcbee

Browse files
authored
v0.2.0: SMB2 compounding, small-file fast paths, tests, and benchmarks (#6)
SMB2 compound operations batch multiple SMB operations into single TCP round trips, cutting latency for small file reads (3 RT -> 1 RT), writes (5+ RT -> 2 RT), head/delete (2 RT -> 1 RT), and directory creation (2N RT -> 1 RT). Router fast paths serve small GETs inline via compound Create+Read+Close and small PUTs via compound Create+Write+Close, bypassing the streaming path overhead. Additional changes: - Add --version flag to the binary - Add port-in-use handling to test scripts - Add CI step to verify binary version and kill stale processes - Add extended integration test building spiceai repo through sccache - Add unit tests across all modules (100 total) - Fix parse_http_date misrouting RFC 7231 dates to ISO 8601 parser - Add criterion benchmarks (crypto, protocol, s3) - Optimize epoch_to_iso8601/http_date from 150ns to 18ns (8x) - Add zero-copy decode_read_response_owned for the hot read path - Bump version to 0.2.0, license to Apache 2.0 - Rewrite README with architecture, use cases, and sccache example
1 parent 2eb7c27 commit 67dcbee

21 files changed

Lines changed: 2776 additions & 131 deletions

.github/workflows/ci.yml

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
name: CI
2+
3+
on:
4+
pull_request:
5+
types: [opened, synchronize, reopened]
6+
7+
concurrency:
8+
group: ${{ github.workflow }}-${{ github.ref }}
9+
cancel-in-progress: true
10+
11+
env:
12+
CARGO_TERM_COLOR: always
13+
CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
14+
CARGO_NET_GIT_FETCH_WITH_CLI: true
15+
16+
jobs:
17+
ci:
18+
name: Format, lint, build, and test
19+
# Self-hosted macOS runner required for CommonCrypto FFI and NAS access.
20+
# Skip fork PRs to prevent untrusted code execution on self-hosted runners.
21+
if: github.event.pull_request.head.repo.full_name == github.repository
22+
runs-on: spiceai-macos
23+
permissions:
24+
contents: read
25+
26+
steps:
27+
- name: Check out repository
28+
uses: actions/checkout@v4
29+
30+
- name: Set up Rust toolchain
31+
uses: dtolnay/rust-toolchain@stable
32+
with:
33+
components: rustfmt,clippy
34+
35+
- name: Cache Rust build artifacts
36+
uses: Swatinem/rust-cache@v2
37+
38+
- name: Check formatting
39+
run: cargo fmt --all --check
40+
41+
- name: Run cargo check
42+
run: cargo check --locked --all-targets --all-features
43+
44+
- name: Run clippy
45+
run: cargo clippy --locked --all-targets --all-features -- -D warnings -D clippy::all -D clippy::cargo -A clippy::cargo-common-metadata
46+
47+
- name: Check rustdoc
48+
run: RUSTDOCFLAGS="-D warnings" cargo doc --locked --workspace --no-deps --document-private-items
49+
50+
- name: Build debug binary
51+
run: cargo build --locked
52+
53+
- name: Run unit tests
54+
run: cargo test --locked
55+
56+
- name: Check SMB credentials
57+
run: |
58+
if [[ -n "$SPICEIO_SMB_PASS" ]]; then
59+
echo "HAS_SMB_PASS=true" >> "$GITHUB_ENV"
60+
fi
61+
env:
62+
SPICEIO_SMB_PASS: ${{ secrets.UNAS_SMB_PASS }}
63+
64+
- name: Install AWS CLI
65+
if: ${{ env.HAS_SMB_PASS == 'true' }}
66+
run: |
67+
if ! command -v aws &>/dev/null; then
68+
brew install awscli
69+
fi
70+
71+
- name: Verify spiceio binary version
72+
run: |
73+
BUILT_VERSION=$(./target/debug/spiceio --version)
74+
echo "Built: ${BUILT_VERSION}"
75+
echo "SPICEIO_BUILT_VERSION=${BUILT_VERSION}" >> "$GITHUB_ENV"
76+
77+
# If an older spiceio is already listening (e.g. from a previous run),
78+
# kill it so the test starts the freshly-built binary.
79+
for PORT in 18333 18334; do
80+
STALE_PID=$(lsof -i ":${PORT}" -sTCP:LISTEN -t 2>/dev/null || true)
81+
if [[ -n "$STALE_PID" ]]; then
82+
echo "Stale process on :${PORT} (pid ${STALE_PID}), killing..."
83+
kill "$STALE_PID" 2>/dev/null || true
84+
fi
85+
done
86+
87+
- name: Run sccache integration test
88+
# Skipped when UNAS_SMB_PASS secret is not configured (e.g. fork PRs)
89+
if: ${{ env.HAS_SMB_PASS == 'true' }}
90+
env:
91+
SPICEIO_SMB_SERVER: ${{ vars.SPICEIO_SMB_SERVER || '192.168.3.148' }}
92+
SPICEIO_SMB_USER: ${{ vars.SPICEIO_SMB_USER || 'runner' }}
93+
SPICEIO_SMB_PASS: ${{ secrets.UNAS_SMB_PASS }}
94+
SPICEIO_SMB_SHARE: ${{ vars.SPICEIO_SMB_SHARE || 'ai_platform_dev' }}
95+
SPICEIO_BUCKET: ${{ vars.SPICEIO_BUCKET || 'spiceio' }}
96+
SPICEIO_REGION: ${{ vars.SPICEIO_REGION || 'us-west-1' }}
97+
run: ./scripts/test-sccache.sh
98+
99+
- name: Build release artifact
100+
run: cargo build --release --locked --bin spiceio
101+
102+
- name: Upload release artifact
103+
uses: actions/upload-artifact@v4
104+
with:
105+
name: spiceio-${{ runner.os }}-${{ runner.arch }}
106+
path: target/release/spiceio
107+
if-no-files-found: error

0 commit comments

Comments
 (0)