Skip to content

Commit 2ac958d

Browse files
russellromneyclaude
russellromney
and
claude
committed
feat: Loadable extension, language packages, CI/CD, PyPI + npm publish
- SQLite loadable extension via C shim routing sqlite-vfs extern symbols through the extension API table (ext_entry.c + ext.rs) - Python package (pure Python, bundles .so/.dylib, load(conn) + connect() API) - Node.js package (napi-rs wrapped Database class) - FFI tests for Python, C, Go, Node + loadable extension tests - Examples for all languages - CI workflow (test + build on push/PR) - Release workflow (cross-platform builds, PyPI trusted publisher, npm provenance) - Published turbolite 0.2.19 to PyPI and npm Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent c0ca73d commit 2ac958d

50 files changed

Lines changed: 18898 additions & 288 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
8+
env:
9+
CARGO_TERM_COLOR: always
10+
11+
jobs:
12+
test:
13+
runs-on: ubuntu-latest
14+
steps:
15+
- uses: actions/checkout@v4
16+
- uses: dtolnay/rust-toolchain@stable
17+
- run: cargo test --features zstd,bundled-sqlite
18+
19+
build-ext:
20+
runs-on: ubuntu-latest
21+
steps:
22+
- uses: actions/checkout@v4
23+
- uses: dtolnay/rust-toolchain@stable
24+
- name: Build loadable extension
25+
run: cargo build --release --lib --no-default-features --features loadable-extension,zstd
26+
27+
build-node:
28+
runs-on: ubuntu-latest
29+
steps:
30+
- uses: actions/checkout@v4
31+
- uses: dtolnay/rust-toolchain@stable
32+
- uses: actions/setup-node@v4
33+
with:
34+
node-version: 20
35+
- working-directory: packages/node
36+
run: npm install && npx @napi-rs/cli build --release --platform

‎.github/workflows/release.yml‎

Lines changed: 194 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,194 @@
1+
name: Release
2+
3+
on:
4+
push:
5+
tags: ["v*"]
6+
7+
permissions:
8+
contents: write
9+
id-token: write
10+
11+
env:
12+
CARGO_TERM_COLOR: always
13+
14+
jobs:
15+
# ── Python wheels (loadable extension) ────────────────────────────
16+
python-build:
17+
strategy:
18+
fail-fast: false
19+
matrix:
20+
include:
21+
- os: ubuntu-latest
22+
target: x86_64-unknown-linux-gnu
23+
plat-name: manylinux2014_x86_64
24+
- os: ubuntu-latest
25+
target: aarch64-unknown-linux-gnu
26+
plat-name: manylinux2014_aarch64
27+
cross: true
28+
- os: macos-13
29+
target: x86_64-apple-darwin
30+
plat-name: macosx_13_0_x86_64
31+
- os: macos-14
32+
target: aarch64-apple-darwin
33+
plat-name: macosx_14_0_arm64
34+
35+
runs-on: ${{ matrix.os }}
36+
steps:
37+
- uses: actions/checkout@v4
38+
39+
- uses: dtolnay/rust-toolchain@stable
40+
with:
41+
targets: ${{ matrix.target }}
42+
43+
- name: Install cross-compilation tools
44+
if: matrix.cross
45+
run: |
46+
sudo apt-get update
47+
sudo apt-get install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu
48+
49+
- name: Build loadable extension
50+
env:
51+
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER: aarch64-linux-gnu-gcc
52+
CC_aarch64_unknown_linux_gnu: aarch64-linux-gnu-gcc
53+
CXX_aarch64_unknown_linux_gnu: aarch64-linux-gnu-g++
54+
run: |
55+
cargo build --release --lib --target ${{ matrix.target }} \
56+
--no-default-features --features loadable-extension,zstd
57+
58+
- name: Determine library filename
59+
id: lib
60+
run: |
61+
if [[ "${{ matrix.os }}" == macos-* ]]; then
62+
echo "ext=dylib" >> "$GITHUB_OUTPUT"
63+
echo "prefix=lib" >> "$GITHUB_OUTPUT"
64+
else
65+
echo "ext=so" >> "$GITHUB_OUTPUT"
66+
echo "prefix=lib" >> "$GITHUB_OUTPUT"
67+
fi
68+
69+
- name: Copy extension into Python package
70+
run: |
71+
cp target/${{ matrix.target }}/release/${{ steps.lib.outputs.prefix }}sqlite_compress_encrypt_vfs.${{ steps.lib.outputs.ext }} \
72+
packages/python/turbolite/turbolite.${{ steps.lib.outputs.ext }}
73+
74+
- uses: actions/setup-python@v5
75+
with:
76+
python-version: "3.12"
77+
78+
- name: Build wheel
79+
run: |
80+
pip install setuptools wheel
81+
cd packages/python
82+
python setup.py bdist_wheel --plat-name ${{ matrix.plat-name }}
83+
84+
- uses: actions/upload-artifact@v4
85+
with:
86+
name: python-${{ matrix.target }}
87+
path: packages/python/dist/*.whl
88+
89+
python-publish:
90+
needs: python-build
91+
runs-on: ubuntu-latest
92+
environment: release
93+
steps:
94+
- uses: actions/download-artifact@v4
95+
with:
96+
pattern: python-*
97+
path: dist
98+
merge-multiple: true
99+
100+
- name: Publish to PyPI
101+
uses: pypa/gh-action-pypi-publish@release/v1
102+
with:
103+
packages-dir: dist/
104+
105+
# ── Node.js native addon (napi-rs) ───────────────────────────────
106+
node-build:
107+
strategy:
108+
fail-fast: false
109+
matrix:
110+
include:
111+
- os: ubuntu-latest
112+
target: x86_64-unknown-linux-gnu
113+
node-arch: linux-x64-gnu
114+
- os: ubuntu-latest
115+
target: aarch64-unknown-linux-gnu
116+
node-arch: linux-arm64-gnu
117+
cross: true
118+
- os: ubuntu-latest
119+
target: x86_64-unknown-linux-musl
120+
node-arch: linux-x64-musl
121+
- os: macos-13
122+
target: x86_64-apple-darwin
123+
node-arch: darwin-x64
124+
- os: macos-14
125+
target: aarch64-apple-darwin
126+
node-arch: darwin-arm64
127+
128+
runs-on: ${{ matrix.os }}
129+
steps:
130+
- uses: actions/checkout@v4
131+
132+
- uses: actions/setup-node@v4
133+
with:
134+
node-version: 20
135+
136+
- uses: dtolnay/rust-toolchain@stable
137+
with:
138+
targets: ${{ matrix.target }}
139+
140+
- name: Install cross-compilation tools (aarch64)
141+
if: matrix.cross
142+
run: |
143+
sudo apt-get update
144+
sudo apt-get install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu
145+
146+
- name: Install musl tools
147+
if: contains(matrix.target, 'musl')
148+
run: |
149+
sudo apt-get update
150+
sudo apt-get install -y musl-tools
151+
152+
- name: Install dependencies
153+
working-directory: packages/node
154+
run: npm install
155+
156+
- name: Build native addon
157+
working-directory: packages/node
158+
env:
159+
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER: aarch64-linux-gnu-gcc
160+
CC_aarch64_unknown_linux_gnu: aarch64-linux-gnu-gcc
161+
CXX_aarch64_unknown_linux_gnu: aarch64-linux-gnu-g++
162+
run: npx @napi-rs/cli build --release --platform --target ${{ matrix.target }}
163+
164+
- uses: actions/upload-artifact@v4
165+
with:
166+
name: node-${{ matrix.node-arch }}
167+
path: packages/node/turbolite.*.node
168+
169+
node-publish:
170+
needs: node-build
171+
runs-on: ubuntu-latest
172+
environment: release
173+
steps:
174+
- uses: actions/checkout@v4
175+
176+
- uses: actions/setup-node@v4
177+
with:
178+
node-version: 20
179+
registry-url: https://registry.npmjs.org
180+
181+
- uses: actions/download-artifact@v4
182+
with:
183+
pattern: node-*
184+
path: packages/node
185+
merge-multiple: true
186+
187+
- name: List artifacts
188+
run: ls -la packages/node/turbolite.*.node
189+
190+
- name: Publish to npm
191+
working-directory: packages/node
192+
env:
193+
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
194+
run: npm publish --access public --provenance

‎.gitignore‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,4 +84,10 @@ backend-go/server
8484

8585
coverage.html
8686

87-
.context
87+
.context
88+
89+
# Python package build artifacts
90+
packages/python/build/
91+
packages/python/*.egg-info/
92+
packages/python/turbolite/*.so
93+
packages/python/turbolite/*.dylib

‎Cargo.toml‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,8 @@ crate-type = ["lib", "cdylib"]
1313

1414
[features]
1515
default = ["zstd", "bundled-sqlite"]
16-
bundled-sqlite = ["rusqlite/bundled", "dep:libsqlite3-sys"]
16+
bundled-sqlite = ["dep:rusqlite", "rusqlite/bundled", "dep:libsqlite3-sys"]
17+
loadable-extension = []
1718
zstd = ["dep:zstd"]
1819
lz4 = ["dep:lz4_flex"]
1920
snappy = ["dep:snap"]
@@ -58,14 +59,15 @@ clap = { version = "4", features = ["derive", "env"] }
5859
anyhow = "1"
5960
serde = { version = "1", features = ["derive"] }
6061
serde_json = "1"
61-
rusqlite = { version = "0.35" }
62+
rusqlite = { version = "0.35", optional = true }
6263
libsqlite3-sys = { version = "0.33", features = ["bundled"], optional = true }
6364
tempfile = "3"
6465
ureq = "2"
6566
lambda_runtime = { version = "0.13", optional = true }
6667

6768
[build-dependencies]
6869
cbindgen = "0.27"
70+
cc = "1"
6971

7072
[dev-dependencies]
7173
rayon = "1.10"

CLAUDE.md renamed to DEVELOPMENT.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Claude Development Guidelines
1+
# Development Guidelines
22

33
## Secrets
44

@@ -15,6 +15,8 @@ Secrets stored:
1515
- `AWS_REGION` — `auto`
1616
- `TIERED_TEST_BUCKET` — S3 bucket for integration tests (`sqlces-test`)
1717

18+
You can use these as environment variables as well.
19+
1820
## Running Tests
1921

2022
```bash

0 commit comments

Comments
 (0)