Skip to content

RELEASING.md

Latest commit

 

History

History
280 lines (203 loc) · 10.2 KB

RELEASING.md

File metadata and controls

280 lines (203 loc) · 10.2 KB
Raw

< Back to README

Releasing to crates.io

Step-by-step guide for publishing readstat-rs crates to crates.io.

Quick Reference

# 0. Merge the release PR, then pull main
git checkout main && git pull origin main

# 1. Run all pre-publish checks
./scripts/release-check.sh        # Linux/macOS
.\scripts\release-check.ps1       # Windows

# 2. Bump versions (including the excluded WASM manifest and ARCHITECTURE.md)
cargo release minor -p readstat -p readstat-cli --dry-run   # preview first
cargo release minor -p readstat -p readstat-cli             # apply

# 3. Re-run release-check, review the bump commit and tag, then push.
./scripts/release-check.sh
#    Push the release tag EXPLICITLY (not --follow-tags, which would push any
#    stray local annotated tag). Only readstat/readstat-cli releases are
#    tagged (`v*`); sys-crate releases are crates.io-only events with no tag.
git push origin main vX.Y.Z

# 4. After CI builds the release artifacts, publish to crates.io:
#    Switch vendor dirs from submodules to copied files
./scripts/vendor.sh prepare       # Linux/macOS
.\scripts\vendor.ps1 prepare      # Windows

#    Publish (in dependency order)
cargo publish -p readstat-iconv-sys --allow-dirty
cargo publish -p readstat-sys --allow-dirty
cargo publish -p readstat
cargo publish -p readstat-cli

#    Restore submodules after publishing
./scripts/vendor.sh restore       # Linux/macOS
.\scripts\vendor.ps1 restore      # Windows

Install cargo-release once: cargo install cargo-release

Pre-Release Checklist

0. Check for Dependency Updates

./scripts/check-updates.sh              # report only (Linux/macOS)
./scripts/check-updates.sh --apply      # update safe deps in Cargo.lock
.\scripts\check-updates.ps1             # report only (Windows)
.\scripts\check-updates.ps1 -Apply      # update safe deps in Cargo.lock

This queries crates.io for outdated dependencies and their publish dates. Updates published less than 7 days ago (configurable via QUARANTINE_DAYS env var or -QuarantineDays parameter) are blocked to reduce supply chain risk.

The --apply / -Apply flag runs cargo update -p <crate> for each safe dependency, updating Cargo.lock within semver-compatible ranges. Major version bumps that require Cargo.toml edits are still manual.

1. Version Bumps

Use cargo-release (cargo install cargo-release). It updates all Cargo.toml version and dependency fields, substitutes the version strings in docs/ARCHITECTURE.md and the excluded readstat-wasm manifest, and creates a single version-bump commit plus a git tag.

# Preview what will change (no files are modified)
cargo release minor -p readstat -p readstat-cli --dry-run

# Apply — updates Cargo.toml files, docs/ARCHITECTURE.md, commits, creates tag
cargo release minor -p readstat -p readstat-cli

Use patch / minor / major as appropriate. After running, verify the diff looks right, then push the branch and the release tag explicitly: git push origin main vX.Y.Z. Avoid --follow-tags — it pushes every reachable annotated tag, not just the release tag. Only readstat/readstat-cli releases create a tag (v*, which triggers the GitHub Release build); sys-crate releases set tag = false and exist only as a bump commit, a CHANGELOG entry, and a crates.io publish.

Version conventions:

  • readstat, readstat-cli, and readstat-wasm share the same version (e.g. 0.29.0). The WASM crate is excluded from the workspace, but the readstat release replacement updates its Rust manifest, package manifest, and standalone lockfile package entries; locked release checks enforce parity.
  • readstat-sys and readstat-iconv-sys version independently — bump each only when its vendored C library, bindings, build behavior, or linking contract changes. Their versions describe the Rust FFI crate contract; they do not mirror the application crates or the vendored project's release number. For these pre-1.0 crates, Cargo treats 0.x.y patch releases as compatible and a change from 0.x to 0.(x+1) as a new compatibility line. Apply these rules:
    • No bump when the sys crate and its vendored source are unchanged.
    • Patch bump for compatible vendored bug/security fixes, additive bindings, and compatible build-script or platform-support changes (for example, cargo release patch -p readstat-sys).
    • Minor bump for breaking or reasonably suspected-incompatible changes to the Rust API, C ABI, bindings, features, linking behavior, or build contract.
    • Move to 1.0.0 only when the crate is ready to promise a stable public API, ABI, and build/linking contract. Their numbers are not expected to match. The compatibility contract is readstat-sys's declared dependency requirement on readstat-iconv-sys, which release-check verifies against the actual crate version (as it does for readstat → readstat-sys).

2. Update CHANGELOG.md

Add an entry for the new version:

## [X.Y.Z] - YYYY-MM-DD

### Added
- ...

### Changed
- ...

### Fixed
- ...

3. Run Automated Checks

./scripts/release-check.sh

This runs:

  • cargo fmt --all -- --check — formatting
  • cargo clippy --workspace --all-targets --all-features -- -D warnings — linting
  • readstat-wasm fmt and clippy (excluded from workspace, checked separately)
  • all-feature/all-target workspace checks and tests (including the CLI's opt-in SQL feature)
  • Arrow/DataFusion lockstep, all-feature rustdocs, and mdBook
  • advertised Rust API-server and PyO3 checks
  • WASM host fmt/clippy and an Emscripten release build when that toolchain is available (otherwise a prominent warning)
  • cargo deny check — license and security audit (if installed)
  • Version consistency checks
  • CHANGELOG entry check
  • cargo package dry-run for each publishable crate

Fix any failures before proceeding.

4. Manual Checks

  • README.md is up to date
  • Documentation reflects any API changes
  • Architecture docs (docs/ARCHITECTURE.md) are current
  • CLI cheatsheet footer (docs/readstat-cheatsheet.html) shows the new release version
  • mdbook builds cleanly: bash scripts/build-book.sh
  • readstat-wasm builds and exports are up to date (excluded from workspace; not published to crates.io)

Vendor Preparation

The readstat-sys and readstat-iconv-sys crates vendor C source code from git submodules. cargo publish cannot include git submodule contents, so the files must be copied as regular files before publishing.

Switch to publish mode

./scripts/vendor.sh prepare       # Linux/macOS
.\scripts\vendor.ps1 prepare      # Windows

This:

  1. Records submodule commit hashes in vendor-lock.txt
  2. Copies only the files needed for building (matching Cargo.toml include patterns)
  3. Deinitializes the git submodules
  4. Places the copied files in the vendor directories

Verify package contents

cargo package --list -p readstat-sys --allow-dirty
cargo package --list -p readstat-iconv-sys --allow-dirty

Publishing

Crates must be published in dependency order. Wait for each crate to appear on the crates.io index before publishing the next one.

After vendor.sh prepare, the vendored C sources are copied in as regular (uncommitted) files and the submodules are deinitialized, so the working tree is dirty. The two *-sys crates bundle those files, so their publishes need --allow-dirty. (readstat and readstat-cli don't carry vendored files, so they publish clean.)

# 1. No crate dependencies (carries vendored win-iconv → --allow-dirty)
cargo publish -p readstat-iconv-sys --allow-dirty

# 2. Depends on readstat-iconv-sys (carries vendored ReadStat → --allow-dirty)
cargo publish -p readstat-sys --allow-dirty

# 3. Depends on readstat-sys
cargo publish -p readstat

# 4. Depends on readstat
cargo publish -p readstat-cli

Note: There may be a delay (30 seconds to a few minutes) between publishing a crate and it appearing in the index. If cargo publish fails with a dependency resolution error, wait and retry.

Post-Publish

1. Restore submodules

./scripts/vendor.sh restore       # Linux/macOS
.\scripts\vendor.ps1 restore      # Windows

2. Verify crates.io

Each published crate appears on crates.io within a few minutes:

3. Verify docs.rs

docs.rs automatically builds documentation for every crate published to crates.io — no separate action is needed. The build is triggered by the crates.io publish and typically completes within 15–30 minutes.

The [package.metadata.docs.rs] section in crates/readstat/Cargo.toml instructs docs.rs to build with all features enabled and the docsrs cfg flag set, which causes feature-gated items to show their #[cfg(feature = "...")] badges.

Check build status and browse the rendered docs at:

4. Verify the GitHub release

The tag push triggers .github/workflows/release.yml. It validates the tag and versions, then runs CI, safety checks, seven CLI candidate builds, and one canonical WASM bundle build. A single final fan-in verifies every expected archive and checksum before creating the GitHub Release; candidate jobs never publish, so partial releases cannot be created. Confirm everything looks right on the Releases page.

5. Clean up

  • Remove vendor-lock.txt (or commit it for reference)

Troubleshooting

cargo publish fails with "no matching package found"

The dependency crate hasn't appeared in the index yet. Wait 30-60 seconds and retry.

cargo package includes too many files

Check the include field in the crate's Cargo.toml. Run cargo package --list to see exactly what will be included.

Vendor files missing after vendor.sh restore

Run git submodule update --init --recursive to re-initialize.

Build fails after switching vendor modes

Clean the build cache: cargo clean then rebuild.