Merge pull request #23 from lfglabs-dev/fricoben/npm-native-binaries

feat(cli): switch to npm optional dependencies for native binaries

Author: fricoben

.claude/skills/publish-cli/SKILL.md

@@ -1,162 +1,156 @@
 ---
 name: publish-cli
 description: >
-  CLI publishing workflow using GitHub releases for binaries and local npm publish.
-  Trigger terms: publish, release, npm, cli, version, tag, github actions,
-  vibetracking, package, deploy cli.
+  CLI publishing workflow using npm optional dependencies for native binaries.
+  CI builds binaries, you publish locally via npm.
+  Trigger terms: publish, release, npm, cli, version, tag, vibetracking, package, deploy cli.
 ---
 
 ## When to Use
 
 - Publishing a new version of the CLI to npm
-- Checking the release workflow
-- Understanding the hybrid publishing model
+- Understanding the npm optional dependencies publishing model
+- Checking the build workflow
 
-## Architecture: Hybrid Publishing
+## Architecture: npm Optional Dependencies
 
-The CLI uses a hybrid publishing approach:
+The CLI uses the npm optional dependencies pattern (same as esbuild, swc, prisma):
 
-| Component | Host | Description |
-|-----------|------|-------------|
-| `vibetracking` | npm | CLI JavaScript wrapper (you publish locally) |
-| Native binaries | GitHub Releases | `.node` files for each platform |
+| Component | Location | Description |
+|-----------|----------|-------------|
+| `vibetracking` | npm | CLI JavaScript package |
+| `@starknetid/vibetracking-core` | npm | Main core package with optional deps |
+| `@starknetid/vibetracking-core-*` | npm | 7 platform-specific binary packages |
 
-**Why?** This simplifies publishing to 1 npm package instead of 9, avoiding npm OIDC/auth issues.
+**How it works**: When a user runs `npm install vibetracking`, npm automatically installs only the platform-specific binary package that matches their OS/arch.
 
-**How it works**: On first run, the CLI downloads the correct native binary from GitHub releases to `~/.vibetracking/bin/{version}/`.
+## Existing npm Packages
+
+These packages already exist on npm - we publish new versions to them:
+
+- `@starknetid/vibetracking-core` - Main package
+- `@starknetid/vibetracking-core-darwin-arm64` - macOS ARM64
+- `@starknetid/vibetracking-core-darwin-x64` - macOS Intel
+- `@starknetid/vibetracking-core-darwin-universal` - macOS Universal
+- `@starknetid/vibetracking-core-linux-x64-gnu` - Linux x64
+- `@starknetid/vibetracking-core-linux-arm64-gnu` - Linux ARM64
+- `@starknetid/vibetracking-core-win32-x64-msvc` - Windows x64
+- `@starknetid/vibetracking-core-win32-arm64-msvc` - Windows ARM64
 
 ## Procedure: Publish a New Version
 
 ### Step 1: Update Version Numbers
 
-Update version in `packages/cli/package.json`:
-```json
-{
-  "version": "X.Y.Z"
-}
+Use the napi version command to sync all package versions:
+
+```bash
+cd packages/core
+pnpm napi version -p X.Y.Z
 ```
 
-Update `BINARY_VERSION` constant in:
-- `packages/cli/src/native.ts`
-- `packages/cli/src/native-runner.ts`
+This updates:
+- `packages/core/package.json`
+- All `packages/core/npm/*/package.json` files
 
-```typescript
-const BINARY_VERSION = "X.Y.Z";
+Also update CLI version manually:
+```bash
+# Edit packages/cli/package.json
+# Change "version": "X.Y.Z"
 ```
 
-### Step 2: Commit and Push
+### Step 2: Commit and Push Tag
 
 ```bash
-git add packages/cli/
-git commit -m "chore: bump CLI version to X.Y.Z"
+git add .
+git commit -m "chore: bump version to X.Y.Z"
 git push origin main
+
+# Create and push tag to trigger CI build
+git tag cli-vX.Y.Z
+git push origin cli-vX.Y.Z
 ```
 
-### Step 3: Create and Push Tag
+### Step 3: Wait for CI Build
 
-**IMPORTANT**: Create the tag AFTER pushing the version bump to main. The tag must point to the commit with the updated version and workflow.
+GitHub Actions will build native binaries for all 7 platforms (~10-15 minutes).
 
+Monitor progress:
 ```bash
-git tag cli-vX.Y.Z
-git push origin cli-vX.Y.Z
+gh run watch --exit-status
 ```
 
-### Step 4: Wait for CI and Verify Release
-
-GitHub Actions will:
-1. Build native binaries for all 6 platforms
-2. Create universal macOS binary
-3. Upload all binaries to GitHub Release
+### Step 4: Download Artifacts
 
-Monitor progress: `gh run watch <RUN_ID> --exit-status`
+Once CI completes, download the built binaries:
 
-**Before proceeding**: Verify binaries were uploaded:
 ```bash
-gh release view cli-vX.Y.Z --json assets --jq '.assets[].name'
+# Get the run ID from the CLI or GitHub UI
+gh run download <RUN_ID> -D ./artifacts
 ```
 
-You should see 7 `.node` files (6 platforms + 1 universal macOS).
+### Step 5: Publish All Packages
 
-### Step 5: Publish CLI to npm (Locally)
-
-Once CI completes and binaries are verified, publish from your local machine:
+Run the publish script:
 
 ```bash
-cd packages/cli
-pnpm build
-npm publish --access public
+./scripts/publish-all.sh
 ```
 
-**Note**: npm will prompt for OTP (one-time password) from your authenticator app.
+This script:
+1. Copies binaries to their package directories
+2. Publishes all 7 platform packages
+3. Publishes the main core package
+4. Builds and publishes the CLI package
+
+**Note**: You'll be prompted for npm OTP if 2FA is enabled on your account.
 
 ### Step 6: Verify
 
 ```bash
-# Check npm
+# Check npm registry
 npm view vibetracking versions
 
-# Test fresh install (should download binary)
-rm -rf ~/.vibetracking/bin
+# Test fresh install
 bunx vibetracking@X.Y.Z --version
 ```
 
 ## Workflow Configuration
 
 - **File**: `.github/workflows/release.yml`
-- **Trigger**: Push tags matching `cli-v*`
-- **Output**: Native binaries uploaded to GitHub Release
+- **Trigger**: Push tags matching `cli-v*` or manual dispatch
+- **Output**: Build artifacts (downloaded locally for publishing)
+
+## Version Synchronization
 
-### Manual Trigger (Dry Run)
+All these must match:
+- `packages/cli/package.json` version
+- `packages/core/package.json` version
+- All `packages/core/npm/*/package.json` versions
 
-Test the build without uploading:
-1. Go to Actions > Release CLI
-2. Click "Run workflow"
-3. Check "Dry run" option
+The `napi version` command syncs the core packages automatically.
 
-## Native Binary Locations
+## Troubleshooting
 
-Binaries are downloaded to `~/.vibetracking/bin/{version}/`:
+### CI Build Failed
 
-| Platform | Binary Name |
-|----------|-------------|
-| macOS Intel | `vibetracking-core.darwin-x64.node` |
-| macOS ARM | `vibetracking-core.darwin-arm64.node` |
-| Linux x64 | `vibetracking-core.linux-x64-gnu.node` |
-| Linux ARM64 | `vibetracking-core.linux-arm64-gnu.node` |
-| Windows x64 | `vibetracking-core.win32-x64-msvc.node` |
-| Windows ARM64 | `vibetracking-core.win32-arm64-msvc.node` |
+Check the GitHub Actions logs. Common issues:
+- Rust toolchain version
+- Cross-compilation toolchain missing
 
-## User Installation
+### npm Publish Failed
 
-After publishing, users can install with:
+If a version already exists on npm:
 ```bash
-bunx vibetracking          # One-off execution (downloads binary on first run)
-bun add -g vibetracking    # Global install
+npm view @starknetid/vibetracking-core versions
 ```
 
-## Troubleshooting
-
-### CI Failed / Tag Points to Old Workflow
+Bump to the next version if needed.
 
-If the release workflow fails or you need to recreate a tag:
+### Binary Not Loading
 
+Check that the correct platform package was installed:
 ```bash
-# Delete tag locally and remotely
-git tag -d cli-vX.Y.Z
-git push origin :refs/tags/cli-vX.Y.Z
-
-# Create new tag on current main
-git tag cli-vX.Y.Z origin/main
-git push origin cli-vX.Y.Z
+npm ls @starknetid/vibetracking-core
 ```
 
-### npm Version Already Exists but Tarball is 404
-
-If `npm view vibetracking versions` shows a version but `bunx vibetracking@X.Y.Z` fails with 404, the version was corrupted. Bump to next patch version (e.g., 0.2.0 → 0.2.1).
-
-### Binary Download Fails
-
-Check that:
-1. GitHub release exists: `gh release view cli-vX.Y.Z`
-2. Repo is public (private repos require auth for release downloads)
-3. `GITHUB_REPO` in `native.ts` matches the actual repo name
+Should show the platform-specific package as an optional dependency.

.github/workflows/release.yml

@@ -1,25 +1,10 @@
-name: Release CLI
+name: Build CLI Binaries
 
 on:
   push:
     tags:
       - 'cli-v*'
   workflow_dispatch:
-    inputs:
-      dry_run:
-        description: 'Dry run (skip GitHub release upload)'
-        required: false
-        default: false
-        type: boolean
-      skip_build:
-        description: 'Skip build (reuse artifacts from previous run)'
-        required: false
-        default: false
-        type: boolean
-      run_id:
-        description: 'Run ID to download artifacts from (required if skip_build is true)'
-        required: false
-        type: string
 
 env:
   DEBUG: napi:*
@@ -31,7 +16,6 @@ permissions:
 
 jobs:
   build:
-    if: ${{ github.event.inputs.skip_build != 'true' }}
     strategy:
       fail-fast: false
       matrix:
@@ -94,7 +78,6 @@ jobs:
           if-no-files-found: error
 
   build-universal-macos:
-    if: ${{ github.event.inputs.skip_build != 'true' }}
     name: Build - universal-apple-darwin
     needs: build
     runs-on: macos-latest
@@ -133,71 +116,44 @@ jobs:
           path: packages/core/*.node
           if-no-files-found: error
 
-  upload-release:
-    name: Upload to GitHub Release
+  summary:
+    name: Build Summary
     runs-on: ubuntu-latest
     needs:
       - build
       - build-universal-macos
-    if: ${{ always() && (github.event.inputs.skip_build == 'true' || (needs.build.result == 'success' && needs.build-universal-macos.result == 'success')) }}
+    if: ${{ needs.build.result == 'success' && needs.build-universal-macos.result == 'success' }}
 
     steps:
       - uses: actions/checkout@v4
 
-      - name: Download artifacts from current run
-        if: ${{ github.event.inputs.skip_build != 'true' }}
-        uses: actions/download-artifact@v4
-        with:
-          path: artifacts
-          pattern: bindings-*
-          merge-multiple: true
-
-      - name: Download artifacts from previous run
-        if: ${{ github.event.inputs.skip_build == 'true' }}
-        uses: actions/download-artifact@v4
-        with:
-          path: artifacts
-          pattern: bindings-*
-          merge-multiple: true
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          run-id: ${{ github.event.inputs.run_id }}
-
-      - name: List artifacts
-        run: ls -la artifacts/
-
-      - name: Get tag name
-        id: tag
+      - name: Get version
+        id: version
         run: |
-          if [[ "${{ github.ref_type }}" == "tag" ]]; then
-            echo "name=${{ github.ref_name }}" >> $GITHUB_OUTPUT
-          else
-            # For manual runs, use the version from package.json
-            VERSION=$(jq -r .version packages/cli/package.json)
-            echo "name=cli-v${VERSION}" >> $GITHUB_OUTPUT
-          fi
-
-      - name: Create or update GitHub Release
-        if: ${{ github.event.inputs.dry_run != 'true' }}
-        uses: softprops/action-gh-release@v2
-        with:
-          tag_name: ${{ steps.tag.outputs.name }}
-          name: CLI ${{ steps.tag.outputs.name }}
-          files: artifacts/*.node
-          fail_on_unmatched_files: true
-          generate_release_notes: true
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          VERSION=$(jq -r .version packages/cli/package.json)
+          echo "version=${VERSION}" >> $GITHUB_OUTPUT
 
       - name: Summary
         run: |
-          echo "## Release ${{ steps.tag.outputs.name }}" >> $GITHUB_STEP_SUMMARY
+          echo "## Build Complete - v${{ steps.version.outputs.version }}" >> $GITHUB_STEP_SUMMARY
+          echo "" >> $GITHUB_STEP_SUMMARY
+          echo "### Artifacts Ready for Download" >> $GITHUB_STEP_SUMMARY
+          echo "" >> $GITHUB_STEP_SUMMARY
+          echo "All platform binaries have been built and uploaded as artifacts." >> $GITHUB_STEP_SUMMARY
+          echo "" >> $GITHUB_STEP_SUMMARY
+          echo "### Next Steps (Local Publishing)" >> $GITHUB_STEP_SUMMARY
           echo "" >> $GITHUB_STEP_SUMMARY
-          echo "### Binaries uploaded to GitHub Release" >> $GITHUB_STEP_SUMMARY
+          echo "1. Download artifacts:" >> $GITHUB_STEP_SUMMARY
+          echo "   \`\`\`bash" >> $GITHUB_STEP_SUMMARY
+          echo "   gh run download ${{ github.run_id }} -D ./artifacts" >> $GITHUB_STEP_SUMMARY
+          echo "   \`\`\`" >> $GITHUB_STEP_SUMMARY
           echo "" >> $GITHUB_STEP_SUMMARY
-          ls -1 artifacts/*.node | while read f; do
-            echo "- $(basename $f)" >> $GITHUB_STEP_SUMMARY
-          done
+          echo "2. Run publish script:" >> $GITHUB_STEP_SUMMARY
+          echo "   \`\`\`bash" >> $GITHUB_STEP_SUMMARY
+          echo "   ./scripts/publish-all.sh" >> $GITHUB_STEP_SUMMARY
+          echo "   \`\`\`" >> $GITHUB_STEP_SUMMARY
           echo "" >> $GITHUB_STEP_SUMMARY
-          echo "### Next Steps" >> $GITHUB_STEP_SUMMARY
-          echo "1. Publish CLI to npm locally: \`cd packages/cli && npm publish --access public\`" >> $GITHUB_STEP_SUMMARY
-          echo "2. Verify: \`bunx vibetracking@latest --version\`" >> $GITHUB_STEP_SUMMARY
+          echo "3. Verify installation:" >> $GITHUB_STEP_SUMMARY
+          echo "   \`\`\`bash" >> $GITHUB_STEP_SUMMARY
+          echo "   bunx vibetracking@${{ steps.version.outputs.version }} --version" >> $GITHUB_STEP_SUMMARY
+          echo "   \`\`\`" >> $GITHUB_STEP_SUMMARY

packages/cli/package.json

@@ -32,6 +32,7 @@
     "prepublishOnly": "bun run build"
   },
   "dependencies": {
+    "@starknetid/vibetracking-core": "0.2.0",
     "clipboardy": "^5.0.2",
     "commander": "^14.0.2",
     "csv-parse": "^5.6.0",