bench: add 1000x params and big-bench job; include Crystal reliably; auto-install deps in CI; add plot+report; README benchmark table and heavy-run docs; ignore GIGAPLAN.md

Author: 0xrinegade

.github/workflows/bench.yml

@@ -16,10 +16,9 @@ jobs:
         run: |
           cargo build -p bmssp
           cargo test -p bmssp --all-features -- --nocapture
-      - name: Install minimal deps
+      - name: Install deps (all languages)
         run: |
-          sudo apt-get update
-          sudo apt-get install -y build-essential default-jdk elixir erlang
+          bash scripts/install_deps.sh --yes || true
           python3 -m pip install --user -r bench/requirements.txt || true
       - name: Quick smoke
         run: |
@@ -39,30 +38,9 @@ jobs:
       - name: Cargo test (unit)
         run: |
           cargo test -p bmssp --verbose
-      - name: Install Crystal
+      - name: Install all deps (Linux)
         run: |
-          sudo apt-get update
-          sudo apt-get install -y curl gnupg build-essential
-          curl -fsSL https://dist.crystal-lang.org/apt/setup.sh | sudo bash
-          sudo apt-get install -y crystal shards
-      - name: Install JDK/Kotlin, Elixir, Erlang (with SDKMAN fallback)
-        run: |
-          set -euxo pipefail
-          sudo apt-get update
-          sudo apt-get install -y default-jdk elixir erlang || true
-          if ! command -v java >/dev/null; then sudo apt-get install -y default-jdk || true; fi
-          if ! command -v kotlinc >/dev/null; then
-            # SDKMAN fallback for Kotlin
-            curl -s "https://get.sdkman.io" | bash
-            source "$HOME/.sdkman/bin/sdkman-init.sh"
-            sdk install java 21.0.3-tem || true
-            sdk install kotlin 2.0.20 || true
-            kotlinc -version || true
-          fi
-      - name: Install Nim
-        uses: iffy/install-nim@v5
-        with:
-          version: stable
+          bash scripts/install_deps.sh --yes
       - name: Cache cargo
         uses: actions/cache@v4
         with:
@@ -80,3 +58,39 @@ jobs:
         with:
           name: results
           path: results
+
+  big-bench-1000x:
+    # manual trigger only to avoid heavy CI on every push
+    if: ${{ github.event_name == 'workflow_dispatch' }}
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: dtolnay/rust-toolchain@stable
+      - name: Install all deps
+        run: |
+          bash scripts/install_deps.sh --yes
+          python3 -m pip install --user -r bench/requirements.txt || true
+      - name: Cache cargo
+        uses: actions/cache@v4
+        with:
+          path: |
+            ~/.cargo/registry
+            ~/.cargo/git
+            target
+          key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
+      - name: Run 1000x bench
+        run: |
+          python3 bench/runner.py --params bench/params_1000x.yaml --release --jobs 4 --timeout-seconds 1800 --out results-1000x
+      - name: Generate plots
+        run: |
+          python3 bench/plots.py results-1000x/agg-*\.csv --out results-1000x || true
+      - name: Build report
+        run: |
+          CSV=$(ls -1 results-1000x/agg-*.csv | tail -n1)
+          META=$(ls -1 results-1000x/meta-*.yaml | tail -n1 || true)
+          python3 bench/make_report.py --csv "$CSV" ${META:+--meta "$META"} --out results-1000x
+      - name: Upload 1000x artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: results-1000x
+          path: results-1000x

.gitignore

@@ -26,3 +26,6 @@ impls/erlang/*.beam
 # OS cruft
 .DS_Store
 Thumbs.db
+
+# repo-local docs (not for git)
+GIGAPLAN.md

GIGAPLAN.md

@@ -4,6 +4,17 @@ Implementation of the algorithm in multiple languages to compare performance in
 
 Languages currently wired in this repo: Rust, C, C++, Nim, Crystal, Kotlin (JAR), Elixir (.exs), Erlang (.erl).
 
+## Benchmark snapshot
+
+| impl          | lang   | graph | n    | m    | k | B  | seed | threads | time_ns | popped | edges_scanned | heap_pushes | B_prime | mem_bytes |
+|---------------|--------|-------|------|------|---|----|------|---------|---------|--------|---------------|-------------|---------|-----------|
+| rust-bmssp    | Rust   | grid  | 2500 | 9800 | 4 | 50 | 1    | 1       | 741251  | 868    | 3423          | 1047        | 50      | 241824    |
+| c-bmssp       | C      | grid  | 2500 | 9800 | 4 | 50 | 1    | 1       | 99065   | 1289   | 5119          | 1565        | 50      | 176800    |
+| cpp-bmssp     | C++    | grid  | 2500 | 9800 | 4 | 50 | 1    | 1       | 117480  | 1064   | 4224          | 1261        | 50      | 176800    |
+| kotlin-bmssp  | Kotlin | grid  | 2500 | 9800 | 4 | 50 | 1    | 1       | 5308820 | 1102   | 4386          | 1309        | 50      | 196800    |
+| elixir-bmssp  | Elixir | grid  | 2500 | 9800 | 4 | 50 | 1    | 1       | 5410039 | 870    | 3447          | 1047        | 50      | 196800    |
+| erlang-bmssp  | Erlang | grid  | 2500 | 9800 | 4 | 50 | 1    | 1       | 1155739 | 691    | 2701          | 818         | 50      | 196800    |
+
 Rust implementation of **bounded multi-source shortest paths** (multi-source Dijkstra cut off at `B`).
 
 ## Run
@@ -15,6 +26,9 @@ python3 bench/runner.py --release --out results
 
 # fast iteration
 python3 bench/runner.py --quick --out results-dev --timeout-seconds 20 --jobs 2
+
+# 1000x larger graphs (heavy):
+python3 bench/runner.py --params bench/params_1000x.yaml --release --jobs 4 --timeout-seconds 600 --out results
 ```
 
 ### One-time setup scripts