review cryptography (Run ID: openSVM_tornado-svm_issue_3_43e335ac)

.github/workflows/README.md

@@ -0,0 +1,163 @@
+# Tornado-SVM GitHub Actions Workflows
+
+## Build Workflow
+
+### Workflow: `build.yml`
+
+**Purpose:** Build, test, and validate the Tornado-SVM codebase using the Bun JavaScript runtime and Solana build tools.
+
+### Trigger Methods:
+
+1. **On Push:** Runs on all branch pushes and version tags
+2. **On Pull Request:** Runs on all pull requests
+
+### What the Workflow Does:
+
+1. Sets up Bun and Rust toolchains
+2. Installs Solana build tools
+3. Builds the Solana program using Cargo build-sbf
+4. Runs program tests
+5. Lints the code with Clippy
+6. Builds and tests the client
+
+### Technologies Used:
+
+- **Bun:** Fast JavaScript runtime and package manager
+- **Rust:** Primary language for the Solana program
+- **Solana CLI:** For building and testing Solana programs
+
+### Solana CLI Installation
+
+The workflow automatically installs the Solana CLI using the following process:
+
+```bash
+# Install Solana CLI tools
+sh -c "$(curl -sSfL https://release.solana.com/v1.16.0/install)"
+
+# Add to GitHub Actions PATH
+echo "$HOME/.local/share/solana/install/active_release/bin" >> $GITHUB_PATH
+
+# Also add to current shell session
+export PATH="$HOME/.local/share/solana/install/active_release/bin:$PATH"
+```
+
+This ensures that the Solana binaries are available for all steps in the workflow that require them.
+
+## Testnet Transaction Metrics Workflow
+
+This workflow automates the process of running Tornado-SVM privacy solution transactions on Solana testnet and generating comprehensive metrics reports.
+
+### Workflow: `tornado_testnet_transaction.yml`
+
+**Purpose:** Execute the complete Tornado-SVM transaction flow on Solana testnet and collect detailed performance metrics.
+
+### Trigger Methods:
+
+1. **Manual Trigger:** Run the workflow on-demand via GitHub UI with configurable parameters
+2. **Scheduled Runs:** Automatically runs weekly on Sundays at midnight UTC
+3. **Pull Request Trigger:** Runs on PRs to the master branch that modify core files
+
+### Configurable Parameters:
+
+- **Denomination:** Amount of SOL to use in the transaction (default: 1 SOL)
+- **Merkle Tree Height:** Height of the Merkle tree for the Tornado instance (default: 20)
+- **RPC URL:** Custom Solana RPC URL (defaults to testnet)
+
+### What the Workflow Does:
+
+1. Sets up Bun runtime and the Solana toolchain
+2. Creates a new Solana wallet and requests an airdrop
+3. Deploys the Tornado-SVM program to the Solana testnet
+4. Initializes a new Tornado instance
+5. Performs a complete deposit and withdrawal flow with zkSNARK proofs
+6. Captures detailed metrics at each step including:
+   - Execution times for each phase
+   - Transaction signatures
+   - Compute unit consumption
+   - Gas fees
+   - Transaction details
+7. Generates a comprehensive markdown report with visualizations
+8. Creates a GitHub job summary
+9. Uploads all reports and raw metrics as artifacts
+
+### Artifacts Generated:
+
+- **transaction_report.md:** Complete markdown report with all metrics and visualizations
+- **metrics/*.json:** Raw JSON data for transaction details
+- **metrics/execution_times.txt:** Detailed timing measurements for each phase
+
+### Using the Report:
+
+1. Download the artifact from the completed workflow run
+2. Open the markdown report to view all metrics and visualizations
+3. The report includes:
+   - Executive summary
+   - Configuration details
+   - Transaction logs
+   - Detailed metrics for each transaction
+   - Explorer links for all on-chain activity
+   - Visualizations of the transaction flow and zkSNARK process
+   - Solana network stats during the test
+
+### Example Usage
+
+To manually trigger the workflow with custom parameters:
+
+1. Go to the "Actions" tab in the GitHub repository
+2. Select "Tornado SVM Testnet Transaction Test" workflow
+3. Click "Run workflow"
+4. Enter your desired parameters (denomination, Merkle tree height, RPC URL)
+5. Click "Run workflow"
+6. Once completed, download the artifacts from the workflow run
+
+### Troubleshooting
+
+#### Solana CLI Not Found
+
+If you encounter the error `solana: command not found`, check the following:
+
+1. Verify that the Solana CLI installation step completed successfully
+2. The workflow now adds Solana binaries to GitHub's persistent PATH variable (`$GITHUB_PATH`), ensuring all subsequent steps can access the commands
+3. We also add `$HOME/.cargo/bin` to PATH to pick up cargo-build-sbf and cargo-test-sbf
+4. The workflow no longer needs explicit PATH exports in each step
+5. The transaction script has robust error handling to provide detailed diagnostic information when Solana is not found
+6. You can use the `SOLANA_PATH` environment variable to override the default Solana binary location
+
+#### Cargo Lock File Version Compatibility
+
+If you encounter Cargo lock file version compatibility issues:
+
+1. The workflow now explicitly updates Cargo to the latest stable version
+2. We've added a specific step that runs `rustup update stable` and `rustup default stable`
+3. Cargo version is explicitly checked and logged for troubleshooting
+4. The workflow now intelligently checks if the installed Cargo version is compatible with Cargo.lock version 4:
+   ```bash
+   CARGO_VERSION=$(cargo --version | grep -oE '[0-9]+\.[0-9]+\.[0-9]+')
+   MAJOR=$(echo "$CARGO_VERSION" | cut -d'.' -f1)
+   MINOR=$(echo "$CARGO_VERSION" | cut -d'.' -f2)
+   if [ "$MAJOR" -lt 1 ] || ([ "$MAJOR" -eq 1 ] && [ "$MINOR" -lt 70 ]); then
+     # If Cargo is too old, upgrade it again
+     curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain stable --profile minimal
+   fi
+   ```
+5. The workflow automatically regenerates the Cargo.lock file to ensure it uses a format compatible with the current Cargo version
+6. After regeneration, it explicitly verifies the lock file format with `grep -q 'version = 4' Cargo.lock`
+7. Any existing Cargo.lock is deleted and freshly regenerated to avoid format conflicts
+8. Detailed debugging output is provided if the Cargo.lock generation fails
+
+#### Build Command Not Found
+
+If you encounter errors with `cargo build-sbf` or `cargo build-bpf`:
+
+1. The workflow now checks if commands are available using `help` flags
+2. It tries both SBF (newer) and BPF (older) variants
+3. If needed, it runs `solana-install update` to get the latest build tools
+4. PATH is updated to include all possible locations for Cargo and Solana binaries
+
+#### Notifications
+
+The workflow previously used Telegram for notifications, which has been replaced with:
+
+1. Console-based logging for better workflow compatibility
+2. No external dependencies or tokens required
+3. Clear notification messages in the workflow logs

.github/workflows/build.yml

@@ -3,36 +3,152 @@ name: build
 on:
   push:
     branches: ['*']
-    tags: ['v[0-9]+.[0-9]+.[0-9]+']
+    tags: ['v[0-9]+.[0-9]+.[0-9]+'] 
   pull_request:
 
 jobs:
   build:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
-        uses: actions/checkout@v2
-      - uses: actions/setup-node@v1
-        with:
-          node-version: 12
-      - run: yarn install
-      - run: yarn download
-      - run: cp .env.example .env
-      - run: npx ganache-cli > /dev/null &
-      - run: npm run migrate:dev
-      - run: yarn test
-      - run: node src/cli.js test
-      - run: yarn lint
-      - run: yarn coverage
-      - name: Coveralls
-        uses: coverallsapp/github-action@master
+        uses: actions/checkout@v3
+
+      - name: Setup Bun
+        uses: oven-sh/setup-bun@v1
         with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-      - name: Telegram Failure Notification
-        uses: appleboy/[email protected]
-        if: failure()
+          bun-version: latest
+
+      - name: Install dependencies
+        run: bun install
+
+      # Rust setup and build with explicit update to latest version
+      - name: Install and Update Rust
+        uses: dtolnay/rust-toolchain@stable
         with:
-          message: ❗ Build failed for [${{ github.repository }}](https://github.com/${{ github.repository }}/actions) because of ${{ github.actor }}
-          format: markdown
-          to: ${{ secrets.TELEGRAM_CHAT_ID }}
-          token: ${{ secrets.TELEGRAM_BOT_TOKEN }}
+          components: rustfmt, clippy
+
+      - name: Update Cargo to latest stable
+        run: |
+          # Update to the latest stable Rust toolchain
+          rustup update stable
+          rustup default stable
+          # Check Cargo version explicitly
+          cargo --version
+          echo "Using Cargo from: $(which cargo)"
+          # Ensure we can handle Cargo.lock version 4
+          CARGO_VERSION=$(cargo --version | grep -oE '[0-9]+\.[0-9]+\.[0-9]+')
+          echo "Cargo version: $CARGO_VERSION"
+          # Check if Cargo version is new enough for lock file version 4
+          MAJOR=$(echo "$CARGO_VERSION" | cut -d'.' -f1)
+          MINOR=$(echo "$CARGO_VERSION" | cut -d'.' -f2)
+          if [ "$MAJOR" -lt 1 ] || ([ "$MAJOR" -eq 1 ] && [ "$MINOR" -lt 70 ]); then
+            echo "Warning: Cargo version $CARGO_VERSION may not fully support Cargo.lock version 4 format"
+            echo "Attempting to update Cargo again"
+            curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain stable --profile minimal
+            source "$HOME/.cargo/env"
+            cargo --version
+          else
+            echo "Cargo $CARGO_VERSION supports Cargo.lock version 4 format"
+          fi
+
+      - name: Regenerate Cargo.lock
+        run: |
+          # Remove any existing Cargo.lock
+          if [ -f Cargo.lock ]; then
+            echo "Removing existing Cargo.lock"
+            rm Cargo.lock
+          fi
+          # Regenerate Cargo.lock with the latest Cargo version
+          echo "Regenerating Cargo.lock"
+          cargo generate-lockfile
+          echo "Cargo.lock regenerated successfully"
+
+          # Verify the Cargo.lock format
+          if [ -f Cargo.lock ]; then
+            echo "Checking Cargo.lock format..."
+            # Quick check to see if it's a version 4 format (contains version = 4)
+            if grep -q 'version = 4' Cargo.lock; then
+              echo "Confirmed: Cargo.lock is using version 4 format"
+            else
+              echo "Warning: Cargo.lock may not be using version 4 format"
+              # For debugging purposes, show the first few lines
+              head -5 Cargo.lock
+            fi
+          else
+            echo "Error: Cargo.lock was not generated!"
+            exit 1
+          fi
+
+      - name: Install Solana CLI
+        run: |
+          # Install Solana CLI tools
+          sh -c "$(curl -sSfL https://release.solana.com/v1.16.0/install)"
+          # Add Solana to PATH for this job
+          echo "$HOME/.local/share/solana/install/active_release/bin" >> $GITHUB_PATH
+          # Also add to PATH for current shell session
+          export PATH="$HOME/.local/share/solana/install/active_release/bin:$PATH"
+          # Verify installation
+          solana --version
+
+      - name: Build Solana program
+        run: |
+          # Ensure Solana binaries are in PATH
+          export PATH="$HOME/.local/share/solana/install/active_release/bin:$PATH"
+          # Try the newer cargo build-sbf command first, fall back to cargo build-bpf if not available
+          # First check if the commands are directly available
+          if cargo build-sbf --help &> /dev/null; then
+            echo "Using cargo build-sbf"
+            cargo build-sbf
+          elif cargo build-bpf --help &> /dev/null; then
+            echo "Using cargo build-bpf"
+            cargo build-bpf
+          else
+            echo "Installing Solana BPF/SBF tools..."
+            solana-install update
+            # Add Solana's .cargo/bin to PATH (where cargo-build-bpf is installed)
+            export PATH="$HOME/.cargo/bin:$PATH"
+            # Try again after update
+            if cargo build-sbf --help &> /dev/null; then
+              echo "Using cargo build-sbf after update"
+              cargo build-sbf
+            else
+              echo "Using cargo build-bpf after update"
+              cargo build-bpf
+            fi
+          fi
+
+      - name: Run Solana tests
+        run: |
+          # Ensure Solana binaries are in PATH
+          export PATH="$HOME/.local/share/solana/install/active_release/bin:$PATH"
+          export PATH="$HOME/.cargo/bin:$PATH"
+          # Try the newer cargo test-sbf command first, fall back to cargo test-bpf if not available
+          if cargo test-sbf --help &> /dev/null; then
+            echo "Using cargo test-sbf"
+            cargo test-sbf
+          elif cargo test-bpf --help &> /dev/null; then
+            echo "Using cargo test-bpf"
+            cargo test-bpf
+          else
+            echo "Installing Solana BPF/SBF tools..."
+            solana-install update
+            # Add Solana's .cargo/bin to PATH (where cargo-test-bpf is installed)
+            export PATH="$HOME/.cargo/bin:$PATH"
+            # Try again after update
+            if cargo test-sbf --help &> /dev/null; then
+              echo "Using cargo test-sbf after update"
+              cargo test-sbf
+            else
+              echo "Using cargo test-bpf after update"
+              cargo test-bpf
+            fi
+          fi
+
+      - name: Run Cargo Clippy
+        run: cargo clippy -- -D warnings
+
+      - name: Build client
+        run: cd client && bun install
+
+      - name: Run client tests
+        run: cd client && bun test