docs: add runner to ADDING_SIGNERS.md and CLAUDE.md

Author: amilz

CLAUDE.md

@@ -76,10 +76,7 @@ make lint-fix-all
 # Run unit tests
 make test
 
-# Setup test environment (for integration tests)
-make setup-test-env
-
-# Run integration tests
+# Run integration tests (automatically handles environment setup)
 make test-integration
 
 # Run all tests
@@ -88,7 +85,7 @@ cargo test --workspace
 
 #### Integration Test Environment Setup
 
-Integration tests are fully automated using a makefile-based approach that handles sequential test execution:
+Integration tests are fully automated using a Rust test runner binary that handles sequential test execution:
 
 **Quick Start:**
 ```bash
@@ -100,24 +97,28 @@ make test-integration
 2. **Test Environment Setup**: Creates test accounts, tokens, and ATAs
 3. **Sequential Test Phases**: Runs 3 test suites with different configurations
 
-**Test Phases (Auto-managed):**
+**Test Phases (Configured in `tests/src/test_runner/test_cases.toml`):**
 
-**Phase 1: Regular integration tests (26 tests)**
+**Regular Tests**
 - Config: `tests/src/common/fixtures/kora-test.toml` (no auth)
-- Server: Standard Kora RPC server 
 - Tests: Core RPC functionality, token operations, compute budget
 
-**Phase 2: Auth integration tests (4 tests)**  
+**Auth Tests**
 - Config: `tests/src/common/fixtures/auth-test.toml` (auth enabled)
-- Server: Restart with auth middleware
 - Tests: API key and HMAC authentication validation
 
-**Phase 3: Payment address tests (2 tests)**
+**Payment Address Tests**
 - Config: `tests/src/common/fixtures/paymaster-address-test.toml` (payment address)
-- Server: Restart with payment address configuration
 - **CLI ATA Initialization**: Automatically runs `kora rpc initialize-atas` before tests
 - Tests: Payment address validation and wrong destination rejection
 
+**Multi-Signer Tests**
+- Config: `tests/src/common/fixtures/multi-signers.toml`
+- Tests: Multiple signer configurations
+
+**TypeScript Tests**
+- Tests: TypeScript SDK integration tests
+
 **File Structure:**
 ```
 tests/
@@ -132,55 +133,48 @@ tests/
 │   │   │   ├── payment-local.json       # Payment address keypair
 │   │   │   ├── sender-local.json        # Sender keypair
 │   │   │   └── usdc-mint-local.json     # USDC mint keypair
-│   │   └── setup_test_env.rs            # Test environment setup
+│   │   └── setup.rs                     # Test environment setup
+│   ├── test_runner/                     # Test runner modules
+│   │   ├── accounts.rs                  # Account management
+│   │   ├── commands.rs                  # Test command execution
+│   │   ├── config.rs                    # Test configuration
+│   │   ├── kora.rs                      # Kora server management
+│   │   ├── output.rs                    # Output handling
+│   │   ├── test_cases.toml              # Test phase configurations
+│   │   └── validator.rs                 # Solana validator management
 │   └── bin/
-│       └── setup_test_env.rs            # Binary wrapper for setup
+│       └── test_runner.rs               # Main test runner binary
 ├── integration/                         # Regular integration tests
 ├── auth/                                # Authentication tests
 └── payment-address/                     # Payment address tests
 ```
 
-**Manual Test Commands (if needed):**
+**Test Runner Commands:**
 ```bash
-# Setup test environment only
-make setup-test-env
+# Run all integration tests (default)
+make test-integration
+
+# Run with verbose output
+make test-integration-verbose
 
-# Clean up test environment
-make clean-test-env
+# Force refresh test accounts (ignore cached)
+make test-integration-fresh
 
-# Run specific test phase
-cargo test -p tests --test integration
-cargo test -p tests --test auth  
-cargo test -p tests --test payment-address
+# Run specific test
+cargo run -p tests --bin test_runner -- --filter regular
+cargo run -p tests --bin test_runner -- --filter auth
+cargo run -p tests --bin test_runner -- --filter payment_address
+cargo run -p tests --bin test_runner -- --filter multi_signer
+cargo run -p tests --bin test_runner -- --filter typescript_basic
+cargo run -p tests --bin test_runner -- --filter typescript_auth
 ```
 
 #### Customize Test Environment
 
-The test suite uses environment variables for configuration (checked before falling back to defaults):
-
-| Variable | Description | Default |
-|----------|-------------|---------|
-| `RPC_URL` | Solana RPC endpoint | `http://127.0.0.1:8899` |
-| `TEST_SERVER_URL` | Kora RPC server URL | `http://127.0.0.1:8080` |
-| `TEST_SENDER_KEYPAIR` | Base58 encoded test sender keypair | Built-in test keypair |
-| `TEST_RECIPIENT_PUBKEY` | Test recipient public key | Built-in test pubkey |
-| `KORA_PRIVATE_KEY` | Kora fee payer private key | Built-in test keypair |
-| `TEST_USDC_MINT_KEYPAIR` | Test USDC mint keypair | Built-in test mint |
-| `TEST_USDC_MINT_DECIMALS` | USDC mint decimals | `6` |
+The test suite uses environment variables for configuration specified in `tests/src/common/constants.rs`.
 
 Make sure to update the appropriate config file (kora.toml for production, tests/common/fixtures/kora-test.toml for testing) to reflect the public key of TEST_USDC_MINT_KEYPAIR.
 
-**Example with custom test configuration:**
-```bash
-# Create .env file for custom test setup
-echo "RPC_URL=https://api.devnet.solana.com" > .env
-echo "TEST_SENDER_KEYPAIR=your_base58_keypair" >> .env
-echo "KORA_PRIVATE_KEY=your_fee_payer_keypair" >> .env
-
-# Run tests with custom config
-make test-integration
-```
-
 ### Running Services
 
 ```bash
@@ -562,29 +556,28 @@ Both skip `/liveness` endpoint and can be used simultaneously. Implementation us
 - **API tests** - Include example JSON payloads in `tests/examples/`
 - **SDK tests** - TypeScript tests in `sdks/*/test/` directories
 
-## Makefile Structure
+## Test Runner Architecture
 
-The project uses a modular makefile approach for better organization:
+The project uses a Rust-based test runner for integration testing:
 
 ```
-/Makefile                  # Main makefile with includes
-/makefiles/
-├── utils.mk              # Common utilities and functions
-├── rs.mk                 # Rust build commands  
-├── tests_rs.mk           # Integration test orchestration
-└── docs.mk               # Documentation generation
+/tests/src/
+├── bin/test_runner.rs     # Main test runner binary
+├── test_runner/           # Test runner modules
+│   ├── test_cases.toml    # Test phase configurations
+│   ├── accounts.rs        # Account management & caching
+│   ├── commands.rs        # Test execution logic
+│   ├── kora.rs            # Kora server lifecycle
+│   └── validator.rs       # Solana validator management
+└── common/                # Shared test utilities
 ```
 
 **Key Features:**
-- **Sequential Test Execution**: Prevents port conflicts and ensures clean config isolation
-- **Server Lifecycle Management**: Automatic start/stop with proper cleanup
-- **CLI Integration**: Automated `kora rpc initialize-atas` for payment address tests
-- **Parallel Tool Execution**: Optimized for development workflow efficiency
-
-**Core Functions (from utils.mk):**
-- `start_solana_validator` / `stop_solana_validator`: Validator lifecycle
-- `start_kora_server` / `stop_kora_server`: Server lifecycle with config switching
-- `run_integration_phase`: Orchestrates server + CLI + tests for each phase
+- **Rust Test Runner**: Single binary manages all test phases
+- **TOML Configuration**: Test phases defined in `test_cases.toml`
+- **Account Caching**: Reuses test accounts for faster execution
+- **Isolated Ports**: Each test phase uses unique ports to avoid conflicts
+- **TypeScript Integration**: Seamlessly runs TS SDK tests alongside Rust tests
 
 ## Development Guidelines
 

docs/contributors/ADDING_SIGNERS.md

@@ -400,48 +400,74 @@ Define a `example-signer.toml` with your signer's configuration and necessary en
 
 ### Integration Tests
 
-Testing should utilize the existing `sdks/ts/test/integration.test.ts` file. We manage this with an environment variable `KORA_SIGNER_TYPE` that is set to the signer type you are testing.
-
-Setup:
-- `sdks/ts/test/setup.ts`: You will need to add an environment variable to `loadEnvironmentVariables`  to set the signer type to your service.
-- `sdks/ts/package.json`: You will need to add a new test script that uses the signer type you added to the setup script: `"test:integration:your-service": "KORA_SIGNER_TYPE=your-service pnpm test integration.test.ts"`. When executed, the test will run with the signer type set to your service.
-
-Integration testing requires a local Solana test validator and a local Kora node. We can use the [TypeScript Test Makefile](/makefiles/tests_ts.mk) to start a local Solana test validator and a local Kora node by adding a new script for your signer:
-
-```makefile
-# Run TypeScript tests with YourService signer  
-test-ts-integration-your-service:
-	@$(call start_solana_validator)
-	@echo "🚀 Starting Kora node with YourService signer..."
-	@$(call stop_kora_server)
-	@cargo run -p kora-cli --bin kora -- --config $(REGULAR_CONFIG) --rpc-url $(TEST_RPC_URL) rpc start --signers-config $(TEST_SIGNERS_YOUR_SERVICE_CONFIG) --port $(TEST_PORT) $(QUIET_OUTPUT) &
-	@echo $$! > .kora.pid
-	@echo "⏳ Waiting for server to start..."
-	@sleep 5
-	@printf "Running TypeScript SDK tests with YourService signer...\n"
-	-@cd sdks/ts && pnpm test:integration:your-service
-	@$(call stop_kora_server)
-	@$(call stop_solana_validator)
+Kora uses a unified test runner (`tests/src/bin/test_runner.rs`) that manages all integration testing phases including TypeScript tests. To add tests for your new signer:
+
+#### 1. Add Test Configuration
+
+Create a new signer configuration file in `tests/src/common/fixtures/` for your service:
+
+```toml
+# tests/src/common/fixtures/signers-your-service.toml
+[[signers]]
+name = "yourservice_main"
+type = "your_service"
+api_key_env = "YOUR_SERVICE_API_KEY"
+api_secret_env = "YOUR_SERVICE_API_SECRET"
+wallet_id_env = "YOUR_SERVICE_WALLET_ID"
 ```
 
-And add your new script to the `test-ts-signers` script:
+#### 2. Add Test Phase to Test Runner
+
+Update `tests/src/test_runner/test_cases.toml` to include a test phase for your signer:
 
-```makefile
-test-ts-signers: test-ts-integration-your-service # ... test-ts-existing-signer-tests
+```toml
+[test.your_service]
+name = "YourService Signer Tests"
+config = "tests/src/common/fixtures/kora-test.toml"
+signers = "tests/src/common/fixtures/signers-your-service.toml"
+port = "8090"  # Use a unique port
+tests = ["your_service"]
 ```
 
-Make sure your local environment is set up:
+#### 3. TypeScript SDK Integration
 
-```makefile
+For TypeScript SDK testing with your signer:
+
+1. Update `sdks/ts/test/setup.ts` to recognize your signer type:
+   - Add environment variable handling for `KORA_SIGNER_TYPE=your-service`
+
+2. Add a test script in `sdks/ts/package.json`:
+   ```json
+   "test:integration:your-service": "KORA_SIGNER_TYPE=your-service pnpm test integration.test.ts"
+   ```
+
+#### 4. Running Tests
+
+Make sure your environment is set up:
+
+```bash
+# Install binaries and dependencies
 make install
 make install-ts-sdk
 make build-ts-sdk
+
+# Set environment variables for your service
+export YOUR_SERVICE_API_KEY="your_key"
+export YOUR_SERVICE_API_SECRET="your_secret"
+export YOUR_SERVICE_WALLET_ID="your_wallet"
 ```
 
-Now you can test your integration by running:
+Run tests using the unified test runner:
 
 ```bash
-make test-ts-integration-your-service
+# Run all integration tests (includes your new signer phase)
+make test-integration
+
+# Run tests with verbose output
+make test-integration-verbose
+
+# Run specific test phase with filter
+cargo run -p tests --bin test_runner -- --phases your_service
 ```