Skip to content

DEVELOPMENT.md

Latest commit

 

History

History
454 lines (337 loc) · 14 KB

DEVELOPMENT.md

File metadata and controls

454 lines (337 loc) · 14 KB
Raw

Development Guide

This guide explains the architecture, development practices, and how to extend the Stellar bulk payment system.

Project Architecture

Layered Design

The project follows a clean layered architecture with clear separation of concerns:

┌─────────────────────────────────────────────────────────┐
│           UI Layer (Web & CLI)                          │
│  (pages, components, cli/index.ts)                      │
└─────────────────────────────────────────────────────────┘
                        ↓
┌─────────────────────────────────────────────────────────┐
│        Application Layer (API Routes)                   │
│  (app/api/batch-submit/route.ts)                        │
└─────────────────────────────────────────────────────────┘
                        ↓
┌─────────────────────────────────────────────────────────┐
│      Core Business Logic (lib/stellar/)                 │
│  - Types, Validation, Parsing, Batching, Services      │
└─────────────────────────────────────────────────────────┘
                        ↓
┌─────────────────────────────────────────────────────────┐
│     External Services (Stellar SDK)                     │
│  (stellar-sdk)                                          │
└─────────────────────────────────────────────────────────┘

Core Library Modules

types.ts

  • Defines all TypeScript interfaces
  • Single source of truth for data structures
  • No dependencies, pure types

validator.ts

  • Input validation logic
  • Validates payments, configuration, and batches
  • Clear error messages for debugging
  • Used by both CLI and Web UI

parser.ts

  • Parses JSON and CSV files
  • Flexible input handling
  • Validates structure and format

batcher.ts

  • Splits payments into transaction batches
  • Respects Stellar's 100-operation limit
  • Utility functions for asset parsing and summaries

stellar-service.ts

  • Main service for Stellar operations
  • Handles transaction building, signing, and submission
  • Manages sequence numbers and error handling
  • Returns structured results

Key Design Decisions

1. No ORM or Additional Abstraction

The service directly uses stellar-sdk without intermediate layers. This keeps the code simple and maintainable.

2. Separation of Concerns

  • Input Parsing: Handled separately from validation
  • Validation: Independent of business logic
  • Batching: Pure function with no side effects
  • Stellar Operations: Isolated in StellarService

3. Structured Results

All operations return structured result objects rather than throwing exceptions. This allows:

  • Partial success reporting (some payments succeed, others fail)
  • Detailed error information per recipient
  • Easy JSON serialization

4. Environment-Based Configuration

Secret keys come from environment variables, never from configuration files. This enforces security best practices.

Adding New Features

Adding a New Asset Type Handler

If you need special handling for a specific asset:

  1. Update parseAsset() in lib/stellar/utils.ts — this is the single shared parser used by both batcher.ts and server.ts
  2. Add validation in validator.ts
  3. Update type definitions in types.ts
  4. Add tests in tests/

Important — server-side signing: StellarService in server.ts (used by /api/batch-submit with STELLAR_SECRET_KEY) builds payment operations by calling parseAsset from lib/stellar/utils.ts. This shared parser handles "XLM", "native", and "CODE:ISSUER" strings. Do not introduce a local parseAsset in server.ts; doing so will break issued-asset batches (USDC, etc.) on the server-submit path.

Adding Rate Limiting

To add rate limiting per account:

  1. Create lib/stellar/rate-limiter.ts
  2. Track submission times per public key
  3. Return error if limit exceeded
  4. Update StellarService to use it

Adding Database Support

For production deployments wanting to track batch history:

  1. Create lib/stellar/persistence.ts
  2. Use database adapter pattern
  3. Store results before returning to user
  4. Add endpoint to retrieve historical results

Adding Multi-Signature Support

For advanced security:

  1. Create lib/stellar/multisig.ts
  2. Handle multiple signers
  3. Update transaction signing in StellarService
  4. Add configuration for signers

Contract Development

Prerequisites

To work on the Soroban smart contracts, you need:

  1. Rust & WASM: Install Rust and add the WASM target: 
    rustup target add wasm32-unknown-unknown
  2. Stellar CLI: Install the Stellar CLI: 
    cargo install --locked stellar-cli --features opt
  3. Docker: Required for running a local Stellar node.

Local Environment Setup

We use the Stellar Quickstart Docker image to run a local network with Soroban RPC enabled.

  1. Start the local node:

    docker-compose up -d

    This starts a standalone network with Horizon on port 8000 and Soroban RPC on port 8001.

  2. Configure Stellar CLI for Local:

    stellar network add --rpc-url http://localhost:8001 --network-passphrase "Standalone Network ; February 2017" local

  3. Generate and Fund Test Keys: Create a local identity for deployment and testing:

    stellar keys generate --network local alice

    Note: In standalone mode, the CLI will automatically try to fund the account via the local friendbot. If you need to fund it manually:

    curl "http://localhost:8000/friendbot?addr=$(stellar keys address alice)"

Building and Testing Contracts

The contracts are located in the contracts/ directory.

  1. Build the contract:

    cd contracts/batch-vesting
cargo build --target wasm32-unknown-unknown --release

  2. Run tests:

    cargo test

Deployment

Deploy to Local

Use the provided script to deploy to your local node:

./scripts/deploy-contract.sh local alice

Deploy to Testnet

  1. Configure Testnet: 
    stellar network add --rpc-url https://soroban-testnet.stellar.org:443 --network-passphrase "Test SDF Test Network ; September 2015" testnet
  2. Generate/Add Testnet Account: 
    stellar keys generate --network testnet deployer
  3. Deploy: 
    ./scripts/deploy-contract.sh testnet deployer

Contract Architecture: Batch Vesting

The batch-vesting contract allows a sender to deposit tokens for multiple recipients with a shared unlock time.

  • Storage: Uses Persistent storage for vesting records.
  • Limits: Enforces MAX_BATCH_SIZE = 100 to ensure transactions fit within ledger limits.
  • Security: Only the recipient can claim their funds, and only after the unlock_time.

Contract Upgrades

The contract uses Soroban's standard upgradeability pattern. Upgrades are gated by admin authorization and a configurable timelock to ensure security and transparency.

  1. Propose Upgrade: The admin calls propose_upgrade(new_wasm_hash). This records the hash and calculates an execution timestamp based on the current upgrade_timelock configuration.
  2. Wait: The community/users have a window to review the new code before the timelock expires.
  3. Execute Upgrade: After the timelock expires, the admin calls execute_upgrade(). This replaces the contract code with the new WASM while preserving all current storage.

Events for Auditability:

  • UpgradeProposed(new_wasm_hash, execute_at)
  • UpgradeExecuted(new_wasm_hash)

Testing Strategy

Unit Tests

Test individual functions in isolation:

// Test validator
validatePaymentInstruction(...)
validatePaymentInstructions(...)

// Test parser
parseJSON(...)
parseCSV(...)

// Test batcher
createBatches(...)
getBatchSummary(...)

Integration Tests

Test the full flow with mock Stellar server:

// Would require mocking stellar-sdk
const service = new StellarService(config);
const result = await service.submitBatch(payments);
// Assert result structure

Manual Testing

  1. CLI Testing:

    STELLAR_SECRET_KEY="..." node cli/index.ts \
  --input examples/payments.json \
  --network testnet

  2. Web UI Testing:

    npm run dev
# Open http://localhost:3000
# Upload test file

  3. Testnet Validation:

Code Quality Guidelines

Naming Conventions

  • Functions: camelCase, verbs for actions (validatePayment, createBatch)
  • Variables: camelCase, nouns for data
  • Types: PascalCase
  • Constants: UPPER_SNAKE_CASE

Comments

Write comments for:

  • Complex algorithms (e.g., sequence number management)
  • Why, not what (code shows what it does)
  • Edge cases and limitations
  • Transaction building logic

Error Handling

  • Throw errors for programmer mistakes (invalid config)
  • Return errors in results for user input issues
  • Provide context in error messages
  • Include what was expected and what was received

Validation

  • Validate at entry points (parseInput, API routes)
  • Validate early, fail fast
  • Provide specific error messages for each field
  • Test validation with both valid and invalid inputs

Performance Considerations

Sequence Number Management

The current implementation increments sequence numbers in memory. For production:

// Current approach (in-memory)
let sequenceNumber = BigInt(sourceAccount.sequenceNumber);

// Production approach
// - Use database to track sequence numbers
// - Or implement read-modify-write with retry logic

Batch Size Optimization

Current implementation uses hardcoded max of 100 operations. Consider:

// Could be configurable based on fee market
const maxOps = network === 'testnet' ? 100 : 50;

Error Retry Logic

Consider implementing retry for transient failures:

async submitWithRetry(transaction, maxRetries = 3) {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await this.submitTransaction(transaction);
    } catch (error) {
      if (isTransientError(error) && i < maxRetries - 1) {
        await sleep(Math.pow(2, i) * 1000); // exponential backoff
      } else {
        throw;
      }
    }
  }
}

Debugging

Enable Debug Logging

Add to stellar-service.ts:

private log(message: string, data?: any) {
  if (process.env.DEBUG) {
    console.error(`[StellarService] ${message}`, data);
  }
}

Inspect Transactions

Before submitting:

const transactionEnvelope = transaction.toXDR();
console.log('Transaction XDR:', transactionEnvelope);

Monitor Stellar Network

Deployment Checklist

  • All tests pass
  • No console.log statements (use proper logging)
  • Environment variables documented
  • Rate limiting configured
  • Error messages don't expose internal details
  • Testnet validation completed
  • Monitoring/alerting configured
  • Backup and recovery plan in place
  • Security review completed

Security Best Practices

  1. Secret Key Management

    • Never log secret keys
    • Always use environment variables
    • Consider using secret management service

  2. Input Validation

    • Validate all external input
    • Check transaction limits
    • Verify addresses are valid Stellar keys

  3. Error Messages

    • Don't reveal internal details
    • Don't expose secrets in errors
    • Log detailed info server-side only

  4. Transaction Safety

    • Always use sequence numbers
    • Implement idempotency if needed
    • Consider transaction timeouts

  5. Network Security

    • Use HTTPS for web UI
    • Validate Stellar server certificates
    • Use testnet for development

Troubleshooting

"Account does not exist" Error

The account hasn't been funded yet.

  • Testnet: Use friendbot.stellar.org
  • Mainnet: Fund with at least 1 XLM

"tx_bad_seq" Error

Sequence number mismatch. Likely causes:

  • Multiple submissions in parallel
  • Account state changed between reads
  • Need to implement sequence number tracking

"op_no_destination" Error

Recipient address is invalid:

  • Check Stellar address format
  • Verify address exists and is funded

Large Batch Timeouts

Batch is too large for single transaction:

  • Reduce maxOperationsPerTransaction
  • Split into multiple batches
  • Consider network congestion

Future Improvements

  1. Event Sourcing: Store all actions for audit trail
  2. WebSocket Support: Real-time progress updates
  3. GraphQL API: More flexible querying
  4. Multi-language SDKs: Python, Go, Rust versions
  5. Scheduler: Automated batch submissions at intervals
  6. Analytics Dashboard: Track batch history and metrics