Skip to content

webhook/crates/soroban-reconciler/ move to crates/ folder #559

Description

@sotoJ24

Issue Summary

webhook/crates/soroban-reconciler/ is located inside the webhook/ directory — a TypeScript-only service layer. All other Rust crates correctly live at the repository root under crates/:

crates/                          ← correct Rust home
├── chunk-processor/
├── stellar-utils/
└── webhook-verifier/

webhook/crates/                  ← wrong — Rust inside TS service
└── soroban-reconciler/          ← must move to crates/

The webhook/ directory should contain only TypeScript source, configuration, and Docker files. Rust crates are language-level infrastructure shared across the repository and belong at the root crates/ workspace alongside chunk-processor, stellar-utils, and webhook-verifier.

This misplacement also prevents adding soroban-reconciler to a shared Cargo workspace — meaning it has its own Cargo.lock, cannot share dependencies with other crates, and cannot be built with cargo build --workspace from crates/.

Related PR: #542
Related Issue: #538

Type of Issue

  • Bug
  • Feature Request
  • Documentation
  • Performance
  • Security
  • Other — Developer Experience / Structural Chore

Current Behavior

backend-SafeTrust/
├── crates/                          ← 3 crates correctly here
│   ├── chunk-processor/
│   ├── stellar-utils/
│   └── webhook-verifier/
└── webhook/
    └── crates/
        └── soroban-reconciler/      ← incorrectly placed here
            ├── Cargo.lock           ← own lockfile, no workspace
            ├── Cargo.toml           ← standalone crate
            ├── package.json
            └── src/
                ├── lib.rs
                ├── reconciler.rs
                ├── rpc_client.rs
                └── types.rs

Running cargo build --workspace from crates/ does NOT build soroban-reconciler — it is invisible to the workspace.

Expected Behavior

backend-SafeTrust/
├── crates/
│   ├── Cargo.toml                   ← workspace root (new file)
│   ├── Cargo.lock                   ← single shared lockfile
│   ├── chunk-processor/
│   ├── stellar-utils/
│   ├── webhook-verifier/
│   └── soroban-reconciler/          ← moved here
│       ├── Cargo.toml               ← workspace member
│       ├── package.json
│       └── src/
│           ├── lib.rs
│           ├── reconciler.rs
│           ├── rpc_client.rs
│           └── types.rs
└── webhook/                         ← TypeScript only, no Rust
    ├── Dockerfile
    ├── package.json
    ├── tsconfig.json
    └── src/

cargo build --workspace from crates/ builds all 4 crates in one command.

Reproduction Steps

  1. cd webhook/crates/soroban-reconciler && cargo build
    → builds in isolation, no workspace
  2. cd crates && cargo build --workspace
    → soroban-reconciler is NOT included
  3. Observe: two separate Cargo.lock files for the same repository

Implementation Steps

1 — Move the crate

mv webhook/crates/soroban-reconciler crates/soroban-reconciler
rmdir webhook/crates

2 — Create crates/Cargo.toml workspace root

# crates/Cargo.toml
[workspace]
members = [
    "chunk-processor",
    "stellar-utils",
    "webhook-verifier",
    "soroban-reconciler",
]
resolver = "2"

[workspace.dependencies]
neon       = { version = "0.10", default-features = false, features = ["napi-6"] }
serde      = { version = "1", features = ["derive"] }
serde_json = "1"
tokio      = { version = "1", features = ["full"] }
hex        = "0.4"
base64     = "0.22"

3 — Remove individual Cargo.lock from moved crate

# Workspace manages a single Cargo.lock at crates/Cargo.lock
rm crates/soroban-reconciler/Cargo.lock

4 — Update import path in webhook handler

// webhook/src/routes/reconciliation/sync-escrows.handler.js
// Before:
const { queryEscrowState, reconcileSingle } =
  require('../crates/soroban-reconciler')

// After:
const { queryEscrowState, reconcileSingle } =
  require('../../crates/soroban-reconciler')

5 — Update webhook/Dockerfile build path

# Before — builds only soroban-reconciler in isolation
RUN cd webhook/crates/soroban-reconciler && \
    cargo build --release && \
    node copy-native.js

# After — builds entire workspace, all crates in one step
RUN cd crates && cargo build --workspace --release
COPY crates/soroban-reconciler/copy-native.js ./crates/soroban-reconciler/
RUN node crates/soroban-reconciler/copy-native.js

6 — Update .gitignore

# Add workspace target directory
echo "crates/target/" >> .gitignore

# Remove old entry if present
# webhook/crates/target/ → no longer needed

7 — Verify

# All 4 crates build together
cd crates && cargo build --workspace
# → should show: Compiling chunk-processor, stellar-utils,
#                webhook-verifier, soroban-reconciler

# Webhook service still starts
cd webhook && npm run dev
# → [api] Webhook service running at http://localhost:3001

# Tests still pass
bin/test

Relevant Files

File Change
webhook/crates/soroban-reconciler/ Moved to crates/soroban-reconciler/
webhook/crates/ Deleted — directory removed entirely
crates/Cargo.toml New — Cargo workspace root
crates/Cargo.lock New — generated by workspace build
crates/soroban-reconciler/Cargo.lock Deleted — workspace manages lockfile
webhook/src/routes/reconciliation/sync-escrows.handler.js Update require path
webhook/Dockerfile Update build command to cargo build --workspace
.gitignore Add crates/target/

Acceptance Criteria

  • webhook/crates/ directory no longer exists
  • crates/soroban-reconciler/ exists alongside other crates
  • crates/Cargo.toml workspace root lists all 4 members
  • Single crates/Cargo.lock — no per-crate lockfiles
  • cd crates && cargo build --workspace builds all 4 crates successfully
  • cd webhook && npm run dev starts at http://localhost:3001
  • require('../../crates/soroban-reconciler') resolves correctly in handler
  • All Karate tests pass: bin/test
  • GitHub language bar reflects Rust at repository root level

Supporting Information

Architectural principle

backend-SafeTrust/
├── crates/     ← Rust infrastructure layer (all crates)
├── metadata/   ← Hasura configuration layer
├── migrations/ ← PostgreSQL schema layer
├── seeds/      ← PostgreSQL data layer
├── tests/      ← Karate integration test layer
└── webhook/    ← TypeScript HTTP service layer (no Rust)

Each directory has a single responsibility. webhook/ is the TypeScript HTTP service — it USES Rust crates via Neon bindings but does not CONTAIN them. This separation means:

  • A contributor working on Rust can cd crates and work without touching webhook/
  • A contributor working on TypeScript can cd webhook and work without a Rust toolchain
  • cargo build --workspace always builds the complete Rust layer in one command
  • Future crates (escrow-state-machine, pg-bulk-upsert, stellar-xdr) are added to crates/Cargo.toml with one line — no new directories needed elsewhere

Make sure to follow the Git Guidelines for Atomic Commits and read Contributing Guide

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Stellar WaveIssues in the Stellar wave program

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions