Added the reconcilation worker to run for deploymenment

[Lynndabel]

closes #39

Summary

I've completed the reconciliation worker concurrency safety implementation. Here's a summary:

Implementation Summary

Files Modified:

• src/reconciliation/worker.ts - Added intra-process overlap guard and integrated Postgres advisory lock
• src/reconciliation/store.ts - Added tryAcquireLeaderLock() and releaseLeaderLock() functions using Postgres advisory lock
• src/reconciliation/INVARIANTS.md - Updated I3 to document the new overlap guard guarantees
• src/reconciliation/worker.test.ts - Created new test file with 7 tests covering all concurrency scenarios

Key Changes:

1. Intra-process overlap guard: Checks processingPromise before scheduling next tick to prevent self-overlap
2. Cluster-wide leader election: Uses Postgres advisory lock (pg_try_advisory_lock) to ensure only one replica runs passes at a time
3. Metrics fix: Fixed counting loops to record exact per-status counts (including 0)
4. Lock release: Lock is released in finally block to ensure cleanup even on errors

CI Results:

• ✅ npm ci - passed
• ✅ npm run lint - passed
• ✅ npm run test:ci - passed (all reconciliation tests pass)
• ✅ npm run openapi:validate - passed

Test Coverage:

• ✅ Intra-process overlap guard test (slow pass skips next tick)
• ✅ Advisory lock tests (skip when held, run when acquired, release on error)
• ✅ Leader takeover test (another instance acquires after release)
• ✅ Drift cap test (concurrent passes cannot absorb past cap)
• ✅ Mismatch escalation test (concurrent passes prevented by lock)

The reconciliation worker is now safe to run in horizontally scaled deployments. The advisory lock ensures a single leader runs passes cluster-wide, and the intra-process guard prevents self-overlap.

Linked issue (recommended)

Example: Closes #123

closes #38

Changes

This section is required for CI PR validation.

Contract Upgrade Details (if applicable)

This section is required for CI PR validation if this is a contract upgrade.

Network

• Testnet
• Mainnet

New Contract

• Contract ID: C...
• WASM Hash: sha256:...
• Deployer Public Key: G...
• Deploy Transaction: [link to transaction explorer]

Upgrade Governance

• Admin/upgrade authority is a multisig requiring maintainer sign-off
• Maintainer has reviewed and approved the upgrade
• Upgrade transaction is ready for maintainer signature (provide transaction XDR if applicable)

Verification Steps

• New contract deployed successfully
• All existing tests pass against the new contract
• Manual testing checklist completed (describe what you tested)
• No breaking changes for existing integrations (or list them)

How to test

This section is required for CI PR validation.

• All automated tests pass
• Integration tests pass (if applicable)
• Manual testing completed (describe what you tested)

Security Considerations

This section is required for CI PR validation.

• No secrets or sensitive data are logged
• No changes to authentication/authorization logic without review
• No changes to admin/upgrade logic without review

Screenshots (if UI)

Include before/after screenshots for any UI changes. For new features, show different states (loading, error, success). For responsive changes, include mobile/tablet/desktop views.

Checklist

This section is required for CI PR validation.

• I linked an issue (or explained why one is not needed)
• I tested locally
• I did not commit secrets
• I updated docs if needed
• Code follows the project's style guidelines
• CI checks pass
• If UI changes: I included before/after screenshots
• If images added/changed: I verified they are optimized and accessible