Skip to content

Fail startup when database migrations fail instead of logging and continuing #219

Description

@N-thnI

Description

Refuse to serve traffic against a schema that may be missing required tables.

Problem Statement

startServer swallows migration failure outright (index.js:131-136):

catch (migrationErr) {
  logger.error({ error: migrationErr.message }, '[startup] Database migrations failed — continuing');
}

The server then binds the port and accepts /github-webhook requests against a schema that may be missing the retry_state table entirely. Every subsequent initRetryState / recordRetry throws — and those throws are themselves swallowed as "non-fatal" in src/workers/event-worker.js:53-55 and :82-84.

Net effect: merged PRs are accepted with 202 { ok: true, queued: true }, then fail to register with no durable retry record — a silent, payment-affecting data-loss path.

Proposed Changes

  • Rethrow (or process.exit(1)) on migration failure in startServer, so a bad deploy fails loudly and the orchestrator doesn't route traffic to it
  • Apply the same to startEventWorker — a worker with no retry_state table cannot do its job
  • If a partial-degradation mode is genuinely wanted, gate it behind an explicit env flag and have /health report DEGRADED while active

Technical Implementation Scaffolding

  • Target Repository: vero-relayer-service
  • Target Path: index.js, src/workers/event-worker.js
  • Branch Naming: fix/issue--fail-startup-on-migration-failure
  • Authority Context: Reliability — silent loss of merged-PR registrations

Acceptance Criteria

  • With an unreachable database, npm start exits non-zero and never binds the port
  • /health returns 503 whenever migration state is unknown or incomplete

Definition of Done

  • Reviewed by lead maintainer
  • Pull request merged via verified status check

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

GrantFox OSSIssue tracked in GrantFox OSSMaybe RewardedIssue may be eligible for a GrantFox rewardThird CampaignCampaign: Third Campaign

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions