Skip to content

Flaky test: lookupSingleWebFinger depends on external server (hollo.social) #480

@dahlia

Description

@dahlia

The test for lookupSingleWebFinger in packages/cli/src/webfinger/mod.test.ts is flaky because it depends on the external remote server hollo.social. When this server is temporarily unavailable, the test fails with:

Error [NotFoundError]: Resource not found: @[email protected]
    at lookupSingleWebFinger (/home/runner/work/fedify/fedify/packages/cli/src/webfinger/action.ts:28:5)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5) {
  resource: '@[email protected]'
}

Current implementation

The test (lines 73-79) makes real HTTP requests to external servers:

test("Test lookupSingleWebFinger", async () => {
  const aliases = (await Array.fromAsync(
    RESOURCES,
    (resource) => lookupSingleWebFinger({ resource }),
  )).map((w) => w?.aliases?.[0]);
  assert.deepEqual(aliases, ALIASES);
});

Where RESOURCES includes (lines 9-12):

Problem

This approach has several issues:

  1. Unreliability: External servers may be temporarily unavailable, causing intermittent test failures
  2. Network dependency: Tests require internet connectivity and DNS resolution
  3. Slowness: Real HTTP requests significantly slow down the test suite
  4. Brittleness: Changes to the external server's configuration or data can break tests unexpectedly

Proposed solution

The test should be refactored to use one of the following approaches:

  1. Mock server: Spin up a local HTTP server that responds to WebFinger queries with known test data
  2. Mock HTTP client: Mock the underlying HTTP fetch calls to return pre-determined responses
  3. Use @fedify/testing utilities: Leverage existing testing utilities from the @fedify/testing package if available

The ideal solution would be to create a minimal local ActivityPub/WebFinger server for testing purposes, ensuring tests are:

  • Fast (no real network I/O)
  • Reliable (no external dependencies)
  • Deterministic (known test data)

Additional context

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions