Serve hashlink media from FEP-ef61 gateways

[dahlia]

Background

FEP-ef61 lets portable objects reference external resources, such as media attachments, using hashlinks and digestMultibase.

For example, a portable object may contain an attachment like this:

{
  "type": "Image",
  "url": "hl:zQmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n",
  "mediaType": "image/png",
  "digestMultibase": "zQmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n"
}

FEP-ef61 says that resources attached to portable objects using hashlinks can be stored by gateways. To retrieve such a resource from a gateway, a client makes a GET request to the gateway endpoint and appends the hashlink URI:

GET https://server.example/.well-known/apgateway/hl:zQmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n

Fedify needs a server-side endpoint for serving these hashlink-addressed media resources from FEP-ef61 gateways. This is separate from FEP-ae97 media upload/delete APIs.

Fedify already has multibase support in @fedify/vocab-runtime, and generated vocabulary code maps https://w3id.org/security#multibase values to Uint8Array. The serving API should account for that existing representation while still exposing the original multibase string if it is useful as a storage key.

Proposed work

Add support in @fedify/fedify for serving hashlink-addressed media through the fixed FEP-ef61 gateway endpoint.

The work should include:

• routing GET /.well-known/apgateway/hl:<resource-hash>;
• parsing and validating the hl: URI using the digest/hashlink helpers before invoking application code;
• dispatching the request to application code so the application can return an HTTP response for the requested digest;
• allowing the application to stream the response body and set HTTP headers such as Content-Type, Cache-Control, Content-Disposition, and range-related headers where needed;
• converting a null dispatcher result to 404 Not Found;
• returning 400 Bad Request for malformed or unsupported hashlinks before the dispatcher is called;
• documenting that the dispatcher must return the representation identified by the requested digest, typically by using digestMultibase as the storage key or by verifying the digest at storage time;
• avoiding mandatory response-body buffering for digest verification, because that would break streaming;
• making sure this route does not conflict with /.well-known/apgateway/{did}/{path...} portable object serving;
• preserving the existing non-gateway routing behavior.

The public API name should be decided during implementation. The name should make clear that this is the hashlink media route under the FEP-ef61 /.well-known/apgateway endpoint, not the FEP-ae97 /.well-known/apgateway-media upload/delete API.

Possible names include:

• setApGatewayMediaDispatcher();
• setHashlinkMediaDispatcher();
• setPortableMediaDispatcher();
• setGatewayHashlinkDispatcher().

The examples below use setApGatewayMediaDispatcher() as a placeholder name only:

interface HashlinkMediaRequest {
  readonly hashlink: string;
  readonly digestMultibase: string;
  readonly algorithm: "sha2-256";
  readonly digest: Uint8Array;
  readonly multihash: Uint8Array;
}

federation.setApGatewayMediaDispatcher(
  async (ctx, media): Promise<Response | null> => {
    const file = await findMediaByDigest(media.digestMultibase);
    if (file == null) return null;

    return new Response(file.stream(), {
      headers: {
        "Content-Type": file.mediaType,
        "Cache-Control": "public, immutable, max-age=31536000",
      },
    });
  },
);

The dispatcher should receive parsed digest/hashlink information rather than a raw unvalidated path string, and should return Response | null | Promise<Response | null> so applications can control streaming and HTTP headers. The exact request type should be decided during implementation, but it should probably include the original hashlink, the digestMultibase string, the hash algorithm, the raw digest bytes, and the decoded multihash bytes.

Scope

This issue is only about read-only serving of hashlink-addressed media from the FEP-ef61 gateway path.

It does not include:

• FEP-ae97 media upload;
• FEP-ae97 media deletion;
• media storage implementation;
• access-controlled media;
• parent-object-based media authorization;
• full implementation of the expired hashlink Internet-Draft;
• gateway object serving for DID-based portable objects.

Access-controlled media should be handled separately. FEP-ef61 notes that plain hashlink media has an access-control limitation: anyone who knows the hash can retrieve the file. The first implementation should focus on public/content-addressed media.

Dependencies

This should depend on the lower-level media digest work under #288, especially:

• FEP-ef61 vocabulary terms for digestMultibase;
• SHA-256 digestMultibase and simple hashlink helpers.

Tests

Add regression tests for hashlink media serving.

The tests should cover:

• routing GET /.well-known/apgateway/hl:z... to the media dispatcher;
• passing parsed digest/hashlink information to the dispatcher;
• returning the media response with application-provided headers;
• streaming a response body without mandatory buffering by Fedify;
• returning 404 Not Found when the dispatcher returns null;
• returning 400 Bad Request for malformed hashlinks before the dispatcher is called;
• preserving a dispatcher-provided Response status and headers where appropriate;
• ensuring the hashlink media route does not get routed as a DID-based portable object request;
• preserving existing non-gateway routes.

This should be added as a sub-issue of #288.