Skip to content

[2.x] No serialize-time batch hook for resources loaded as included from other endpoints #4636

Description

@imorland

The new 2.x JSON:API surface gives extensions Endpoint::beforeSerialization to batch-prime caches before resources are serialized, but the hook is bound to a specific (resource, endpoint) pair. There's no equivalent hook for the case where a resource is loaded as an included resource from an endpoint on a different resource — and that's the most common path in production traffic for many resource types.

Concrete shape

UserResource is the clearest example. Most user serialization in a typical forum doesn't come from /api/users — it comes through:

  • /api/discussions → discussion firstUser, lastPostedUser, recipients, plus post authors / mention targets / likers from included posts
  • /api/posts → post authors, mention targets, likers
  • search results, notification lists, mentions, etc.

If an extension adds a field to UserResource and wants to avoid N+1 queries, the natural hook is beforeSerialization on UserResource::Index:

(new Extend\ApiResource(UserResource::class))
    ->endpoint(Endpoint\Index::class, function (Endpoint\Index $endpoint) {
        return $endpoint->beforeSerialization(function (Context $context, array $results) {
            // batch-load the data needed by the fields, prime caches
        });
    })

That fires for /api/users but never for /api/discussions, even though /api/discussions may serialize 50+ User resources via included. Each of those User resources runs the field getters with empty caches, falling through to per-user DB queries.

Pattern observed across multiple extensions

I hit this in imorland/follow-users#56 and noticed the same shape on every extension I'm running that has User-resource fields with batch-priming logic:

  • flarum/likes — likers per post, default-included on post serialization
  • fof/badgesselect * from fof_badge_user where user_id = ? per user
  • fof/moderator-warningsselect count(*) from warnings where user_id = ? per user
  • fof/termsselect fof_terms_policies join fof_terms_policy_user where user_id = ? per user
  • ianm/follow-users — three query patterns per user (follow state, follower count, following count)

Even core's own select permission from group_permission where group_id in (...) repeats per-user during serialization (~80 times per page in my profiling).

Each of these extensions can be fixed individually by writing a beforeSerialization hook on every endpoint that might include their target resource type — but that means knowing about DiscussionResource::Index/Show, PostResource::Index/Show, search endpoints, future endpoints, etc. It's not a stable API surface for an extension to reason about.

What seems to be missing

A hook that fires once per request the first time a resource of type X is added to the serializer's map, with access to the full collection of resources-to-be-serialized of that type, regardless of which endpoint started the request. Something like:

(new Extend\ApiResource(UserResource::class))
    ->beforeSerializing(function (Context $context, Collection $users) {
        // fires once per request, with all users that will be serialized
    });

That would let the extension's batch-load run regardless of entry point.

I'm not proposing a specific implementation here — the right shape probably depends on internal details of the serializer's map-building / deferred-resolve loop and how it interacts with tobyz/json-api-server's own lifecycle. Filing this primarily so the gap is documented and the systemic nature is visible alongside the per-extension symptoms.

Reproduction / impact

In our environment (Flarum 2.0.0-rc.1, ~70 enabled extensions, 25k discussions / 100k users on db.t3.small):

  • /api/discussions issued ~772 queries on a guest request, ~440 of them following one of the per-user N+1 patterns above
  • After fixing the avatar S3 round-trip (flarum/framework#4635), the remaining cost is dominated by these extensions falling through to per-user DB queries during included-User serialization
  • Extension-level beforeSerialization hooks bound to UserResource::Index are present and working, but never fire on this path

Environment

  • Flarum core 2.0.0-rc.1
  • ~70 enabled 2.x-compatible extensions
  • 25k discussions / 168k posts / 100k users
  • ~85 distinct users serialized per /api/discussions page render

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions