Pagination on dynamic (manual) relationship fails when queried

[Dam-99]

Code of Conduct

• I agree to follow this project's Code of Conduct

AI Policy

• I agree to follow this project's AI Policy, or I agree that AI was not used while creating this issue.

Versions

• Elixir 1.18.4, OTP 28
• Ash 3.7.6
• AshPostgres 2.6.65
• AshGraphQL 1.8.5

I assume it's also present in some earlier versions of Ash, but I didn't investigate

Operating system

Ubuntu 24.04

Current Behavior

This issue arises when trying to paginate a manual relationship on a resource (relay pagination in my case)

defmodule Source do
 use Ash.Resource # ...
 graphql do
   type :source
   paginate_relationship_with: dest: :relay # <---- problematic line
 end
 # ... 
 relationships do
   has_many :dest, Destination do
     public? true
     description "The `source`s belonging to the destination."
     writable? false
     manual ManualRelationship
   end
 end
 # ...
end

More specifically, the reason I'm using a manual relationship in this case is because I dynamically compute the related records on runtime. For example, it takes one or more attribute from the records and creates an Ash expression starting from them to create the appropriate filter:

defmodule ManualRelationship do
  use Ash.Resource.ManualRelationship

  @impl ManualRelationship
  def load(sources, _opts, %{query: query}) do
    source_id_to_dests =
      Map.new(sources, fn source ->
        # Example inspired from the actual code 
        {:ok, ast_root} = Lang.parse(source.filter_attr)

        filter = Lang.to_ash_expr(ast_root) # filter obtained from the attribute at runtime

        destinations = query |> Ash.Query.filter(^filter) |> Ash.read!()

        {source.id, destinations}
      end)

    {:ok, source_id_to_dests}
  end
end

If the best solution would be to remove the relationship and substitute it with something else, I'm very open to change. I've already tried some options a calculation returning an Ash expression, or changing the type of the filter attribute to Ash expression, but both proved problematic for various reasons.

The result is that the graphql query fails. Here's a the error obtained from the reproduction repo linked below:

[debug] QUERY OK source="elements" db=0.2ms idle=1667.0ms
SELECT e0."id", e0."tags" FROM "elements" AS e0 ORDER BY e0."id" LIMIT $1 [251]
↳ anonymous fn/3 in AshPostgres.DataLayer.run_query/2, at: lib/data_layer.ex:835
[error] ba0013a6-872b-4dd8-b4e2-86acb57f73ac: Exception raised while resolving query.

** (Ash.Error.Unknown)
Bread Crumbs:
  > Exception raised in: DynRelRepro.Domain.Group.read
  > Exception raised in: DynRelRepro.Domain.Element.read

Unknown Error

* ** (UndefinedFunctionError) function DynRelRepro.ManualRelationships.ElementToGroup.ash_postgres_subquery/4 is undefined or private
  (dyn_rel_repro 0.1.0) DynRelRepro.ManualRelationships.ElementToGroup.ash_postgres_subquery([], 0, 0, #Ecto.Query<from g0 in DynRelRepro.Domain.Group, as: 500, windows: [order: [order_by: [asc: as(500).id]]], order_by: [asc: as(500).id], limit: ^251, select: merge(struct(g0, [:id, :included_tags]), %{__order__: over(row_number(), :order)})>)
  (ash_postgres 2.6.25) lib/data_layer.ex:1114: AshPostgres.DataLayer.lateral_join_query/3
  (ash_postgres 2.6.25) lib/data_layer.ex:1046: AshPostgres.DataLayer.run_query_with_lateral_join/4
  (ash 3.12.0) lib/ash/actions/read/read.ex:4089: Ash.Actions.Read.run_query/4
  (ash 3.12.0) lib/ash/actions/read/read.ex:806: anonymous fn/9 in Ash.Actions.Read.do_read/5
  (ash 3.12.0) lib/ash/actions/read/read.ex:1591: Ash.Actions.Read.maybe_in_transaction/3
  (ash 3.12.0) lib/ash/actions/read/read.ex:436: Ash.Actions.Read.do_run/3
  (ash 3.12.0) lib/ash/actions/read/read.ex:90: anonymous fn/3 in Ash.Actions.Read.run/3
  (ash 3.12.0) lib/ash/actions/read/read.ex:89: Ash.Actions.Read.run/3
  (ash 3.12.0) lib/ash.ex:2786: Ash.read/2
  (ash 3.12.0) lib/ash.ex:2721: Ash.read!/2
  (dyn_rel_repro 0.1.0) lib/dyn_rel_repro/manual_relationships/element_to_group.ex:20: DynRelRepro.ManualRelationships.ElementToGroup.load/3
  (ash 3.12.0) lib/ash/actions/read/relationships.ex:489: anonymous fn/7 in Ash.Actions.Read.Relationships.do_fetch_related_records/5
  (ash 3.12.0) lib/ash/actions/read/relationships.ex:85: Ash.Actions.Read.Relationships.fetch_related_records/5
  (ash 3.12.0) lib/ash/actions/read/relationships.ex:30: Ash.Actions.Read.Relationships.load/4
  (ash 3.12.0) lib/ash/actions/read/read.ex:473: Ash.Actions.Read.do_run/3
  (ash 3.12.0) lib/ash

    (dyn_rel_repro 0.1.0) DynRelRepro.ManualRelationships.ElementToGroup.ash_postgres_subquery([], 0, 0, #Ecto.Query<from g0 in DynRelRepro.Domain.Group, as: 500, windows: [order: [order_by: [asc: as(500).id]]], order_by: [asc: as(500).id], limit: ^251, select: merge(struct(g0, [:id, :included_tags]), %{__order__: over(row_number(), :order)})>)
    (ash_postgres 2.6.25) lib/data_layer.ex:1114: AshPostgres.DataLayer.lateral_join_query/3
    (ash_postgres 2.6.25) lib/data_layer.ex:1046: AshPostgres.DataLayer.run_query_with_lateral_join/4
    (ash 3.12.0) lib/ash/actions/read/read.ex:4089: Ash.Actions.Read.run_query/4
    (ash 3.12.0) lib/ash/actions/read/read.ex:806: anonymous fn/9 in Ash.Actions.Read.do_read/5
    (ash 3.12.0) lib/ash/actions/read/read.ex:1591: Ash.Actions.Read.maybe_in_transaction/3
    (ash 3.12.0) lib/ash/actions/read/read.ex:436: Ash.Actions.Read.do_run/3
    (ash 3.12.0) lib/ash/actions/read/read.ex:90: anonymous fn/3 in Ash.Actions.Read.run/3
    (ash 3.12.0) lib/ash/actions/read/read.ex:89: Ash.Actions.Read.run/3
    (ash 3.12.0) lib/ash.ex:2786: Ash.read/2
    (ash 3.12.0) lib/ash.ex:2721: Ash.read!/2
    (dyn_rel_repro 0.1.0) lib/dyn_rel_repro/manual_relationships/element_to_group.ex:20: DynRelRepro.ManualRelationships.ElementToGroup.load/3
    (ash 3.12.0) lib/ash/actions/read/relationships.ex:489: anonymous fn/7 in Ash.Actions.Read.Relationships.do_fetch_related_records/5
    (ash 3.12.0) lib/ash/actions/read/relationships.ex:85: Ash.Actions.Read.Relationships.fetch_related_records/5
    (ash 3.12.0) lib/ash/actions/read/relationships.ex:30: Ash.Actions.Read.Relationships.load/4
    (ash 3.12.0) lib/ash/actions/read/read.ex:473: Ash.Actions.Read.do_run/3
    (ash 3.12.0) lib/ash/actions/read/read.ex:90: anonymous fn/3 in Ash.Actions.Read.run/3
    (ash 3.12.0) lib/ash/actions/read/read.ex:89: Ash.Actions.Read.run/3
    (ash 3.12.0) lib/ash.ex:2786: Ash.read/2
    (ash_graphql 1.8.5) lib/graphql/resolver.ex:516: AshGraphql.Graphql.Resolver.resolve/2

Reproduction

Minimal reproduction repo.
You can see the steps via the commit history.

Expected Behavior

Ideally the relationship is paginated automatically, but if not possible an easy method to do so is added and documented (via errors and docs).
Alternatively, a different method for a dynamic relationship for which pagination already works

[zachdaniel]

Ah, interesting. I'm not yet sure how we should approach fixing the fact that it is making its way to AshPostgres, but what is happening is that in order to do the pagination its thinking that it needs to do a lateral join with an aggregate which requires implementing the following for your relationship:

https://hexdocs.pm/ash_postgres/2.6.27/manual-relationships.html

[Dam-99]

For my case, i think it is not possible to implement any of those callbacks, since none of the arguments seem to include the resource itself, as far as I can tell.
The "definition" of how the records should be related is an attribute of the source itself, and not having access to it means not being able to create a query.

[zachdaniel]

This is a bit hacky, but you can get the resource from query.__ash_bindings__.resource and/or destination_query.__ash_bindings__.resource

[Dam-99]

I will try to use those for the time being then