[CursorPaginator] Various tweaks and improvements by seb-jean · Pull Request #12404 · doctrine/orm

seb-jean · 2026-03-19T11:53:00Z

Problem

When a DQL query joins a toMany relation, a single entity can span multiple SQL rows. The previous implementation applied setMaxResults() at the SQL level, causing pages to silently return fewer items than expected and incorrect hasMore detection. Raised by @stof in Add cursor-based pagination #12364 (review).
Countable inconsistency — The previous count() returned the number of items on the current page, inconsistent with Paginator which returns the total. Raised by @stof in Add cursor-based pagination #12364 (review).
paginate() only accepted an encoded string — Passing a Cursor instance directly required an unnecessary round-trip through encodeToString(). Raised in
Allow Cursor instance in CursorPaginator::paginate() instead of only encoded string #12405.

Changes

`queryProducesDuplicates` defaults to `true`

CursorPaginator — queryProducesDuplicates now defaults to true:

First query — fetches paginated root entity IDs via LimitSubqueryOutputWalker / LimitSubqueryWalker
Second query — fetches full results filtered by those IDs via WhereInWalker

<?php

// Before: silently returns wrong results with toMany joins
new CursorPaginator($qb);

// After: two-query strategy (distinct IDs subquery + WHERE IN fetch)
new CursorPaginator($qb); // queryProducesDuplicates: true by default

// Simple queries without toMany joins
new CursorPaginator($qb, queryProducesDuplicates: false);

CursorWalker — Throws LogicException when queryProducesDuplicates: false is used on a query that joins a toMany relation:

// Throws: Cannot use CursorPaginator without queryProducesDuplicates on a query
//         that joins a toMany relation.
new CursorPaginator($qb->leftJoin('a.missions', 'm'), queryProducesDuplicates: false);

getTotalCount() + count()

count() — returns the number of items on the current page (no DB query)
getTotalCount() — runs a COUNT query (identical implementation to Paginator::count())

Countable is not implemented to avoid triggering an implicit COUNT query.

`paginate()` accepts `Cursor|string|null`

paginate() now accepts a Cursor instance directly in addition to an encoded string, removing the need for an unnecessary encode/decode round-trip when the cursor is already available as an object:

// Before: required encoding even when the Cursor instance was already available
$paginator->paginate($cursor->encodeToString(), 15);

// After: pass the Cursor instance directly
$paginator->paginate($cursor, 15);

// Encoded strings and null still work as before
$paginator->paginate($encodedString, 15);
$paginator->paginate(null, 15);

stof · 2026-03-19T11:57:22Z

I suggest using a better name than fetchJoinCollection to describe what it actually means (see the discussion in #11595 (comment))

seb-jean · 2026-03-19T13:33:12Z

I suggest using a better name than fetchJoinCollection to describe what it actually means (see the discussion in #11595 (comment))

I renamed fetchJoinCollection to queryProducesDuplicates.

src/Tools/CursorPagination/RowNumberOverFunction.php

src/Tools/CursorPagination/LimitSubqueryWalker.php

src/Tools/CursorPagination/WhereInWalker.php

src/Tools/CursorPagination/Exception/RowNumberOverFunctionNotEnabled.php

docs/en/tutorials/pagination.rst

src/Tools/CursorPagination/CursorPaginator.php

src/Tools/CursorPagination/LimitSubqueryOutputWalker.php

src/Tools/CursorPagination/LimitSubqueryWalker.php

src/Tools/CursorPagination/LimitSubqueryOutputWalker.php

src/Tools/CursorPagination/RootTypeWalker.php

stof · 2026-04-01T18:58:53Z

src/Tools/CursorPagination/LimitSubqueryWalker.php

+{
+    protected function isDistinctEnabled(AbstractQuery $query): bool
+    {
+        return ($query->getHints()[CursorPaginator::HINT_ENABLE_DISTINCT] ?? true) === true;


couldn't we keep the paginator.distinct.enable hint even when using the cursor paginator ?

Done. CursorPagination\LimitSubqueryWalker has been removed entirely. CursorPaginator now uses Pagination\LimitSubqueryWalker directly, which already checks Paginator::HINT_ENABLE_DISTINCT. The
CursorPaginator::HINT_ENABLE_DISTINCT constant has been removed as well.

stof · 2026-04-01T19:01:35Z

src/Tools/CursorPagination/CursorPaginator.php

    /**
     * Appends a custom tree walker to the tree walkers hint.
+     *
+     * @param class-string $walkerClass


I suggest using class-string<TreeWalker> here for extra safety.

stof · 2026-04-01T19:05:30Z

src/Tools/CursorPagination/CursorWalker.php

        }

+        if ($query->getHint(self::HINT_QUERY_PRODUCES_DUPLICATES) === false) {
+            foreach ($this->getQueryComponents() as $component) {


Technically, you could have a filtering clause that ensures that a toMany join does not actually produce duplicates in the result. Should we forbid such case from using the more optimized paginated by doing an explicit choice ?

Good point. We removed the automatic detection entirely. Since queryProducesDuplicates: false is already an explicit choice by the developer (mirroring fetchJoinCollection in Paginator)

stof · 2026-04-01T19:08:29Z

src/Tools/Pagination/LimitSubqueryOutputWalker.php

    private readonly AbstractPlatform $platform;
    private readonly ResultSetMapping $rsm;
-    private readonly int $firstResult;
+    protected int $firstResult;


instead of turning this into a mutable protected parameter just to reset it to 0 in the cursor-related LimitSubqueryOutputWalker child class, I would suggest that the CursorPaginator sets the first result to 0 in the Query itself (just like it already sets the max results in it)

Done. CursorPagination\LimitSubqueryOutputWalker has been removed entirely. CursorPaginator now calls $subQuery->setFirstResult(0) directly alongside setMaxResults($limit + 1), and uses Pagination\LimitSubqueryOutputWalker directly. $firstResult is back to private readonly in the base class.

stof · 2026-04-01T19:10:33Z

src/Tools/CursorPagination/CursorPaginator.php

+        return $this->count;
+    }
+
+    private function getCountQuery(): Query


As the counting implementation is the same in both paginators, I suggest extracting an internal trait used in both classes to avoid duplicating maintenance.

Done. The shared logic (getCountQuery and unbindUnusedQueryParams) has been extracted into an internal CountQuery trait in src/Tools/Pagination/CountQuery.php, used by both Paginator and CursorPaginator.
The $count property has also been moved into the trait.

stof · 2026-04-01T19:11:40Z

src/Tools/CursorPagination/CursorPaginator.php

        return $this->items->count();
    }

+    public function getTotalCount(): int


getTotalCount does not seem to be covered by tests.

Fixed, two tests have been added.

stof · 2026-04-02T09:34:59Z

docs/en/tutorials/pagination.rst

+    .. note::
+
+        Passing ``false`` on a query that joins a to-many relation may throw a
+        ``LogicException`` at runtime when Doctrine can detect the mistake.


this LogicException has now been removed.

stof · 2026-04-02T09:38:01Z

src/Tools/CursorPagination/CursorPaginator.php

@@ -4,42 +4,60 @@

 namespace Doctrine\ORM\Tools\CursorPagination;


@greg0ire I suggest moving this CursorPaginator to the Pagination namespace instead of having 2 namespaces related to pagination (with CursorPagination reusing big parts of Pagination). What do you think about that ?
This is a decision we need to take before the 3.7.0 release, as moving the class after that would be a BC break.

stof · 2026-04-02T09:38:39Z

src/Tools/Pagination/CountQuery.php

+use function array_key_exists;
+
+/**
+ * Provides the count query implementation shared by {@see Paginator} and {@see CursorPaginator}.


This @see referencing a short class name would need a use statement.

stof · 2026-04-02T09:39:54Z

src/Tools/Pagination/CountQuery.php

+            $countQuery->setHint(Query::HINT_CUSTOM_OUTPUT_WALKER, CountOutputWalker::class);
+            $countQuery->setResultSetMapping($rsm);
+        } else {
+            $this->appendTreeWalker($countQuery, CountWalker::class);


appendTreeWalker should be moved to the trait as it is used here.

stof · 2026-04-02T09:40:35Z

src/Tools/Pagination/CountQuery.php

+            $countQuery->setHint(CountWalker::HINT_DISTINCT, true);
+        }
+
+        if ($this->useOutputWalker($countQuery)) {


useOutputWalker should probably be moved to the trait (or the trait should define an abstract private method for it)

stof · 2026-04-02T09:44:58Z

src/Tools/Pagination/CountQuery.php

+ *
+ * @internal
+ */
+trait CountQuery


maybe this should be a PaginatorTrait instead, copying more private methods (cloneQuery, appendTreeWalker and convertWhereInIdentifiersToDatabaseValues for instance, and maybe everything about the output walker config)

stof · 2026-04-02T09:48:14Z

src/Tools/CursorPagination/CursorPaginator.php

+
+            $this->items = new ArrayCollection($whereInQuery->getResult());
+        } else {
+            $this->items   = new ArrayCollection($subQuery->getResult());


I would use a local variable storing the array here instead of creating an ArrayCollection that will be dropped just after. the usage of an ArrayCollection is overkill here.

stof · 2026-04-02T09:48:59Z

src/Tools/CursorPagination/CursorPaginator.php

+            $this->orderByItems = $subQuery->getHint(CursorWalker::HINT_CURSOR_ORDER_BY_ITEMS) ?: [];
+
+            if ($this->cursor !== null && $this->cursor->isPrevious()) {
+                $this->items = new ArrayCollection(array_reverse($this->items->toArray(), true));


if the result is a list, we should not preserve keys when reversing.

seb-jean force-pushed the cursor-pagination-fetch-join branch from d13637d to 61add19 Compare March 19, 2026 12:42

seb-jean changed the title ~~[CursorPaginator] Support toMany joins via fetchJoinCollection~~ [CursorPaginator] Support toMany joins via queryProducesDuplicates Mar 19, 2026

seb-jean force-pushed the cursor-pagination-fetch-join branch 2 times, most recently from a44d5c9 to ac7634f Compare March 19, 2026 12:54

stof reviewed Mar 19, 2026

View reviewed changes

src/Tools/CursorPagination/RowNumberOverFunction.php Outdated Show resolved Hide resolved

src/Tools/CursorPagination/LimitSubqueryWalker.php Outdated Show resolved Hide resolved

stof reviewed Mar 19, 2026

View reviewed changes

src/Tools/CursorPagination/WhereInWalker.php Outdated Show resolved Hide resolved

seb-jean force-pushed the cursor-pagination-fetch-join branch from ac7634f to 1f128e4 Compare March 19, 2026 18:49

stof reviewed Mar 20, 2026

View reviewed changes

src/Tools/CursorPagination/Exception/RowNumberOverFunctionNotEnabled.php Outdated Show resolved Hide resolved

seb-jean force-pushed the cursor-pagination-fetch-join branch 3 times, most recently from 7470a01 to dbfe2e4 Compare March 20, 2026 15:12

seb-jean changed the title ~~[CursorPaginator] Support toMany joins via queryProducesDuplicates~~ [CursorPaginator] Various tweaks and improvements Mar 20, 2026

seb-jean mentioned this pull request Mar 20, 2026

Add cursor-based pagination #12364

Merged

greg0ire reviewed Mar 20, 2026

View reviewed changes

docs/en/tutorials/pagination.rst Outdated Show resolved Hide resolved

stof reviewed Mar 20, 2026

View reviewed changes

seb-jean force-pushed the cursor-pagination-fetch-join branch 3 times, most recently from 3ca0823 to 2cfb7c2 Compare March 22, 2026 15:51

seb-jean mentioned this pull request Mar 22, 2026

Allow Cursor instance in CursorPaginator::paginate() instead of only encoded string #12405

Open

seb-jean force-pushed the cursor-pagination-fetch-join branch from 2cfb7c2 to f7ca25b Compare March 22, 2026 20:12

greg0ire requested a review from stof March 29, 2026 11:18

greg0ire reviewed Mar 29, 2026

View reviewed changes

src/Tools/CursorPagination/LimitSubqueryWalker.php Outdated Show resolved Hide resolved

src/Tools/CursorPagination/LimitSubqueryOutputWalker.php Outdated Show resolved Hide resolved

src/Tools/CursorPagination/RootTypeWalker.php Outdated Show resolved Hide resolved

seb-jean force-pushed the cursor-pagination-fetch-join branch 3 times, most recently from 703c842 to 3d7894b Compare March 29, 2026 15:40

greg0ire previously approved these changes Mar 29, 2026

View reviewed changes

greg0ire requested a review from derrabus March 29, 2026 16:28

stof reviewed Apr 1, 2026

View reviewed changes

feat(CursorPaginator): support toMany joins

796cec7

seb-jean dismissed greg0ire’s stale review via 796cec7 April 1, 2026 20:36

seb-jean force-pushed the cursor-pagination-fetch-join branch from 3d7894b to 796cec7 Compare April 1, 2026 20:36

stof reviewed Apr 2, 2026

View reviewed changes

		@@ -4,42 +4,60 @@

		namespace Doctrine\ORM\Tools\CursorPagination;

Conversation

seb-jean commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Changes

queryProducesDuplicates defaults to true

getTotalCount() + count()

paginate() accepts Cursor|string|null

Uh oh!

stof commented Mar 19, 2026

Uh oh!

seb-jean commented Mar 19, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

seb-jean commented Mar 19, 2026 •

edited

Loading

`queryProducesDuplicates` defaults to `true`

`paginate()` accepts `Cursor|string|null`