Advanced Search - Prevent duplicates from prev/next cache affecting pager

[colemanw]

Overview

• Fixes dev/core#6638
• Replaces remove dupes from prevnext cache #36214

Technical Details

The query uses a lot of LEFT joins which cause duplicate contact ids to be inserted into the prevNext cache. This ensures unique cids are fetched from the cache so that the duplicates don't mess up the limit and offset.

[civibot]

🤖 Thank you for contributing to CiviCRM! ❤️ We will need to test and review this PR. 👷

Introduction for new contributors...

• If this is your first PR, an admin will greenlight automated testing with the command ok to test or add to whitelist.
• A series of tests will automatically run. You can see the results at the bottom of this page (if there are any problems, it will include a link to see what went wrong).
• A demo site will be built where anyone can try out a version of CiviCRM that includes your changes.
• If this process needs to be repeated, an admin will issue the command test this please to rerun tests and build a new demo site.
• Before this PR can be merged, it needs to be reviewed. Please keep in mind that reviewers are volunteers, and their response time can vary from a few hours to a few weeks depending on their availability and their knowledge of this particular part of CiviCRM.
• A great way to speed up this process is to "trade reviews" with someone - find an open PR that you feel able to review, and leave a comment like "I'm reviewing this now, could you please review mine?" (include a link to yours). You don't have to wait for a response to get started (and you don't have to stop at one!) the more you review, the faster this process goes for everyone 😄
• To ensure that you are credited properly in the final release notes, please add yourself to contributor-key.yml
• For more information about contributing, see CONTRIBUTING.md.

PR commands & links...

• /rebase <branch-name> will rebase your branch and change the base of the PR.
• /squash will combine all commits (keeping only the first commit messsage).
• /port <branch-name> will create a copy of this PR against a different branch.
• /lintroll will automatically fix linting errors, amending commits as needed.
• retest this please will rerun the tests and rebuild the demo site.
• 📖 Review standards
• 🗒️ Review template (brief or verbose)

➡️ Online demo of this PR 🔗

[jmcclelland]

Thanks @colemanw - I just tested this revised version and it works for me.

[mlutfy]

Could the groupby/min have a performance impact on very large installs, ex WMF?

[colemanw]

@mlutfy the prev_next_cache table is just a list of ids returned by the search, and this is a simple query to fetch those ids. No joins onto enormous tables, etc.

[colemanw]

@eileenmcnaughton volunteered to test this one :)

[colemanw]

Well I haven't heard any objections from @eileenmcnaughton so I guess this is merge-ready :)

[jmcclelland]

FWIW, I've been running it in production on 150 installs for the last week without any reports of problems.

[mlutfy]

Thank you @jmcclelland @colemanw !