Add `generalizedCostMaxLimit` field to the `PageCursor` to enable using `RemoveTransitIfStreetOnlyIsBetter` filter with paging #6474

VillePihlava · 2025-02-20T12:44:50Z

Summary

This PR adds the generalizedCostMaxLimit field to the PageCursor. This enables using the RemoveTransitIfStreetOnlyIsBetter filter with paging. Before this PR, this was not possible because paging only used the walk mode to create a direct itinerary which made the filter work incorrectly.

Issue

N/A

Unit tests

Changed existing unit tests for compatibility and created new ones. Also performed manual testing.

Documentation

Code comments, no other documentation

codecov · 2025-02-20T12:50:50Z

Codecov Report

Attention: Patch coverage is 89.26174% with 16 lines in your changes missing coverage. Please review.

Project coverage is 71.06%. Comparing base (9a5c2aa) to head (d830c17).
Report is 371 commits behind head on dev-2.x.

Files with missing lines	Patch %	Lines
...thm/filterchain/paging/DefaultPageCursorInput.java	82.14%	5 Missing ⚠️
...ansit/RemoveTransitIfStreetOnlyIsBetterResult.java	25.00%	3 Missing ⚠️
...m/filterchain/ItineraryListFilterChainBuilder.java	81.81%	2 Missing ⚠️
...rithm/mapping/RouteRequestToFilterChainMapper.java	33.33%	1 Missing and 1 partial ⚠️
...er/model/plan/paging/cursor/PageCursorFactory.java	94.73%	1 Missing ⚠️
...entripplanner/routing/algorithm/RoutingWorker.java	50.00%	0 Missing and 1 partial ⚠️
...ain/filters/system/NumItinerariesFilterResult.java	83.33%	1 Missing ⚠️
.../opentripplanner/service/paging/PagingService.java	91.66%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             dev-2.x    #6474      +/-   ##
=============================================
+ Coverage      70.25%   71.06%   +0.81%     
+ Complexity     18385    18321      -64     
=============================================
  Files           2088     2008      -80     
  Lines          77417    75962    -1455     
  Branches        7840     7786      -54     
=============================================
- Hits           54386    53983     -403     
+ Misses         20262    19232    -1030     
+ Partials        2769     2747      -22

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

VillePihlava · 2025-02-21T08:40:06Z

I removed the direct mode search with paging and refactored bestStreetOnlyCost to streetOnlyCost. This should now be ready-for-review.

...ation/src/main/java/org/opentripplanner/model/plan/paging/cursor/DefaultPageCursorInput.java

application/src/main/java/org/opentripplanner/model/plan/paging/cursor/PageCursor.java

application/src/main/java/org/opentripplanner/model/plan/paging/cursor/PageCursorFactory.java

...planner/routing/algorithm/filterchain/filters/transit/RemoveTransitIfStreetOnlyIsBetter.java

...main/java/org/opentripplanner/routing/algorithm/mapping/RouteRequestToFilterChainMapper.java

application/src/main/java/org/opentripplanner/routing/algorithm/RoutingWorker.java

...ner/routing/algorithm/filterchain/filters/transit/RemoveTransitIfStreetOnlyIsBetterTest.java

application/src/main/java/org/opentripplanner/routing/algorithm/RoutingWorker.java

…ror with case that should not happen.

…into paging-filtering-fix

VillePihlava · 2025-03-25T15:58:34Z

This PR depends on how the new logic in RemoveTransitIfStreetOnlyIsBetter should be structured. I did not fix all comments because of this. We had a short discussion with @optionsome and @t2gran on what to do, but a decision was not made. Once it is clear what is wanted, I'll make the changes

t2gran

I did a fast architecture review - I did not read all code-lines. If you want we can go through it in a meeting.

t2gran · 2025-03-26T11:08:26Z

...ation/src/main/java/org/opentripplanner/model/plan/paging/cursor/DefaultPageCursorInput.java

+import org.opentripplanner.routing.algorithm.filterchain.filters.system.NumItinerariesFilterResult;
+import org.opentripplanner.routing.algorithm.filterchain.filters.transit.RemoveTransitIfStreetOnlyIsBetterResult;


The PageCursorInput define a role implemented by the itinerary filter chain - we did this to invert the dependencies and to decouple the paging from the itinerary filter chain. There is no point in having the implementation of an interface in the same package (unless its just the default of many) - it indicates that there is a design mistake.

There should NOT be any dependencies from paging to the itinerary-filter-chain.

So, this should be moved inside the filter-chain, witch should be responsible for produsing it.

I moved the DefaultPageCursorInput into the org.opentripplanner.routing.algorithm.filterchain package

t2gran · 2025-03-26T11:30:14Z

application/src/main/java/org/opentripplanner/routing/algorithm/RoutingWorker.java

+      DefaultPageCursorInput.of()
+        .withNumItinerariesFilterResult(numItinerariesFilterResult)
+        .withRemoveTransitIfStreetOnlyIsBetterResult(removeTransitIfStreetOnlyIsBetterResult)
+        .build(),


I think the itinerary-filter-chain should be responsible for producing the PageCursorInput - this adds an extra layer of abstraction, but the coupling at field/information level is so high, that I do not se the point of doing it. The responsibility of the RoutingWorker is to wire things together, not mapping from one domain to another. The mapping logic should then be extracted, but it is simpler to just push it into he filter-chain.

So, we need the result of filters to produce the event to send back to the paging. There are many ways to do it:

Alt 1. We can store the result in the filters and add the filters to a result aggregator. When the filter-chain is compleate, we notify the aggregator and it produces the paging event.

Alt 2. The filters can send a event to the result aggregator and the aggregator can send an event to the paging when it has rescieved an event from both of the filters. This is complex, and if one of the filters is disabled it might fail.

I created a PageCursorInputAggregator and use a callback to set the PageCursorInput

…rithm.filterchain.

…tter.

.../java/org/opentripplanner/routing/algorithm/filterchain/ItineraryListFilterChainBuilder.java

optionsome · 2025-03-28T20:16:13Z

...java/org/opentripplanner/routing/algorithm/filterchain/paging/PageCursorInputAggregator.java

+    this.pageCursorInputSubscriber = builder.pageCursorInputSubscriber();
+  }
+
+  public void createPageCursorInput() {


I'm not sure if create is the right word here since this is not a factory method. Maybe providePageCursorInput?

I changed this to providePageCursorInput

...main/java/org/opentripplanner/routing/algorithm/mapping/RouteRequestToFilterChainMapper.java

optionsome · 2025-03-28T20:20:19Z

...ipplanner/routing/algorithm/raptoradapter/router/FilterTransitWhenDirectModeIsEmptyTest.java

This test class should be restored.

I restored it and added new tests for when the pagecursor exists

optionsome · 2025-03-28T20:25:10Z

...entripplanner/routing/algorithm/raptoradapter/router/FilterTransitWhenDirectModeIsEmpty.java

  }

-  public FilterTransitWhenDirectModeIsEmpty(StreetMode originalDirectMode) {
+  public FilterTransitWhenDirectModeIsEmpty(


You should update this class' javadoc and add some to the public methods as well.

I updated the class' javadoc and added documentation to two methods

...cation/src/test/java/org/opentripplanner/model/plan/paging/cursor/PageCursorFactoryTest.java

...est/java/org/opentripplanner/routing/algorithm/filterchain/ItineraryListFilterChainTest.java

optionsome · 2025-03-28T20:39:09Z

I resolved the comments that I think have been fixed/are not relevant anymore. I think the ones that are open either are still relevant, need changes and/or should be taken a look at by @t2gran if everything is ok now.

...entripplanner/routing/algorithm/raptoradapter/router/FilterTransitWhenDirectModeIsEmpty.java

...cation/src/test/java/org/opentripplanner/model/plan/paging/cursor/PageCursorFactoryTest.java

optionsome · 2025-04-11T20:16:28Z

...opentripplanner/routing/algorithm/filterchain/filters/system/NumItinerariesFilterResult.java

+ * The earliest-removed-departure defines the start of the search-window following the
+ * current window. To include this removed itinerary (and all other removed itineraries)
+ * in the next-page search the search windows must overlap.


Is this correct when searching to the other direction? The purpose of latestRemovedDeparture should probably be also explained.

optionsome · 2025-04-11T20:18:20Z

...opentripplanner/routing/algorithm/filterchain/filters/system/NumItinerariesFilterResult.java

+ * current window. To include this removed itinerary (and all other removed itineraries)
+ * in the next-page search the search windows must overlap.
+ * <p>
+ * In case the result has too many results: The {@code numberOfItineraries} request parameter


Suggested change

* In case the result has too many results: The {@code numberOfItineraries} request parameter

* In case the result has too many results, if the {@code numberOfItineraries} request parameter

optionsome · 2025-04-11T20:19:07Z

...opentripplanner/routing/algorithm/filterchain/filters/system/NumItinerariesFilterResult.java

+ * is less than the number of itineraries found, then we keep the last itinerary kept and
+ * returned as part of the result. The sort vector will be included in the page-cursor and


The "then we keep the last itinerary kept and returned as part of the result" part is difficult to understand.

optionsome · 2025-04-11T20:20:23Z

...planner/routing/algorithm/filterchain/filters/transit/RemoveTransitIfStreetOnlyIsBetter.java

+   * @param costLimitFunction the cost limit function to use with the filter
+   * @param generalizedCostMaxLimit this limit is not null when paging is used


If the param can be null in some case, it should be marked as @Nullable and field can be marked as such also.

optionsome · 2025-04-11T20:31:13Z

...entripplanner/routing/algorithm/raptoradapter/router/FilterTransitWhenDirectModeIsEmpty.java

+ * based on a direct mode used in the search. We achieve the filtering by setting the directMode
+ * before searching. This triggers the direct street search, and later the result is passed into
+ * the filter chain where none optimal results are removed. Finally the street itinerary is removed
+ * and the request street mode is assigned back to the original state.


Suggested change

* based on a direct mode used in the search. We achieve the filtering by setting the directMode

* before searching. This triggers the direct street search, and later the result is passed into

* the filter chain where none optimal results are removed. Finally the street itinerary is removed

* and the request street mode is assigned back to the original state.

* based on a direct search cost from the original search. We achieve the filtering by setting the

* directMode before searching. This triggers the direct street search, and later the result is

* passed into the filter chain where none optimal results are removed. Finally the street itinerary

* is removed and the request street mode is assigned back to the original state.

optionsome · 2025-04-11T21:09:37Z

...entripplanner/routing/algorithm/raptoradapter/router/FilterTransitWhenDirectModeIsEmpty.java

+  /**
+   * If the direct mode was not set and if the page cursor does not exist,
+   * the WALK mode is returned for filtering itineraries.
+   */


This class feels really weird in many ways. The name makes it sound like this is a filter but this actually isn't one. This class has too many responsibilities in my opinion and this pr makes it even harder to follow the logic. It feels a bit hacky how we need to set the direct mode back-and-forth in many places. However, we have plans in the future to perhaps refactor the "direct filter" logic a bit so maybe this isn't the right time to do refactoring here.

optionsome · 2025-04-11T21:16:18Z

application/src/main/java/org/opentripplanner/model/plan/paging/cursor/PageCursorInput.java

-   * The earliest-removed-departure defines the start of the search-window following the
-   * current window. To include this removed itinerary (and all other removed itineraries)
-   * in the next-page search the search windows must overlap.
+   * This contains the results from {@link org.opentripplanner.routing.algorithm.filterchain.filters.system.NumItinerariesFilter}.


We still haven't decided what to do about this?

VillePihlava added 6 commits February 14, 2025 15:32

Initial version.

316db17

Fix null checks.

4391ac3

Fix tests.

cd6a152

Fix tests and implementation.

2324f68

Fix implementation.

5b1e251

Add tests.

77c3cfc

VillePihlava requested a review from a team as a code owner February 20, 2025 12:44

VillePihlava added 2 commits February 21, 2025 10:12

Remove direct mode search with paging.

9af6665

Refactor bestStreetOnlyCost to streetOnlyCost, related to PageCursor.

7c84581

VillePihlava changed the title ~~Add bestStreetOnlyCost field to the PageCursor to enable using RemoveTransitIfStreetOnlyIsBetter filter with paging~~ Add streetOnlyCost field to the PageCursor to enable using RemoveTransitIfStreetOnlyIsBetter filter with paging Feb 21, 2025

optionsome requested review from eibakke and optionsome February 24, 2025 16:27

optionsome requested changes Feb 25, 2025

View reviewed changes

eibakke reviewed Feb 25, 2025

View reviewed changes

application/src/main/java/org/opentripplanner/routing/algorithm/RoutingWorker.java Outdated Show resolved Hide resolved

VillePihlava added 8 commits February 27, 2025 15:21

Add RemoveTransitIfStreetOnlyIsBetterResults and refactoring.

ba3a817

Make DefaultPageCursorInput constructors private.

97a4749

Change comment.

e59b70a

Change comments.

45a7c43

Refactor how the DefaultPageCursorInput is built in RoutingWorker.

7cabb40

Add toString method to NumItinerariesFilterResults.

b66e83f

Refactor and document RemoveTransitIfStreetOnlyIsBetter, and throw er…

e7b641f

…ror with case that should not happen.

Add documentation.

833acec

VillePihlava changed the title ~~Add streetOnlyCost field to the PageCursor to enable using RemoveTransitIfStreetOnlyIsBetter filter with paging~~ Add generalizedCostMaxLimit field to the PageCursor to enable using RemoveTransitIfStreetOnlyIsBetter filter with paging Mar 6, 2025

t2gran added this to the 2.8 (next release) milestone Mar 12, 2025

VillePihlava added 2 commits March 18, 2025 15:55

Merge branch 'dev-2.x' of github.com:opentripplanner/OpenTripPlanner …

56009b7

…into paging-filtering-fix

Remove unnecessary changes.

d46147f

VillePihlava requested review from eibakke and optionsome March 18, 2025 14:33

t2gran requested changes Mar 26, 2025

View reviewed changes

VillePihlava added 11 commits March 27, 2025 10:32

Refactor DefaultPageCursorInput into org.opentripplanner.routing.algo…

63723fe

…rithm.filterchain.

Rename wholeSwUsed to wholeSearchWindowUsed.

4663c03

Change comments.

6c15475

Change generalizedCostMaxLimit logic in RemoveTransitIfStreetOnlyIsBe…

49eb97b

…tter.

Add get method for numItinerariesFilterResult.

2b7b7e4

Add get method for removeTransitIfStreetOnlyIsBetterResult.

53c8e0f

Add PageCursorInputAggregator.

643f69c

Add PageCursorInputAggregator to ItineraryListFilterChain.

61a3e55

Add functionality for aggregator.

73d96c1

Fix tests.

bfab7ce

Change location of DefaultPageCursorInput.

b2a541e

VillePihlava requested a review from optionsome March 28, 2025 13:53

optionsome reviewed Mar 28, 2025

View reviewed changes

VillePihlava added 6 commits April 1, 2025 10:35

Remove unused imports.

3e99206

Change createPageCursorInput to providePageCursorInput.

d8219ec

Add back removed file FilterTransitWhenDirectModeIsEmptyTest.

1df39de

Change documentation.

92c68fc

Clarify parameter in test.

58fedb9

Remove unnecessary code from tests.

a3801e2

VillePihlava mentioned this pull request Apr 1, 2025

When transitOnly is used, a user should be able to specify a direct mode to filter with that does not appear in the results #6591

Open

VillePihlava added the Improvement A functional improvement label Apr 4, 2025

optionsome requested review from t2gran and optionsome April 8, 2025 09:15

t2gran requested changes Apr 11, 2025

View reviewed changes

...entripplanner/routing/algorithm/raptoradapter/router/FilterTransitWhenDirectModeIsEmpty.java Outdated Show resolved Hide resolved

...cation/src/test/java/org/opentripplanner/model/plan/paging/cursor/PageCursorFactoryTest.java Outdated Show resolved Hide resolved

VillePihlava added 2 commits April 11, 2025 14:36

Change hasDeduplicationParams back to hasDedupeParams.

cf9dde8

Break comment lines at 100 characters.

d830c17

optionsome reviewed Apr 11, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `generalizedCostMaxLimit` field to the `PageCursor` to enable using `RemoveTransitIfStreetOnlyIsBetter` filter with paging #6474

Add `generalizedCostMaxLimit` field to the `PageCursor` to enable using `RemoveTransitIfStreetOnlyIsBetter` filter with paging #6474

VillePihlava commented Feb 20, 2025 •

edited

Loading

codecov bot commented Feb 20, 2025 •

edited

Loading

VillePihlava commented Feb 21, 2025

VillePihlava commented Mar 25, 2025

t2gran left a comment

t2gran Mar 26, 2025 •

edited

Loading

VillePihlava Mar 27, 2025

t2gran Mar 26, 2025

t2gran Mar 26, 2025

VillePihlava Mar 28, 2025

optionsome Mar 28, 2025

VillePihlava Apr 1, 2025

optionsome Mar 28, 2025

VillePihlava Apr 1, 2025

optionsome Mar 28, 2025

VillePihlava Apr 1, 2025

optionsome commented Mar 28, 2025

optionsome Apr 11, 2025

optionsome Apr 11, 2025

optionsome Apr 11, 2025

optionsome Apr 11, 2025

optionsome Apr 11, 2025

optionsome Apr 11, 2025

optionsome Apr 11, 2025

		import org.opentripplanner.routing.algorithm.filterchain.filters.system.NumItinerariesFilterResult;
		import org.opentripplanner.routing.algorithm.filterchain.filters.transit.RemoveTransitIfStreetOnlyIsBetterResult;

	* In case the result has too many results: The {@code numberOfItineraries} request parameter
	* In case the result has too many results, if the {@code numberOfItineraries} request parameter

		* is less than the number of itineraries found, then we keep the last itinerary kept and
		* returned as part of the result. The sort vector will be included in the page-cursor and

		* @param costLimitFunction the cost limit function to use with the filter
		* @param generalizedCostMaxLimit this limit is not null when paging is used

Add generalizedCostMaxLimit field to the PageCursor to enable using RemoveTransitIfStreetOnlyIsBetter filter with paging #6474

Are you sure you want to change the base?

Add generalizedCostMaxLimit field to the PageCursor to enable using RemoveTransitIfStreetOnlyIsBetter filter with paging #6474

Conversation

VillePihlava commented Feb 20, 2025 • edited Loading

Summary

Issue

Unit tests

Documentation

codecov bot commented Feb 20, 2025 • edited Loading

Codecov Report

VillePihlava commented Feb 21, 2025

VillePihlava commented Mar 25, 2025

t2gran left a comment

Choose a reason for hiding this comment

t2gran Mar 26, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

optionsome commented Mar 28, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Add `generalizedCostMaxLimit` field to the `PageCursor` to enable using `RemoveTransitIfStreetOnlyIsBetter` filter with paging #6474

Add `generalizedCostMaxLimit` field to the `PageCursor` to enable using `RemoveTransitIfStreetOnlyIsBetter` filter with paging #6474

VillePihlava commented Feb 20, 2025 •

edited

Loading

codecov bot commented Feb 20, 2025 •

edited

Loading

t2gran Mar 26, 2025 •

edited

Loading