Filtering Bug

[nyoung697]

Update

I created an MRE here: https://github.com/nyoung697/geospatial-mre

Reproduce:

1. exec function data/loadData (no args)
2. exec function data/get args:

{
  distance: 10000,
  point: { latitude: 50.9753869, longitude: -114.0897771 },
  weekdays: [3],
  timeFilter: { start: 1140, end: 1150 },
}

Change the end time to 1160, there will be 0 results returned...

---

The sortKey doesn't work...

I've loaded about 2500 meetings into the database and have the sortKey set as the startTime in minutes since midnight.

I am trying to query all meetings nearby that start between a certain time frame:

let cursor: string | undefined = undefined;
const candidates: GeospatialResults = [];

  do {
    const { results, nextCursor } = await geospatial.query(
      ctx,
       {
        limit,
        shape: { type: "rectangle", rectangle },
        filter: (q) =>
          q
            .gte("sortKey", timeFilter.start)
            .lt("sortKey", timeFilter.end)
            .in("weekday", weekdays),
      },
      cursor,
     );

  candidates.push(...results);
  cursor = nextCursor; // stop when nextCursor is undefined/null
} while (cursor);

I can execute:

{
  distance: 10000,
  point: { latitude: 50.97537, longitude: -114.090383 },
  weekdays: [3],
  timeFilter: { start: 1140, end: 1150 },
  limit: 5
}

And I will get back the 3 meetings nearby that start at 7pm... However, if I change the end time to 1160 or 2500 or whatever value that is higher than that, I get 0 results back...

I can search the points and see that the sortKey is correct and it falls between that range.

If I remove the filtering by sortKey, the result will come back.

I have tried this with both the .query and .nearest functions. I have tried changing the limit, and adding/removing other filters.

I have no idea what is happening here... The filtering seems to be quite buggy.

[nyoung697]

Here are screenshots of the query results. Showing that if I change the end time to 1160, it returns 0 results. Even though the sortKey is still within the range:

[ianmacartney]

Very strange, thanks for the minimal repro. I can confirm this looks like a bug.
I have a "fix" in #28, but it's not backwards compatible. You can use that version to validate it works for you, but you'll need to re-insert your data.
It will take some work to make a migration script to re-encode the data, and I think we should to change the key encoding method to unify it with other sorted tuple implementations.

[nyoung697]

Thank you @ianmacartney. I can confirm this PR fixes the issue on my dataset.