Timestamp sorting seems incorrect

[ekeric13]

When sorting by timestamp by newest first, cannot seem to return latest events. Instead the partition/offset is still prioritized.

Furthermore it seems that setting a timestamp does not filter out events that are older than it.

[github-actions]

Hello there ekeric13! 👋

Thank you and congratulations 🎉 for opening your very first issue in this project! 💖

In case you want to claim this issue, please comment down below! We will try to get back to you as soon as we can. 👀

[iliax]

Hello @ekeric13
Thank you for the ticket.

Behaviour you provided is expected, since offset/timestamp selection doesnt provided sorting. It only provides the way to seek to specific position where we start reading from (find offset related to that time).

In Oldest first mode we will start reading from this position to the direction of newer records (and older messages will be in the beginning on this list)
In Newest first mode we treat found position as a until position and will read to the direction of "older" records.

Note: the only ordering that we can easily maintain in resulting page is per-partition ordering. Global ordering maintenance is hard to implement, since it require unpredictable amount of records to be read.

Btw, we know that this naming is very confusing and will are currently working on page redesign. So, I think it will be clearer in the future.

[iliax]

NOTE
currently timestamp/offset seek functionality broken in master branch.
should be fixed after #2624