Skip to content

Fix history in depth bombed rooms - #20157

Open
catfromplan9 wants to merge 1 commit into
element-hq:developfrom
catfromplan9:fix-depth-history
Open

Fix history in depth bombed rooms#20157
catfromplan9 wants to merge 1 commit into
element-hq:developfrom
catfromplan9:fix-depth-history

Conversation

@catfromplan9

Copy link
Copy Markdown
Contributor

In depth bombed rooms (GHSA-v56r-hwv5-mxg6) on room version 6 and above, the server returned everything sent since the attack in the order it received it, not the order it was sent. Federation lag and backfill made those two orders differ, so pagination could jump back months mid-scroll. Rooms on versions 1 to 5, which #18447 skipped, were worse off. There the attack-era events permanently sorted as the newest in the room, and everything sent since sorted as older than them.

The cause in both cases was topological_ordering. Every event created since the attack carried a depth capped at MAX_DEPTH, so the affected stretch of the room collapsed onto one ordering value, and on old room versions it also sat below the uncapped attack events. This PR adds a background update that re-spreads all events at or above MAX_DEPTH across the unused ordering range above the room's highest real depth, ranked by origin_server_ts. Event persistence now stores a new at-cap event just above the room's current maximum, which keeps a repaired room repaired. topological_ordering is local to each server, so nothing changes over federation.

Pull Request Checklist

  • Pull request is based on the develop branch
  • Pull request includes a changelog file. The entry should:
    • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
    • Use markdown where necessary, mostly for code blocks.
    • End with either a period (.) or an exclamation mark (!).
    • Start with a capital letter.
    • Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
  • Code style is correct (run the linters)

In depth bombed rooms (GHSA-v56r-hwv5-mxg6) on room version 6 and
above, the server returned everything sent since the attack in the
order it received it, not the order it was sent. Federation lag and
backfill made those two orders differ, so pagination could jump back
months mid-scroll. Rooms on versions 1 to 5, which element-hq#18447 skipped, were
worse off. There the attack-era events permanently sorted as the newest
in the room, and everything sent since sorted as older than them.

The cause in both cases was `topological_ordering`. Every event created
since the attack carried a depth capped at `MAX_DEPTH`, so the affected
stretch of the room collapsed onto one ordering value, and on old room
versions it also sat below the uncapped attack events. This adds a
background update that re-spreads all events at or above `MAX_DEPTH`
across the unused ordering range above the room's highest real depth,
ranked by `origin_server_ts`. Event persistence now stores a new at-cap
event just above the room's current maximum, which keeps a repaired
room repaired. `topological_ordering` is local to each server, so
nothing changes over federation.

Signed-off-by: cat <cat@plan9.rocks>
@catfromplan9
catfromplan9 requested a review from a team as a code owner August 27, 2026 14:36
@catfromplan9
catfromplan9 requested review from erikjohnston and removed request for a team August 27, 2026 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant