Skip to content

Revert "perf(wintermute): create bulk staging tables once per connection" - #245

Open
afbase wants to merge 1 commit into
mainfrom
revert/wintermute-bulk-staging-hoist
Open

Revert "perf(wintermute): create bulk staging tables once per connection"#245
afbase wants to merge 1 commit into
mainfrom
revert/wintermute-bulk-staging-hoist

Conversation

@afbase

@afbase afbase commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Reverts #240. Deployed and rolled back the same night: indexing dropped to roughly a tenth of its normal rate, the live queue grew to ~265k, and feeds stopped refreshing. Reverting so main is deployable again while the cause is found.

The change produced no errors at all — no failed writes, nothing in the journal, health endpoint green throughout. Only throughput moved:

this change:  44/s
reverted:     6,100-6,900/s while draining the backlog
steady state: ~400/s

The mechanism is not yet identified. Round trips are unchanged at eleven batch_execute calls per batch either way, the SQL is strictly smaller after the change, and the pool wiring in indexer/mod.rs is a pure refactor with identical settings. So the obvious explanations do not hold and it is something subtler — connection churn making the post_create hook expensive, or a TRUNCATE-versus-CREATE interaction under sharding, are the current leads.

Worth noting for whoever picks this up: the full test suite passes 75/0 against a correctly built schema both with and without this change, in the same wall-clock time. Nothing in the suite measures throughput, so no test would have caught this.

The search_path fix that rode along in #240 goes away with it. That is correct — the bug it fixed was introduced by #240's own Config::options usage, so reverting removes both. Nothing on main depends on it.

Test plan

  • cargo clippy -p rsky-wintermute --all-targets --no-deps -- -D warnings
  • cargo test -p rsky-wintermute against a database with the appview schema
  • cargo build --release -p rsky-wintermute
  • Deployed indexing rate returns to its normal steady state

…ute-hoist-bulk-staging-ddl"

This reverts commit 8116613, reversing
changes made to c77f233.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant