Revert "perf(wintermute): create bulk staging tables once per connection" - #245
Open
afbase wants to merge 1 commit into
Open
Revert "perf(wintermute): create bulk staging tables once per connection"#245afbase wants to merge 1 commit into
afbase wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reverts #240. Deployed and rolled back the same night: indexing dropped to roughly a tenth of its normal rate, the live queue grew to ~265k, and feeds stopped refreshing. Reverting so
mainis deployable again while the cause is found.The change produced no errors at all — no failed writes, nothing in the journal, health endpoint green throughout. Only throughput moved:
The mechanism is not yet identified. Round trips are unchanged at eleven
batch_executecalls per batch either way, the SQL is strictly smaller after the change, and the pool wiring inindexer/mod.rsis a pure refactor with identical settings. So the obvious explanations do not hold and it is something subtler — connection churn making thepost_createhook expensive, or aTRUNCATE-versus-CREATEinteraction under sharding, are the current leads.Worth noting for whoever picks this up: the full test suite passes 75/0 against a correctly built schema both with and without this change, in the same wall-clock time. Nothing in the suite measures throughput, so no test would have caught this.
The
search_pathfix that rode along in #240 goes away with it. That is correct — the bug it fixed was introduced by #240's ownConfig::optionsusage, so reverting removes both. Nothing onmaindepends on it.Test plan
cargo clippy -p rsky-wintermute --all-targets --no-deps -- -D warningscargo test -p rsky-wintermuteagainst a database with the appview schemacargo build --release -p rsky-wintermute