[Refactor] Transform Table/Db/Partition related simple edit logs to WAL format #67245
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gengjun-git
changed the title
Contributor
|
@cursor review |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
fe/fe-core/src/main/java/com/starrocks/transaction/TransactionIdGenerator.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/com/starrocks/server/LocalMetastore.java
Outdated
Show resolved
Hide resolved
Contributor
Author
|
@mergify rebase |
Signed-off-by: gengjun-git <[email protected]>
Signed-off-by: gengjun-git <[email protected]>
Signed-off-by: gengjun-git <[email protected]>
Signed-off-by: gengjun-git <[email protected]>
Signed-off-by: gengjun-git <[email protected]>
Signed-off-by: gengjun-git <[email protected]>
Contributor
✅ Branch has been successfully rebased |
gengjun-git
force-pushed
the
refactor_db_table_simple_logs
branch
from
57b789a to
341e388
Compare
|
[Java-Extensions Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
[FE Incremental Coverage Report]❌ fail : 137 / 222 (61.71%) file detail
|
[BE Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
Contributor
|
@cursor review |
|
|
||
| GlobalStateMgr.getCurrentState().getEditLog() | ||
| .logRecoverPartitionVersion(new PartitionVersionRecoveryInfo(partitionsToRecover, System.currentTimeMillis())); | ||
| .logRecoverPartitionVersion(recoveryInfo, wal -> recoverPartitionVersion(recoveryInfo)); |
There was a problem hiding this comment.
Partition version recovery executed twice in MetaRecoveryDaemon
The recoverPartitionVersion(recoveryInfo) method is called twice during WAL refactoring: once directly at line 181, and again in the WAL callback at line 184. This appears to be an incomplete refactoring where the direct call at line 181 was not removed when the WAL callback pattern was introduced. The correct WAL pattern requires removing the direct call and only executing the operation within the callback. This causes partition versions to be recovered twice, which could lead to redundant operations and potentially unexpected behavior.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why I'm doing:
What I'm doing:
Refer #63357
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check:
Note
Modernizes metadata persistence across FE by shifting many edit-log writes to WAL with in-log appliers and adding replay handlers to ensure atomic, ordered state changes.
WALApplierand updatesEditLogwith applier-based variants (e.g.,logModifyViewDef,logTableRename,logErasePartition,logDeleteReplica,logBackendTabletsInfo, etc.), removing legacy counterpartsAlterJobExecutor/Mgr,AlterMVJobExecutor,SchemaChangeHandler,BackupHandler/RepositoryMgr,CatalogRecycleBin,HiveTable,OlapTablename checks,ReportHandler,TabletScheduler,DeleteJob,LocalMetastore,MaterializedViewMgr,TransactionIdGeneratorcheckNameConflictvscheckAndSetName, alter-view now validates viacheckInlineViewDef, drop-partitions consolidated to batch WAL, range-partition recover split intocheckRecoverableandrecover, and tests updated accordinglyWritten by Cursor Bugbot for commit 341e388. This will update automatically on new commits. Configure here.