Skip to content

Conversation

@luohaha
Copy link
Contributor

@luohaha luohaha commented Dec 26, 2025

Why I'm doing:

After dropping the database, the load jobs in the Load Manager were not cleared. This causes the system to frequently print error logs when querying the system table loads :

Get transaction xxx in db xxx failed. msg: databaseTransactionMgr[xxx] does not exist

What I'm doing:

This pull request introduces a new mechanism to clean up load jobs associated with a database when the database is erased. The main changes add a method to remove all load jobs for a given database, integrate it into the database deletion workflow, and provide comprehensive unit tests for this functionality.

Load Job Cleanup Improvements:

  • Added a new removeLoadJobsByDb(long dbId) method to the LoadMgr class to remove all load jobs belonging to a specified database, ensuring proper locking and cleanup.
  • Updated the LocalMetastore.onEraseDatabase(long dbId) method to call removeLoadJobsByDb, so that all load jobs are removed when a database is erased.

Testing Enhancements:

  • Added a new unit test testRemoveLoadJobsByDb in LoadMgrTest.java to verify that load jobs are correctly removed for a given database, including checks for multiple databases and job states.

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
    • This pr needs auto generate documentation
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 4.0
    • 3.5
    • 3.4
    • 3.3

Note

Ensures load jobs tied to a database are purged when the database is erased.

  • Adds LoadMgr.removeLoadJobsByDb(long dbId) to remove all jobs for a DB under write lock using unprotectedRemoveJobReleatedMeta
  • Hooks cleanup into LocalMetastore.onEraseDatabase after removing the DB transaction manager
  • Adds LoadMgrTest.testRemoveLoadJobsByDb to verify per-DB job removal and internal map cleanup

Written by Cursor Bugbot for commit 32dbf7c. This will update automatically on new commits. Configure here.

@alvin-celerdata
Copy link
Contributor

@cursor review

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no bugs!

@sonarqubecloud
Copy link

@github-actions
Copy link

[Java-Extensions Incremental Coverage Report]

pass : 0 / 0 (0%)

@github-actions
Copy link

[FE Incremental Coverage Report]

pass : 11 / 11 (100.00%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 com/starrocks/server/LocalMetastore.java 1 1 100.00% []
🔵 com/starrocks/load/loadv2/LoadMgr.java 10 10 100.00% []

@github-actions
Copy link

[BE Incremental Coverage Report]

pass : 0 / 0 (0%)

@alvin-celerdata
Copy link
Contributor

@cursor review

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no bugs!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants