Skip to content

Add cleanup method; add support for merged metric files collection#1

Open
agoncharov-reef wants to merge 2 commits intomasterfrom
multiproc-db-cleanup
Open

Add cleanup method; add support for merged metric files collection#1
agoncharov-reef wants to merge 2 commits intomasterfrom
multiproc-db-cleanup

Conversation

@agoncharov-reef
Copy link
Collaborator

Usually built-in mark_process_dead() removes stale *.db metrics files. However, there are cases when it doesn't happen (for example, celery worker dies), and stale files are left. We want to collect & collapse them.

  • First, when creating a *.db file, we acquire advisory lock (flock).
  • Second, when we perform a cleanup, we select files which are not locked, read them and write combined result into merged_metrics.pkl. If the latter already exists, we combine results with it, too.
  • Third, when we collect metrics, we not only read *.db files but also read merged_metrics.pkl and merge everything and then output it.

@agoncharov-reef
Copy link
Collaborator Author

@copilot Check this stuff and make it so nice that everybody immediately wants to raise my salary.

Copy link

Copilot AI commented Mar 16, 2026

@agoncharov-reef I've opened a new pull request, #2, to work on those changes. Once the pull request is ready, I'll request review from you.

@agoncharov-reef
Copy link
Collaborator Author

Mf, you should've made it pretty, not show all my ugly mistakes!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants