Skip to content

feat(state): allow storing of state in tmpfs instead of disk#961

Merged
xu282934741 merged 13 commits intokubewharf:mainfrom
JustinChengLZ:dev/tmpfs-state-migration
Nov 12, 2025
Merged

feat(state): allow storing of state in tmpfs instead of disk#961
xu282934741 merged 13 commits intokubewharf:mainfrom
JustinChengLZ:dev/tmpfs-state-migration

Conversation

@JustinChengLZ
Copy link
Copy Markdown
Collaborator

What type of PR is this?

Some machines have bad disk performance so we can provide the option to store in memory (tmpfs). We need to ensure that when we want to store state in memory, we migrate the old state file from disk to memory.

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

@codecov
Copy link
Copy Markdown

codecov bot commented Sep 4, 2025

Codecov Report

❌ Patch coverage is 68.01347% with 95 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.79%. Comparing base (499f48f) to head (161c219).
⚠️ Report is 42 commits behind head on main.

Files with missing lines Patch % Lines
...ins/memory/dynamicpolicy/state/state_checkpoint.go 55.55% 18 Missing and 6 partials ⚠️
...gent/qrm-plugins/network/state/state_checkpoint.go 60.37% 15 Missing and 6 partials ⚠️
...lugins/cpu/dynamicpolicy/state/state_checkpoint.go 61.22% 14 Missing and 5 partials ⚠️
.../util/qrmcheckpointmanager/qrmcheckpointmanager.go 77.77% 9 Missing and 9 partials ⚠️
.../options/qrm/statedirectory/statedirectory_base.go 52.63% 9 Missing ⚠️
cmd/katalyst-agent/app/options/qrm/qrm_base.go 25.00% 2 Missing and 1 partial ⚠️
pkg/agent/qrm-plugins/cpu/nativepolicy/policy.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #961      +/-   ##
==========================================
+ Coverage   59.65%   59.79%   +0.13%     
==========================================
  Files         682      684       +2     
  Lines       63929    64165     +236     
==========================================
+ Hits        38137    38366     +229     
+ Misses      21385    21372      -13     
- Partials     4407     4427      +20     
Flag Coverage Δ
unittest 59.79% <68.01%> (+0.13%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@JustinChengLZ JustinChengLZ force-pushed the dev/tmpfs-state-migration branch 6 times, most recently from 82167ca to bafbde4 Compare September 11, 2025 06:11
@JustinChengLZ JustinChengLZ force-pushed the dev/tmpfs-state-migration branch 13 times, most recently from 8879d87 to a29aa69 Compare September 17, 2025 08:17
@JustinChengLZ JustinChengLZ force-pushed the dev/tmpfs-state-migration branch 5 times, most recently from 7e5e442 to 3caa357 Compare September 22, 2025 03:46
@JustinChengLZ JustinChengLZ added the workflow/need-review review: test succeeded, need to review label Sep 23, 2025
@JustinChengLZ JustinChengLZ force-pushed the dev/tmpfs-state-migration branch from e02a9e6 to 1fa3948 Compare September 24, 2025 04:15
@JustinChengLZ JustinChengLZ force-pushed the dev/tmpfs-state-migration branch from 1fa3948 to e5cb966 Compare September 24, 2025 07:57
@JustinChengLZ JustinChengLZ changed the title [WIP] feat(state): allow storing of state in tmpfs instead of disk feat(state): allow storing of state in tmpfs instead of disk Sep 24, 2025
@JustinChengLZ JustinChengLZ force-pushed the dev/tmpfs-state-migration branch from e5cb966 to b8ad643 Compare September 24, 2025 08:04
@JustinChengLZ JustinChengLZ force-pushed the dev/tmpfs-state-migration branch from b8ad643 to 7857a16 Compare October 16, 2025 03:02
fix: bug in storing state
fix: tests

chore: format
feat: unit tests

chore: debug

fix: bug

fix: breaking unit tests

chore: lint
feat: remove old checkpoint file

feat: remove old checkpoint file

revert: sysadvisor changes

feat: refactor

chore: format

fix: build errors

chore: format

fix: build errors

fix: unit tests

# Conflicts:
#	pkg/agent/qrm-plugins/cpu/dynamicpolicy/hintoptimizer/policy/memorybandwidth/optimizer_test.go
feat: implement pre-stop support and checkpoint fallback

feat: implement pre-stop support and checkpoint fallback

feat: implement pre-stop support and checkpoint fallback

feat: implement pre-stop support and checkpoint fallback
feat: clean up test cases and improve code coverage

feat: clean up test cases and improve code coverage

feat: clean up test cases and improve code coverage

feat: clean up test cases and improve code coverage

feat: clean up test cases and improve code coverage
feat: refactor logic into new file

feat: refactor logic into new file
fix: issue with error not propagating properly

fix: issue with error not propagating properly

fix: issue with error not propagating properly

chore: goimport

chore: goimport
@JustinChengLZ JustinChengLZ force-pushed the dev/tmpfs-state-migration branch from 7857a16 to be74612 Compare November 5, 2025 07:53
feat: implement time check to check if state files are up to date

feat: implement time check to check if state files are up to date

feat: implement time check to check if state files are up to date

chore: rebase

chore: rebase
@JustinChengLZ JustinChengLZ force-pushed the dev/tmpfs-state-migration branch from be74612 to e3bbf3c Compare November 5, 2025 08:10
@xu282934741 xu282934741 merged commit 2730b5d into kubewharf:main Nov 12, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

workflow/need-review review: test succeeded, need to review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants