Skip to content

feat(logger): implement async logger#985

Merged
xu282934741 merged 7 commits intokubewharf:mainfrom
JustinChengLZ:dev/async-logging
Nov 11, 2025
Merged

feat(logger): implement async logger#985
xu282934741 merged 7 commits intokubewharf:mainfrom
JustinChengLZ:dev/async-logging

Conversation

@JustinChengLZ
Copy link
Copy Markdown
Collaborator

What type of PR is this?

We want to optimise async klog's native logging mechanism by making the write function asynchronous.

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

@codecov
Copy link
Copy Markdown

codecov bot commented Sep 22, 2025

Codecov Report

❌ Patch coverage is 82.05128% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.70%. Comparing base (4bbf9a0) to head (533503c).
⚠️ Report is 26 commits behind head on main.

Files with missing lines Patch % Lines
cmd/base/options/log.go 60.00% 4 Missing ⚠️
pkg/util/logging/logger.go 89.28% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #985      +/-   ##
==========================================
- Coverage   59.70%   59.70%   -0.01%     
==========================================
  Files         682      683       +1     
  Lines       63844    63881      +37     
==========================================
+ Hits        38117    38138      +21     
- Misses      21322    21336      +14     
- Partials     4405     4407       +2     
Flag Coverage Δ
unittest 59.70% <82.05%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@JustinChengLZ JustinChengLZ force-pushed the dev/async-logging branch 8 times, most recently from 314fb9a to 7d6b5bd Compare September 26, 2025 09:27
@JustinChengLZ JustinChengLZ force-pushed the dev/async-logging branch 2 times, most recently from 067657d to af06886 Compare October 17, 2025 08:02
@JustinChengLZ JustinChengLZ changed the title [WIP] feat(logger): implement async logger feat(logger): implement async logger Oct 17, 2025
@JustinChengLZ JustinChengLZ added the workflow/need-review review: test succeeded, need to review label Oct 17, 2025
@JustinChengLZ JustinChengLZ force-pushed the dev/async-logging branch 10 times, most recently from 2da90cc to 3a74795 Compare October 21, 2025 09:36
@JustinChengLZ JustinChengLZ force-pushed the dev/async-logging branch 5 times, most recently from 100a8ba to 986347c Compare October 22, 2025 05:33
feat: implement async logger

feat: implement async logger
chore: unit tests

chore: unit tests
feat: split up logs based on severity

feat: split up logs based on severity

feat: split up logs based on severity

feat: split up logs based on severity

feat: split up logs based on severity
feat: add cleanup parameters in async logging

feat: add cleanup parameters in async logging
fix: PR comments

fix: PR comments

fix: PR comments

fix: PR comments

fix: PR comments

fix: PR comments

fix: PR comments

fix: PR comments

fix: PR comments

fix: PR comments

fix: PR comments
feat: make logger more dynamic by allowing synchronous lumberjack

feat: make logger more dynamic by allowing synchronous lumberjack

feat: make logger more dynamic by allowing synchronous lumberjack

feat: make logger more dynamic by allowing synchronous lumberjack

feat: make logger more dynamic by allowing synchronous lumberjack
// Enable async logger if buffer size is more than 0; otherwise, use synchronous lumberjack logger
if bufferSize > 0 {
// diodeWriter is a writer that stores logs in a ring buffer and asynchronously flushes them to disk
diodeWriter := diode.NewWriter(lumberjackLogger, bufferSize, 10*time.Millisecond, func(missed int) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why using poller rather than waiter?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Polling performs better with high frequency logs; for waiting, the high frequency of logs will likely cause high frequency of context switching for waking up goroutines at every log, probably leading to high overhead.

@xu282934741 xu282934741 merged commit aa29c4b into kubewharf:main Nov 11, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

workflow/need-review review: test succeeded, need to review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants