Skip to content

Fix: 优化日志体系,避免重复落盘,缓解日志文件过大的问题#267

Merged
1bananachicken merged 1 commit into
1bananachicken:devfrom
XUANHLGG:dev
Jun 5, 2026
Merged

Fix: 优化日志体系,避免重复落盘,缓解日志文件过大的问题#267
1bananachicken merged 1 commit into
1bananachicken:devfrom
XUANHLGG:dev

Conversation

@XUANHLGG
Copy link
Copy Markdown
Contributor

@XUANHLGG XUANHLGG commented Jun 2, 2026

原本日志体系有两条链路 ,agent/custom/action/Common/logger.py 中额外维护了一套独立日志输出,持续写入 debug/agent.log ,这可能造成同类日志重复记录且该链路没有轮转与保留策略,导致仅使用几天后 debug 文件夹大小持续增长至50GB
所以将 custom action 的独立日志并入统一日志体系,复用 agent/utils/logger.py 中已有的轮转与保留策略

Summary by Sourcery

将自定义动作日志记录统一到共享的应用程序日志记录器中,以减少重复的日志写入,并利用现有的轮转/保留策略。

Enhancements:

  • 通过共享的 get_logger 帮助函数集中获取日志记录器,并使用一致的应用程序日志记录器名称。
  • 通过根日志记录器配置日志,包括对噪声较大的第三方库进行标准化的日志级别调整,同时为调用方返回一个应用程序特定的日志记录器。
  • 用共享的日志基础设施替换自定义 action 模块中独立的文件/控制台日志记录器配置。
Original summary in English

Summary by Sourcery

Unify custom action logging with the shared application logger to reduce duplicate log writes and leverage existing rotation/retention policies.

Enhancements:

  • Centralize logger retrieval through a shared get_logger helper using a consistent application logger name.
  • Configure logging via the root logger, including standardized noisy-library level adjustments, while returning an application-specific logger for use by callers.
  • Replace the custom action module's standalone file/console logger setup with reuse of the shared logging infrastructure.

@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai Bot commented Jun 2, 2026

审阅者指南

将自定义 Action 日志统一接入共享日志系统,使所有日志都通过已配置的 root/app logger 并带日志轮转,同时移除单独的 agent.log 路径,并收紧对噪声较大的第三方 logger 的配置。

统一自定义 Action 日志流水线流程图

flowchart LR
    A[CustomAction code
    get_logger] --> B[utils.logger
    get_logger]
    B --> C[AppLogger maante]
    C --> D[RootLogger]
    D --> E[ConsoleHandler]
    D --> F[TimedRotatingFileHandler
    debug/custom/runtime.log]
    D -.configured level .-> G[Noisy third party loggers
    cv2/numpy/PIL/...]
Loading

文件级变更

Change Details Files
将 logger 初始化重构为使用 root logger 加具名应用 logger,并集中配置噪声较大的第三方 logger。
  • 为应用 logger 名称和 root logger 引入常量,并新增一个辅助方法通过名称获取 app logger。
  • 新增辅助函数,将噪声较大的第三方 logger 的日志级别统一设置为 WARNING,并在 loguru 和标准库 logging 的初始化中复用。
  • 修改 loguru 和标准库 logger 的初始化方法,在 root logger 上配置 handler,然后返回启用传播(propagate)的、配置正确的 app logger。
  • 更新 setup_logger,使其仅负责初始化 logging,并通过一个基于 get_logger 的独立 logger 实例对外暴露;相应调整模块导出。
agent/utils/logger.py
移除自定义 Action 使用的独立日志实现,复用共享 logger 辅助方法。
  • 删除直接写入 debug/agent.log 的本地日志配置逻辑。
  • 从共享的 utils.logger 模块中以内部别名方式导入 get_logger。
  • 通过委托给共享 logger 辅助方法对外暴露自定义 Action 使用的 get_logger,确保其纳入统一的日志配置与轮转策略。
agent/custom/action/Common/logger.py

可能关联的 Issue

  • #alpha.2日志膨胀: PR 移除独立的 agent.log 写入,并接入带轮转与保留策略的统一日志系统,以缓解日志膨胀问题。
  • #日志文件过大: Issue 提到日志过大;PR 统一日志链路并启用轮转与保留策略,以缓解日志膨胀。

技巧与命令

与 Sourcery 交互

  • 触发新审阅: 在 pull request 中评论 @sourcery-ai review
  • 继续讨论: 直接回复 Sourcery 的审阅评论。
  • 从审阅评论生成 GitHub issue: 回复 Sourcery 的审阅评论,要求其从该评论创建 issue。你也可以直接回复该审阅评论 @sourcery-ai issue 来创建 issue。
  • 生成 pull request 标题: 在 pull request 标题任意位置写上 @sourcery-ai,即可随时生成标题。也可以在 pull request 中评论 @sourcery-ai title 来(重新)生成标题。
  • 生成 pull request 摘要: 在 pull request 内容任意位置写上 @sourcery-ai summary,即可在该位置生成 PR 摘要。也可以在 pull request 中评论 @sourcery-ai summary 来(重新)生成摘要。
  • 生成审阅者指南: 在 pull request 中评论 @sourcery-ai guide,即可随时(重新)生成审阅者指南。
  • 一次性解决所有 Sourcery 评论: 在 pull request 中评论 @sourcery-ai resolve,即可标记解决所有 Sourcery 评论。如果你已经处理完所有评论且不希望再看到它们,这会很有用。
  • 一次性忽略所有 Sourcery 审阅: 在 pull request 中评论 @sourcery-ai dismiss,即可忽略所有现有的 Sourcery 审阅。如果你想从一次全新的审阅开始,这尤其有用 —— 别忘了再评论 @sourcery-ai review 触发新的审阅!

自定义使用体验

访问你的控制台 来:

  • 启用或停用审阅功能,例如 Sourcery 自动生成的 pull request 摘要、审阅者指南等。
  • 更改审阅语言。
  • 添加、删除或编辑自定义审阅指令。
  • 调整其他审阅相关设置。

获取帮助

Original review guide in English

Reviewer's Guide

Unifies custom action logging with the shared logging system so all logs go through the configured root/app logger with rotation, and removes the separate agent.log path while tightening how noisy third‑party loggers are configured.

Flow diagram for unified custom action logging pipeline

flowchart LR
    A[CustomAction code
    get_logger] --> B[utils.logger
    get_logger]
    B --> C[AppLogger maante]
    C --> D[RootLogger]
    D --> E[ConsoleHandler]
    D --> F[TimedRotatingFileHandler
    debug/custom/runtime.log]
    D -.configured level .-> G[Noisy third party loggers
    cv2/numpy/PIL/...]
Loading

File-Level Changes

Change Details Files
Refactor logger setup to use root logger plus a named application logger, and centralize configuration of noisy third‑party loggers.
  • Introduce constants for application logger name and root logger, and add a helper to obtain the app logger by name.
  • Add a helper function to uniformly set WARNING level on noisy third‑party loggers and use it in both loguru and std logging setups.
  • Change loguru and std logger setup functions to configure handlers on the root logger, then return a properly configured app logger with propagate enabled.
  • Update setup_logger to only initialize logging and expose a separate get_logger-based logger instance; adjust module exports accordingly.
agent/utils/logger.py
Remove the custom, standalone logging implementation for custom actions and reuse the shared logger helper.
  • Delete the local logging configuration logic that wrote directly to debug/agent.log.
  • Import get_logger from the shared utils.logger module under an internal alias.
  • Expose get_logger for custom actions by delegating to the shared logger helper, ensuring they participate in the unified logging configuration and rotation.
agent/custom/action/Common/logger.py

Possibly linked issues

  • #alpha.2日志膨胀: PR移除独立agent.log写入并接入带轮转与保留策略的统一日志系统,缓解日志膨胀问题。
  • #日志文件过大: Issue提到日志过大;PR统一日志链路并启用轮转保留策略以缓解日志膨胀。

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - 我发现了两个问题,并给出了一些整体层面的反馈:

  • _setup_loguru_logger 中覆盖 _ROOT_LOGGER.handlers(以及在 _setup_std_logger 中清空 handlers)可能会意外清除由嵌入该库的应用程序所设置的日志配置;建议改为在现有 handlers 基础上追加你的 handler,或提供一个显式的可选开关来接管 root logger。
  • 由于 get_logger 现在返回的是 logging.getLogger(name or _APP_LOGGER_NAME),而 setup_logger 只配置了 root logger 和 app logger,其他任何具名 logger 都完全依赖日志传播;如果你期望自定义组件使用自己的名称进行日志记录,建议在文档中说明,或显式配置这些 logger 的日志级别/传播行为。
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Overwriting `_ROOT_LOGGER.handlers` in `_setup_loguru_logger` (and clearing handlers in `_setup_std_logger`) can unexpectedly wipe logging configuration set up by the embedding application; consider appending your handlers or providing an opt-in flag for taking over the root logger.
- Since `get_logger` now returns `logging.getLogger(name or _APP_LOGGER_NAME)` and `setup_logger` only configures the root and the app logger, any other named loggers rely solely on propagation; if you expect custom components to log with their own names, consider documenting or explicitly configuring those logger levels/propagation behavior.

## Individual Comments

### Comment 1
<location path="agent/utils/logger.py" line_range="186-187" />
<code_context>
-    logging.root.setLevel(logging.DEBUG)
-    for name in ("cv2", "numpy", "PIL", "matplotlib", "urllib3", "asyncio"):
-        logging.getLogger(name).setLevel(logging.WARNING)
+    _ROOT_LOGGER.handlers = [_InterceptHandler()]
+    _ROOT_LOGGER.setLevel(logging.DEBUG)
+    _configure_noisy_loggers()

</code_context>
<issue_to_address>
**issue (bug_risk):** Overwriting root handlers can clobber existing logging configuration in embedding applications.

Assigning `[_InterceptHandler()]` to `_ROOT_LOGGER.handlers` (plus the `handlers.clear()` path for std logging) wipes out any handlers configured by the host process or other libraries, making this change globally invasive. In environments where logging is preconfigured (frameworks, notebooks, multi-service apps), consider appending the handler only when missing or guarding the reset behind an explicit opt-in flag to avoid unexpectedly discarding existing setups.
</issue_to_address>

### Comment 2
<location path="agent/utils/logger.py" line_range="190-193" />
<code_context>
+    _configure_noisy_loggers()

-    return _loguru_logger
+    app_logger = get_logger(_APP_LOGGER_NAME)
+    app_logger.setLevel(logging.NOTSET)
+    app_logger.propagate = True
+    return app_logger


</code_context>
<issue_to_address>
**question (bug_risk):** Changing `setup_logger` to return an app `logging.Logger` instead of the underlying loguru logger may break call sites relying on loguru APIs.

Previously `setup_logger()` returned the loguru logger (`_loguru_logger`), exposing loguru-specific methods like `.bind` and `.catch`. It now returns a stdlib `logging.Logger`, so any callers that stored this value and used loguru APIs will break or change behavior. If callers depend on that richer API, consider either preserving the original return type or splitting responsibilities (e.g., a `get_app_logger()` plus a `setup_logging()` that returns `None`).
</issue_to_address>

Sourcery 对开源项目是免费的 - 如果你觉得这次 review 有帮助,欢迎分享 ✨
帮我变得更有用!请对每条评论点 👍 或 👎,我会根据你的反馈改进后续的 review。
Original comment in English

Hey - I've found 2 issues, and left some high level feedback:

  • Overwriting _ROOT_LOGGER.handlers in _setup_loguru_logger (and clearing handlers in _setup_std_logger) can unexpectedly wipe logging configuration set up by the embedding application; consider appending your handlers or providing an opt-in flag for taking over the root logger.
  • Since get_logger now returns logging.getLogger(name or _APP_LOGGER_NAME) and setup_logger only configures the root and the app logger, any other named loggers rely solely on propagation; if you expect custom components to log with their own names, consider documenting or explicitly configuring those logger levels/propagation behavior.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Overwriting `_ROOT_LOGGER.handlers` in `_setup_loguru_logger` (and clearing handlers in `_setup_std_logger`) can unexpectedly wipe logging configuration set up by the embedding application; consider appending your handlers or providing an opt-in flag for taking over the root logger.
- Since `get_logger` now returns `logging.getLogger(name or _APP_LOGGER_NAME)` and `setup_logger` only configures the root and the app logger, any other named loggers rely solely on propagation; if you expect custom components to log with their own names, consider documenting or explicitly configuring those logger levels/propagation behavior.

## Individual Comments

### Comment 1
<location path="agent/utils/logger.py" line_range="186-187" />
<code_context>
-    logging.root.setLevel(logging.DEBUG)
-    for name in ("cv2", "numpy", "PIL", "matplotlib", "urllib3", "asyncio"):
-        logging.getLogger(name).setLevel(logging.WARNING)
+    _ROOT_LOGGER.handlers = [_InterceptHandler()]
+    _ROOT_LOGGER.setLevel(logging.DEBUG)
+    _configure_noisy_loggers()

</code_context>
<issue_to_address>
**issue (bug_risk):** Overwriting root handlers can clobber existing logging configuration in embedding applications.

Assigning `[_InterceptHandler()]` to `_ROOT_LOGGER.handlers` (plus the `handlers.clear()` path for std logging) wipes out any handlers configured by the host process or other libraries, making this change globally invasive. In environments where logging is preconfigured (frameworks, notebooks, multi-service apps), consider appending the handler only when missing or guarding the reset behind an explicit opt-in flag to avoid unexpectedly discarding existing setups.
</issue_to_address>

### Comment 2
<location path="agent/utils/logger.py" line_range="190-193" />
<code_context>
+    _configure_noisy_loggers()

-    return _loguru_logger
+    app_logger = get_logger(_APP_LOGGER_NAME)
+    app_logger.setLevel(logging.NOTSET)
+    app_logger.propagate = True
+    return app_logger


</code_context>
<issue_to_address>
**question (bug_risk):** Changing `setup_logger` to return an app `logging.Logger` instead of the underlying loguru logger may break call sites relying on loguru APIs.

Previously `setup_logger()` returned the loguru logger (`_loguru_logger`), exposing loguru-specific methods like `.bind` and `.catch`. It now returns a stdlib `logging.Logger`, so any callers that stored this value and used loguru APIs will break or change behavior. If callers depend on that richer API, consider either preserving the original return type or splitting responsibilities (e.g., a `get_app_logger()` plus a `setup_logging()` that returns `None`).
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread agent/utils/logger.py
Comment thread agent/utils/logger.py
@1bananachicken
Copy link
Copy Markdown
Owner

加一下开发群喵 1092630280

@XUANHLGG
Copy link
Copy Markdown
Contributor Author

XUANHLGG commented Jun 3, 2026

加一下开发群喵 1092630280

加了喵

@EeeMaoY
Copy link
Copy Markdown
Collaborator

EeeMaoY commented Jun 3, 2026

其实应该把所有用agent\custom\action\Common\logger.py的地方换成agent\utils\logger.py,之前用的算是历史遗留问题(没发现从m9a那里copy来了logger

@Watanabehato Watanabehato changed the title 优化日志体系,避免重复落盘,缓解日志文件过大的问题 Fix: 优化日志体系,避免重复落盘,缓解日志文件过大的问题 Jun 5, 2026
@1bananachicken 1bananachicken merged commit 8060e15 into 1bananachicken:dev Jun 5, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants