Skip to content

Conversation

@ReagentX
Copy link
Owner

@ReagentX ReagentX commented Jun 10, 2025

  • Refactor Processor functionality
    • Fix Togging analytics mode does not reset statistics #130 by implementing reset() on ProcessorMethods trait
    • Refactor Date processor
      • Cache format string for faster updates
      • Compute determine_rate() lazily instead of on update()
    • Refactor Counter processor
      • Use single state HashMap
      • Remove almost all string clones
      • Calculate order only on render
  • Add progress indicator to long process_matches() jobs
  • Remove format_num crate
    • Replace crate with with simple native functions
      • Reduces CPU usage from ≈30% for a 1ms/message stream to ≈4%
    • The format_num crate compiles a regex on every format call, taking up > 90% of app runtime:
      • Click to expand flamegraphsBefore:imageAfter:image

@ReagentX ReagentX self-assigned this Jun 10, 2025
@ReagentX ReagentX added bug Something isn't working enhancement New feature or request extensions Problems with the extensions module filtering Problems with regex search parsing Problems with parsers handler Problems with the input handler classes aggregators Problems with the aggregator modules labels Jun 10, 2025
@ReagentX ReagentX requested a review from Copilot June 10, 2025 00:48

This comment was marked as outdated.

@ReagentX ReagentX requested a review from Copilot June 10, 2025 01:08

This comment was marked as outdated.

@ReagentX ReagentX requested a review from Copilot June 10, 2025 02:49
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request refactors several aggregator processors and processing handlers to improve performance, readability, and overall maintainability. Key changes include the implementation of reset() on the Aggregator trait, removal of the format_num crate in favor of native formatting functions, and enhancements to progress indicators during long-running match processes.

Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/util/aggregators/sum.rs Refactored to use native formatting and added a reset() method.
src/util/aggregators/none.rs Added a no-op reset() method for the disabled aggregator.
src/util/aggregators/mean.rs Updated aggregator to use native formatting and added reset().
src/util/aggregators/date.rs Refactored to cache parsed format and lazily compute rate.
src/util/aggregators/counter.rs Simplified internal state, replaced ordering logic, and refactored constructor APIs.
src/util/aggregators/aggregator.rs Introduced native formatting functions replacing format_num.
src/extensions/parser.rs Updated aggregator construction for mode aggregation.
src/communication/handlers/* Added progress indicator updates and minor refactoring.
Cargo.toml Removed the dependency on the format_num crate.
Comments suppressed due to low confidence (2)

src/util/aggregators/aggregator.rs:60

  • The doc comment for 'format_int' is misleading as it still refers to formatting a float. Consider updating it to accurately describe that it formats an integer.
/// Formats a float as a string with commas separating thousands

src/util/aggregators/counter.rs:51

  • [nitpick] The method name 'mean' for constructing a Counter for mode aggregation may be confusing given the existence of a 'Mean' aggregator. Consider renaming it to something more descriptive such as 'with_single_mode' or 'for_mode_aggregation'.
pub fn mean() -> Counter {

@ReagentX ReagentX merged commit aec2a3e into develop Jun 10, 2025
1 check passed
@ReagentX ReagentX deleted the feat/cs/processor-refactor branch June 10, 2025 03:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aggregators Problems with the aggregator modules bug Something isn't working enhancement New feature or request extensions Problems with the extensions module filtering Problems with regex search handler Problems with the input handler classes parsing Problems with parsers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Togging analytics mode does not reset statistics

2 participants