-
Notifications
You must be signed in to change notification settings - Fork 400
Perf improvements #727
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mcollina
wants to merge
21
commits into
prometheus:main
Choose a base branch
from
mcollina:perf-improvements
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+4,318
−1,463
Open
Perf improvements #727
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
73dd95b
docs: add CLAUDE.md for Claude Code guidance
mcollina 2cf3295
feat: migrate test suite from Jest to Node.js built-in test runner
mcollina 3108ad2
fix: resolve test failures after Jest to node:test migration
mcollina abdbda9
feat: vendor tdigest and bintrees dependencies
mcollina 755e146
perf: optimize tdigest by replacing forEach/map with for loops
mcollina bcbdaf3
Switch findBound function to a binary search
mcollina 77fa56e
Optimize keyFrom
mcollina 6d8eb17
optimize keyFrom
mcollina c73ea76
refactor: use async fs/promises in osMemoryHeapLinux
mcollina 125404b
refactor: add concurrency control and promise-based collection to osM…
mcollina 5c34897
perf: eliminate duplicate sorting in metric creation
mcollina 0e2bb32
fix: skip Linux-only test on non-Linux platforms
mcollina ea63574
perf: optimize histogram and string escaping for better metrics seria…
mcollina 24a18db
perf: replace .map() with for loop in registry.metrics()
mcollina a51e465
perf: avoid array conversion in getMetricsAsJSON
mcollina 60e29f5
fix: resolve linting errors in test files
mcollina 5251e50
perf: eliminate unnecessary promise allocation in metrics and registr…
mcollina 4d589c6
removed TODO
mcollina 480f0ca
docs: update CHANGELOG.md with recent performance improvements and te…
mcollina 176df08
perf: optimize validate() and keyFrom() for metrics without labels
mcollina e25e777
Merge branch 'main' into perf-improvements
mcollina File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| # CLAUDE.md | ||
|
|
||
| This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. | ||
|
|
||
| ## Project Overview | ||
|
|
||
| This is **prom-client**, a Prometheus client library for Node.js that provides metrics collection and exposure functionality. It supports all standard Prometheus metric types: counters, gauges, histograms, and summaries. | ||
|
|
||
| ## Development Commands | ||
|
|
||
| ### Essential Commands | ||
|
|
||
| - `npm test` - Full test suite (lint + prettier + typescript + unit tests with coverage) | ||
| - `npm run test-unit` - Run node:test unit tests only | ||
| - `npm run lint` - ESLint validation | ||
| - `npm run check-prettier` - Prettier formatting check | ||
| - `npm run compile-typescript` - TypeScript compilation check | ||
|
|
||
| ### Running Single Tests | ||
|
|
||
| Tests are located in `/test/` and follow the pattern `*Test.js`. Use node:test directly: | ||
|
|
||
| ```bash | ||
| node --test test/counterTest.js | ||
| node --test test/metrics/ | ||
| node --test "test/**/*Test.js" | ||
| ``` | ||
|
|
||
| ## Code Architecture | ||
|
|
||
| ### Core Structure | ||
|
|
||
| The library is organized around four main metric types in `/lib/`: | ||
|
|
||
| - **counter.js** - Cumulative metrics that only increase | ||
| - **gauge.js** - Metrics that can go up and down | ||
| - **histogram.js** - Samples observations in configurable buckets | ||
| - **summary.js** - Calculates percentiles of observed values | ||
|
|
||
| ### Key Components | ||
|
|
||
| - **registry.js** - Central registry for all metrics, supports Prometheus and OpenMetrics formats | ||
| - **defaultMetrics.js** - Orchestrates collection of Node.js system metrics (CPU, memory, GC, etc.) | ||
| - **cluster.js** - Aggregator registry for Node.js cluster support | ||
| - **pushgateway.js** - Push metrics to Prometheus Pushgateway | ||
|
|
||
| ### Entry Points | ||
|
|
||
| - **index.js** - Main entry point exporting all public APIs | ||
| - **index.d.ts** - Comprehensive TypeScript definitions | ||
|
|
||
| ### Default Metrics (/lib/metrics/) | ||
|
|
||
| System metrics are modular and platform-aware: | ||
|
|
||
| - Some metrics (like file descriptors) are Linux-only | ||
| - Event loop lag, garbage collection, heap usage for Node.js internals | ||
| - Process information and resource usage | ||
|
|
||
| ## Development Patterns | ||
|
|
||
| ### Metric Creation Pattern | ||
|
|
||
| Each metric type follows a consistent pattern: | ||
|
|
||
| 1. Extends base `Metric` class | ||
| 2. Implements `collect()` method returning metric samples | ||
| 3. Supports labels for dimensional metrics | ||
| 4. Registry handles serialization to Prometheus format | ||
|
|
||
| ### Testing Approach | ||
|
|
||
| - Unit tests in `/test/` with Node.js built-in test runner (node:test) | ||
| - Metrics tests in `/test/metrics/` for default metrics | ||
| - Examples in `/example/` demonstrate real usage patterns | ||
| - Mock HTTP requests with `nock` library | ||
| - Timer mocking with `@sinonjs/fake-timers` | ||
| - Test helpers and utilities in `/test/helpers.js` | ||
|
|
||
| ### TypeScript Support | ||
|
|
||
| - Full type definitions maintained alongside JS code | ||
| - `noEmit: true` - types only, no compilation to JS | ||
| - Strict mode enabled for type safety |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,226 @@ | ||
| import { group, bench, run } from 'mitata'; | ||
| import RBTree from '../lib/bintrees/rbtree.js'; | ||
|
|
||
| // Comparator function for numbers | ||
| const compareNumbers = (a, b) => a - b; | ||
|
|
||
| // Comparator function for objects with 'value' property | ||
| const compareObjects = (a, b) => a.value - b.value; | ||
|
|
||
| group('RBTree insert operations', () => { | ||
| bench('insert 10 sequential values', () => { | ||
| const tree = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 10; i++) { | ||
| tree.insert(i); | ||
| } | ||
| }); | ||
|
|
||
| bench('insert 100 sequential values', () => { | ||
| const tree = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 100; i++) { | ||
| tree.insert(i); | ||
| } | ||
| }); | ||
|
|
||
| bench('insert 1000 sequential values', () => { | ||
| const tree = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 1000; i++) { | ||
| tree.insert(i); | ||
| } | ||
| }); | ||
|
|
||
| bench('insert 100 random values', () => { | ||
| const tree = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 100; i++) { | ||
| tree.insert(Math.random() * 10000); | ||
| } | ||
| }); | ||
|
|
||
| bench('insert 1000 random values', () => { | ||
| const tree = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 1000; i++) { | ||
| tree.insert(Math.random() * 10000); | ||
| } | ||
| }); | ||
|
|
||
| bench('insert 100 reverse sequential values', () => { | ||
| const tree = new RBTree(compareNumbers); | ||
| for (let i = 99; i >= 0; i--) { | ||
| tree.insert(i); | ||
| } | ||
| }); | ||
| }); | ||
|
|
||
| group('RBTree find operations', () => { | ||
| const tree100 = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 100; i++) { | ||
| tree100.insert(i); | ||
| } | ||
|
|
||
| const tree1000 = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 1000; i++) { | ||
| tree1000.insert(i); | ||
| } | ||
|
|
||
| const tree10000 = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 10000; i++) { | ||
| tree10000.insert(i); | ||
| } | ||
|
|
||
| bench('find in tree with 100 values', () => { | ||
| tree100.find(50); | ||
| }); | ||
|
|
||
| bench('find in tree with 1000 values', () => { | ||
| tree1000.find(500); | ||
| }); | ||
|
|
||
| bench('find in tree with 10000 values', () => { | ||
| tree10000.find(5000); | ||
| }); | ||
|
|
||
| bench('find non-existent in tree with 1000 values', () => { | ||
| tree1000.find(-1); | ||
| }); | ||
| }); | ||
|
|
||
| group('RBTree min/max operations', () => { | ||
| const tree100 = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 100; i++) { | ||
| tree100.insert(Math.random() * 1000); | ||
| } | ||
|
|
||
| const tree1000 = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 1000; i++) { | ||
| tree1000.insert(Math.random() * 1000); | ||
| } | ||
|
|
||
| bench('min with 100 values', () => { | ||
| tree100.min(); | ||
| }); | ||
|
|
||
| bench('max with 100 values', () => { | ||
| tree100.max(); | ||
| }); | ||
|
|
||
| bench('min with 1000 values', () => { | ||
| tree1000.min(); | ||
| }); | ||
|
|
||
| bench('max with 1000 values', () => { | ||
| tree1000.max(); | ||
| }); | ||
| }); | ||
|
|
||
| group('RBTree iteration operations', () => { | ||
| const tree100 = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 100; i++) { | ||
| tree100.insert(Math.random() * 1000); | ||
| } | ||
|
|
||
| const tree1000 = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 1000; i++) { | ||
| tree1000.insert(Math.random() * 1000); | ||
| } | ||
|
|
||
| bench('iterate all 100 values with each()', () => { | ||
| tree100.each(() => {}); | ||
| }); | ||
|
|
||
| bench('iterate all 1000 values with each()', () => { | ||
| tree1000.each(() => {}); | ||
| }); | ||
|
|
||
| bench('iterate 10 values with iterator', () => { | ||
| const iter = tree1000.iterator(); | ||
| for (let i = 0; i < 10; i++) { | ||
| iter.next(); | ||
| } | ||
| }); | ||
| }); | ||
|
|
||
| group('RBTree lowerBound/upperBound operations', () => { | ||
| const tree = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 1000; i++) { | ||
| tree.insert(i * 2); // Even numbers only | ||
| } | ||
|
|
||
| bench('lowerBound exact match', () => { | ||
| tree.lowerBound(500); | ||
| }); | ||
|
|
||
| bench('lowerBound between values', () => { | ||
| tree.lowerBound(501); | ||
| }); | ||
|
|
||
| bench('upperBound', () => { | ||
| tree.upperBound(500); | ||
| }); | ||
| }); | ||
|
|
||
| group('RBTree remove operations', () => { | ||
| bench('insert and remove 100 values', () => { | ||
| const tree = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 100; i++) { | ||
| tree.insert(i); | ||
| } | ||
| for (let i = 0; i < 100; i++) { | ||
| tree.remove(i); | ||
| } | ||
| }); | ||
|
|
||
| bench('insert 100, remove 50', () => { | ||
| const tree = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 100; i++) { | ||
| tree.insert(i); | ||
| } | ||
| for (let i = 0; i < 50; i++) { | ||
| tree.remove(i); | ||
| } | ||
| }); | ||
|
|
||
| bench('remove from middle', () => { | ||
| const tree = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 100; i++) { | ||
| tree.insert(i); | ||
| } | ||
| tree.remove(50); | ||
| }); | ||
| }); | ||
|
|
||
| group('RBTree with complex objects', () => { | ||
| bench('insert 100 objects', () => { | ||
| const tree = new RBTree(compareObjects); | ||
| for (let i = 0; i < 100; i++) { | ||
| tree.insert({ value: i, data: `item${i}` }); | ||
| } | ||
| }); | ||
|
|
||
| bench('find in tree with 100 objects', () => { | ||
| const tree = new RBTree(compareObjects); | ||
| for (let i = 0; i < 100; i++) { | ||
| tree.insert({ value: i, data: `item${i}` }); | ||
| } | ||
| tree.find({ value: 50 }); | ||
| }); | ||
| }); | ||
|
|
||
| group('RBTree clear operation', () => { | ||
| bench('clear tree with 100 values', () => { | ||
| const tree = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 100; i++) { | ||
| tree.insert(i); | ||
| } | ||
| tree.clear(); | ||
| }); | ||
|
|
||
| bench('clear tree with 1000 values', () => { | ||
| const tree = new RBTree(compareNumbers); | ||
| for (let i = 0; i < 1000; i++) { | ||
| tree.insert(i); | ||
| } | ||
| tree.clear(); | ||
| }); | ||
| }); | ||
|
|
||
| run(); | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had too many other PRs outstanding so I didn’t post this yet, but I have a PR to replace benchmark-regressions with a workalike that wraps bench-node, which is maintained by a NodeJS core contributor. If I can get him to land one more PR I can clean up some junk in the output and then file that here, clearing the dead deps.