chore(ci): add telemetry-enabled perf task with local HTTPS server MONGOSH-3454 by nbbeeken · Pull Request #2784 · mongodb-js/mongosh

nbbeeken · 2026-07-22T22:07:59Z

No description provided.

Copilot

Pull request overview

Adds a telemetry-enabled performance CI task so Evergreen can produce side-by-side perf series (baseline vs telemetry-on) by routing telemetry to a locally launched HTTPS sink and validating that expected telemetry events were emitted.

Changes:

Adds a new Evergreen task perf_tests_linux_x64_telemetry and gates telemetry setup/verification in run_perf_tests via mongosh_perf_telemetry.
Introduces a minimal local HTTPS telemetry sink (.evergreen/telemetry-sink.js) that records received events to NDJSON for post-run assertions/logging.
Updates the perf smoke-test harness to reset cross-process telemetry throttling state between iterations.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
packages/cli-repl/src/smoke-tests.ts	Resets on-disk telemetry throttling state between perf iterations.
docs/superpowers/specs/2026-07-22-telemetry-perf-ci-task-design.md	Design doc describing the telemetry-enabled perf task approach and verification.
.evergreen/telemetry-sink.js	Adds a local HTTPS NDJSON-recording sink used by the telemetry perf task.
.evergreen/evergreen.yml.in	Adds telemetry-gated sink orchestration + verification and introduces the new perf task.
.evergreen.yml	Regenerated Evergreen config reflecting the new task and gated telemetry block.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

nbbeeken · 2026-07-23T03:32:09Z

test	baseline (s)	telemetry (s)	Δ
eval_nodb_print_plainvm	0.307	0.372	+21%
db_eval_plainvm	0.370	0.444	+20%
db_eval_repl	0.436	0.511	+17%
eval_nodb_print_repl	0.393	0.445	+13.5%
db_cursor_iteration_repl	1.164	1.282	+10%
async_rewrite_foreach	1.481	1.556	+5%
db_cursor_iteration_plainvm	1.298	1.346	+3.6%
db_repeat_command	2.177	2.196	+0.9%
mongosh_version	0.158	0.159	~0%
executable_size	198341600	198341600	identical

nbbeeken · 2026-07-23T03:47:32Z

      script: |
        mkdir -p npm-logs
-        tar cvzf npm-logs.tgz npm-logs
+        tar czf npm-logs.tgz npm-logs


removing the verbose flag here makes most conclusions visible without clicking through to one of the log viewers

nbbeeken · 2026-07-23T03:48:40Z

@@ -0,0 +1,36 @@
+#! /usr/bin/env bash
+
+# Fetch raw performance results for a given Evergreen version/patch from the


Makes comparisons easy

alenakhineika · 2026-07-23T18:23:59Z

-    - command: shell.exec
+          # Passed explicitly instead of add_expansions_to_env: the spawned
+          # mongosh processes inherit this environment, so we avoid handing
+          # every expansion (including secrets) to 160 child processes.


From where do we get this number? How do we know it is exactly 160 child processes?

Each scenario runs 20 times, so it actually should be 180 I didn't count mongodb_version, but also not necessary to mention this count, removed!

alenakhineika · 2026-07-23T18:29:25Z

+
+  process.on('SIGTERM', () => {
+    server.closeAllConnections?.();
+    server.close(() => process.exit(0));


Shouldn't we wait here for the write stream to actually finish flushing to disk?

alenakhineika · 2026-07-23T18:31:17Z

+  connection_count=$(grep -c '"event":"new-connection"' telemetry-sink-events.ldjson || true)
+  echo "identify=$identify_count new-connection=$connection_count"
+  [ "$identify_count" -ge 100 ]
+  [ "$connection_count" -ge 80 ]


Also these numbers, how do we decide on those?

I am just trying to see if anything arrived on the server, in case of misconfiguration I didn't want the perf tests to happily be green but nothing reached the server. I've lowered the barrier to pass by a ton, just 10 events need to show up. We have unit/int tests elsewhere to check this keeps working.

alenakhineika · 2026-07-23T18:33:12Z

-          fi
+        add_expansions_to_env: true
+        binary: bash
+        args: ["src/.evergreen/perf-send.sh"]


Where do we use perf-get.sh?

No where! That is for humans/agents to use locally, as long as you're on the corp vpn you can pull down perf resutls, makes it easier to browse and compare than using the perf baron ui

nbbeeken requested a review from a team as a code owner July 22, 2026 22:08

nbbeeken requested review from Copilot and esvm July 22, 2026 22:08

Copilot started reviewing on behalf of nbbeeken July 22, 2026 22:08 View session

Copilot AI reviewed Jul 22, 2026

View reviewed changes

Comment thread packages/cli-repl/src/smoke-tests.ts Outdated

Comment thread packages/cli-repl/src/smoke-tests.ts

Comment thread .evergreen/telemetry-sink.js Outdated

Comment thread .evergreen/evergreen.yml.in Outdated

nbbeeken marked this pull request as draft July 22, 2026 22:19

nbbeeken removed the request for review from esvm July 22, 2026 22:19

nbbeeken force-pushed the MONGOSH-3454-telemetry-perf branch 2 times, most recently from e9bd1f5 to fda2b62 Compare July 23, 2026 02:43

nbbeeken force-pushed the MONGOSH-3454-telemetry-perf branch from fda2b62 to 70130af Compare July 23, 2026 03:46

nbbeeken marked this pull request as ready for review July 23, 2026 03:46

nbbeeken requested a review from alenakhineika July 23, 2026 03:46

chore(perf): implement performance testing with telemetry

36e1e94

nbbeeken force-pushed the MONGOSH-3454-telemetry-perf branch from 70130af to 36e1e94 Compare July 23, 2026 03:49

nbbeeken commented Jul 23, 2026

View reviewed changes

nbbeeken changed the title ~~chore(ci): add telemetry-enabled perf task with local HTTPS sink MONGOSH-3454~~ chore(ci): add telemetry-enabled perf task with local HTTPS server MONGOSH-3454 Jul 23, 2026

alenakhineika reviewed Jul 23, 2026

View reviewed changes

fixes

ad2fb45

nbbeeken requested a review from alenakhineika July 23, 2026 19:20

alenakhineika approved these changes Jul 24, 2026

View reviewed changes

nbbeeken merged commit 01d5272 into main Jul 24, 2026
151 of 154 checks passed

nbbeeken deleted the MONGOSH-3454-telemetry-perf branch July 24, 2026 12:31

		@@ -0,0 +1,36 @@
		#! /usr/bin/env bash

		# Fetch raw performance results for a given Evergreen version/patch from the

Uh oh!

Conversation

nbbeeken commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nbbeeken commented Jul 23, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nbbeeken Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nbbeeken commented Jul 22, 2026 •

edited

Loading

nbbeeken Jul 23, 2026 •

edited

Loading