forked from diegosouzapw/OmniRoute
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathstryker.conf.json
More file actions
325 lines (325 loc) · 17.3 KB
/
Copy pathstryker.conf.json
File metadata and controls
325 lines (325 loc) · 17.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
{
"$schema": "https://stryker-mutator.io/schemas/stryker-schema.json",
"_comment": [
"Mutation testing for the ~8 critical modules (Task 11 — Fase 7).",
"NIGHTLY ONLY — DO NOT run on every PR. Mutation testing is expensive:",
" - Each mutant requires a full test suite execution.",
" - The 8 modules produce ~200–500 mutants; est. 30–90 min per run.",
" - Wired to the nightly CI workflow (.github/workflows/nightly-mutation.yml),",
" NOT to the 'lint' / 'quality-gate' PR jobs.",
"",
"TEST RUNNER — @stryker-mutator/tap-runner (NOT vitest):",
" The 8 critical modules are covered by node:test files in tests/unit/",
" (run via `node --import tsx --test`), NOT by vitest. The vitest config",
" only includes a small set of .test.tsx + open-sse/**/__tests__ files, so",
" the vitest-runner would find ZERO covering tests for these modules. The",
" tap-runner spawns each node:test file individually and parses its TAP",
" output, which matches how this repo actually exercises the modules.",
" Each test file is loaded with tsx (TypeScript/ESM) + the project polyfill,",
" the same way `npm run test:unit` runs.",
"",
"Install before running (not bundled to avoid E2E / CI bloat):",
" npm install --save-dev @stryker-mutator/core @stryker-mutator/tap-runner",
"",
"Run manually:",
" npm run test:mutation # full run (slow — nightly budget)",
" npx stryker run --dryRunOnly # validate the baseline only (no mutants)",
" (single-module probe: temporarily narrow `mutate` + `tap.testFiles` in this file)",
"",
"VALIDATED 2026-06-15: `npx stryker run --dryRunOnly` exits 0 — all 129 covering",
"test files run green in the Stryker sandbox and the perTest coverage map builds for",
"all 8 instrumented modules (15k+ mutants). The baseline dry-run takes ~20 min with",
"concurrency=1; the full mutation phase runs on top (advisory, capped by the workflow",
"timeout). So the nightly produces REAL mutation scores for the 8 modules.",
"",
"Mutation score per module → quality-baseline.json key 'mutationScore.<module>'",
"Direction: up (score can only improve; ratchet blocks drops — wired in a later INT phase)."
],
"packageManager": "npm",
"incremental": true,
"incrementalFile": "reports/mutation/stryker-incremental.json",
"testRunner": "tap",
"plugins": ["@stryker-mutator/tap-runner"],
"tap": {
"testFiles": [
"tests/unit/account-fallback-anthropic-quota.test.ts",
"tests/unit/account-fallback-route-restriction-403.test.ts",
"tests/unit/account-fallback-service.test.ts",
"tests/unit/api-key-rotator-health.test.ts",
"tests/unit/appearance-widget-settings-schema.test.ts",
"tests/unit/auth-clear-account-error.test.ts",
"tests/unit/auth-disable-cooling-2997.test.ts",
"tests/unit/auth-extract-api-key.test.ts",
"tests/unit/auth-noauth-fallback-loop-3061.test.ts",
"tests/unit/auth-ollama-cloud-per-model-403-3027.test.ts",
"tests/unit/auth-opencode-zen-noauth-fallback.test.ts",
"tests/unit/auth-terminal-status.test.ts",
"tests/unit/authz/routeGuard.test.ts",
"tests/unit/auto-combo-context-advertising.test.ts",
"tests/unit/auto-combo-engine.test.ts",
"tests/unit/auto-combo-scoring-clamp.test.ts",
"tests/unit/build/check-circular-deps.test.ts",
"tests/unit/cache-sweeps.test.ts",
"tests/unit/cc-compatible-provider.test.ts",
"tests/unit/chat-context-relay.test.ts",
"tests/unit/chat-cooldown-aware-retry.test.ts",
"tests/unit/chat-helpers.test.ts",
"tests/unit/chat-route-coverage.test.ts",
"tests/unit/chat-route-edge-cases.test.ts",
"tests/unit/chatcore-compression-integration.test.ts",
"tests/unit/chatcore-executor-helpers.test.ts",
"tests/unit/chatcore-extracted-modules-3821.test.ts",
"tests/unit/chatcore-headers.test.ts",
"tests/unit/chatcore-imports-cleanly.test.ts",
"tests/unit/chatcore-log-truncation.test.ts",
"tests/unit/chatcore-memory-extraction.test.ts",
"tests/unit/chatcore-memory-skills-injection.test.ts",
"tests/unit/chatcore-non-streaming-sse.test.ts",
"tests/unit/chatcore-passthrough-tool-names.test.ts",
"tests/unit/chatcore-sanitization.test.ts",
"tests/unit/chatcore-semantic-cache.test.ts",
"tests/unit/chatcore-strip-stale-headers.test.ts",
"tests/unit/chatcore-telemetry-helpers.test.ts",
"tests/unit/chatcore-translation-paths.test.ts",
"tests/unit/check-error-helper.test.ts",
"tests/unit/check-route-guard-membership.test.ts",
"tests/unit/check-test-discovery.test.ts",
"tests/unit/circuit-breaker-failure-kind.test.ts",
"tests/unit/claude-code-parity.test.ts",
"tests/unit/claude-effort-suffix-strip.test.ts",
"tests/unit/claude-oauth-provider.test.ts",
"tests/unit/claude-passthrough-stream-boolean.test.ts",
"tests/unit/claude-passthrough-thinking-2454.test.ts",
"tests/unit/cli-simulate.test.ts",
"tests/unit/codex-failover.test.ts",
"tests/unit/codex-stream-false.test.ts",
"tests/unit/collect-metrics-module-coverage.test.ts",
"tests/unit/combo-499-abort.test.ts",
"tests/unit/combo-auto-candidate-expansion.test.ts",
"tests/unit/combo-cache-invalidation.test.ts",
"tests/unit/combo-config.test.ts",
"tests/unit/combo-context-relay.test.ts",
"tests/unit/combo-health-autopilot.test.ts",
"tests/unit/combo-health-dashboard.test.ts",
"tests/unit/combo-health-route.test.ts",
"tests/unit/combo-hedging.test.ts",
"tests/unit/combo-max-depth-config.test.ts",
"tests/unit/combo-omnimodel-tag-stripping.test.ts",
"tests/unit/combo-prescreen.test.ts",
"tests/unit/combo-provider-cooldown.test.ts",
"tests/unit/combo-provider-diversity-wiring.test.ts",
"tests/unit/combo-quality-validator-reasoning.test.ts",
"tests/unit/combo-quota-soft-penalty.test.ts",
"tests/unit/combo-round-robin-streaming-lock-3811.test.ts",
"tests/unit/combo-routing-engine.test.ts",
"tests/unit/combo-scoring-inspector.test.ts",
"tests/unit/combo-sessionless-pin-3825.test.ts",
"tests/unit/combo-strategies.test.ts",
"tests/unit/combo-strategy-fallbacks.test.ts",
"tests/unit/combo-streaming-empty-content-failover.test.ts",
"tests/unit/combo-target-defensive-modelstr.test.ts",
"tests/unit/complexity-aware-scoring-wiring.test.ts",
"tests/unit/context-pinning-tool-calls.test.ts",
"tests/unit/correctness/combo.property.test.ts",
"tests/unit/correctness/sanitizers.property.test.ts",
"tests/unit/custom-model-target-format.test.ts",
"tests/unit/db-reset-module-state.test.ts",
"tests/unit/domain-persistence.test.ts",
"tests/unit/embeddings-auth.test.ts",
"tests/unit/error-classification.test.ts",
"tests/unit/error-message-sanitization.test.ts",
"tests/unit/executor-antigravity.test.ts",
"tests/unit/executor-web-cookie-sweep.test.ts",
"tests/unit/gemini-web-missing-browser-3516.test.ts",
"tests/unit/guardrails-api-3496.test.ts",
"tests/unit/memory-embedding-remote.test.ts",
"tests/unit/memory-embedding-transformers.test.ts",
"tests/unit/model-cooldowns-route-auth.test.ts",
"tests/unit/model-cooldowns-route.test.ts",
"tests/unit/model-lockout-decay.test.ts",
"tests/unit/oauth-providers-config.test.ts",
"tests/unit/oauth-redirect-uri-mismatch.test.ts",
"tests/unit/observability-fase04.test.ts",
"tests/unit/observability-payloads.test.ts",
"tests/unit/plan3-p0.test.ts",
"tests/unit/plugin-sandbox-permissions.test.ts",
"tests/unit/plugins-route-error-sanitization.test.ts",
"tests/unit/provider-error-rules.test.ts",
"tests/unit/provider-health-autopilot.test.ts",
"tests/unit/provider-health-matrix.test.ts",
"tests/unit/provider-request-failure-pipeline.test.ts",
"tests/unit/public-client-ids-3493.test.ts",
"tests/unit/publicCreds.test.ts",
"tests/unit/qoder-oauth-config.test.ts",
"tests/unit/quota-groups-route.test.ts",
"tests/unit/quota-key-models-route.test.ts",
"tests/unit/quota-policy-generalization.test.ts",
"tests/unit/quota-pool-log-route.test.ts",
"tests/unit/quota-streaming-consumption-usd.test.ts",
"tests/unit/rate-limit-enhanced.test.ts",
"tests/unit/rate-limit-manager.test.ts",
"tests/unit/responses-handler.test.ts",
"tests/unit/route-explainability.test.ts",
"tests/unit/route-guard-plugins-local-only.test.ts",
"tests/unit/route-guard-private-lan.test.ts",
"tests/unit/route-guard-provider-login-local-only.test.ts",
"tests/unit/router-strategies.test.ts",
"tests/unit/service-combo-metrics.test.ts",
"tests/unit/services-branch-hardening.test.ts",
"tests/unit/services/combo-metrics-memory.test.ts",
"tests/unit/settings/authz-bypass.test.ts",
"tests/unit/skip-provider-breaker-consumer-2743.test.ts",
"tests/unit/sse-auth.test.ts",
"tests/unit/strict-random-deck.test.ts",
"tests/unit/system-role-extraction.test.ts",
"tests/unit/t23-t24-fallback-resilience.test.ts",
"tests/unit/tag-routing.test.ts",
"tests/unit/thundering-herd.test.ts",
"tests/unit/token-refresh-race-comprehensive.test.ts",
"tests/unit/token-refresh-service.test.ts",
"tests/unit/tools-filter-anthropic-format.test.ts",
"tests/unit/usage-service-hardening.test.ts",
"tests/unit/validate-response-quality.test.ts"
],
"nodeArgs": [
"--import",
"tsx",
"--import",
"./open-sse/utils/setupPolyfill.ts",
"--import",
"./tests/_setup/isolateDataDir.ts",
"--test-reporter=tap",
"-r",
"{{hookFile}}",
"{{testFile}}"
]
},
"_mutate_godfiles_excluded_comment": [
"2026-06-18 (Onda 2 budget): chatCore.ts + combo.ts — the two god-files — were REMOVED",
"from `mutate`. They dominated ~2/3 of the ~15k mutants; the full 8-module run TIMED OUT",
"at the 180min nightly cap (run 27705123780: 16:47:33 -> killed 19:47:48 = exactly 180min;",
"the prior 120min scheduled run also timed out). #4078 made concurrency safe but the",
"tap-runner re-spawns a node process per test file PER MUTANT, so spawn cost dominates and",
"15k mutants does not fit.",
"",
"2026-06-18 (Onda 3 / Fase 9 T5 re-add): the combo.ts god-file was split into 11 small",
"leaf modules under open-sse/services/combo/ (PRs #4162/#4175/#4186/#4196/#4204). The",
"routing LOGIC that justified combo.ts being in `mutate` now lives in those leaves, so the",
"10 well-covered combo/* leaves are ADDED back to `mutate` here (comboStructure/autoStrategy/",
"validateQuality/shadowRouting/targetSorters/comboPredicates/rrState/comboData + the reset-aware",
"quota pair quotaScoring/quotaStrategies, added after #4204 (D7b) merged). They are covered by",
"the 24 combo-*.test.ts files already in tap.testFiles (quota by combo-prescreen/combo-config/",
"combo-strategy-fallbacks). types.ts is omitted (pure type declarations produce 0 mutants).",
"",
"chatCore/* leaves: a covering-test audit found 6 with direct unit coverage (batch g:",
"comboContextCache/idempotency/passthroughHelpers/responseHeaders/sanitization/upstreamTimeouts).",
"A follow-up then added DEDICATED unit tests for 6 more leaves (tests/unit/chatcore-headers,",
"-log-truncation, -memory-extraction, -non-streaming-sse, -passthrough-tool-names,",
"-executor-helpers — wired into tap.testFiles above) and added those leaves as batch h",
"(headers/logTruncation/memoryExtraction/nonStreamingSse/passthroughToolNames/executorHelpers).",
"A later follow-up added dedicated tests (NO mock.module — unavailable under the tap-runner; used",
"fetch-override + crafted inputs + temp-DATA_DIR) for telemetryHelpers (both fns, all branches) and",
"memorySkillsInjection (getSkillsProviderForFormat fully + injectMemoryAndSkills guards/empty-DB",
"path) and added them as batch i.",
"",
"The FINAL chatCore leaf, semanticCache.ts, was added to batch i once its cache-HIT block had a",
"fixture: chatcore-semantic-cache now SEEDS the real cache via setCachedResponse (the in-memory",
"store getCachedResponse checks first — no mock.module needed) under the exact signature",
"checkSemanticCache rebuilds, so the HIT branch runs end-to-end (status 200 / 'semantic' / 'HIT' /",
"the stream + content-type ternaries / the cost fallback / the side-effect calls all get killed).",
"ALL 15 chatCore leaves are now mutated.",
"",
"STILL EXCLUDED (follow-ups, NOT in `mutate` yet):",
" - combo.ts + chatCore.ts barrels: their handleComboChat/handleChatCore CORES were not",
" split (out of scope — Fase 3 ChatCoreContext refactor). The barrels are now thin-ish",
" but still large; keep excluded until the cores are split.",
"See project memory: Quality Gate v2 / Fase 9 (project-combo-split)."
],
"mutate": [
"open-sse/services/accountFallback.ts",
"src/sse/services/auth.ts",
"src/server/authz/routeGuard.ts",
"open-sse/utils/error.ts",
"open-sse/utils/publicCreds.ts",
"src/shared/utils/circuitBreaker.ts",
"open-sse/services/combo/comboStructure.ts",
"open-sse/services/combo/autoStrategy.ts",
"open-sse/services/combo/validateQuality.ts",
"open-sse/services/combo/shadowRouting.ts",
"open-sse/services/combo/targetSorters.ts",
"open-sse/services/combo/comboPredicates.ts",
"open-sse/services/combo/rrState.ts",
"open-sse/services/combo/comboData.ts",
"open-sse/services/combo/quotaScoring.ts",
"open-sse/services/combo/quotaStrategies.ts",
"open-sse/handlers/chatCore/comboContextCache.ts",
"open-sse/handlers/chatCore/idempotency.ts",
"open-sse/handlers/chatCore/passthroughHelpers.ts",
"open-sse/handlers/chatCore/responseHeaders.ts",
"open-sse/handlers/chatCore/sanitization.ts",
"open-sse/handlers/chatCore/upstreamTimeouts.ts",
"open-sse/handlers/chatCore/headers.ts",
"open-sse/handlers/chatCore/logTruncation.ts",
"open-sse/handlers/chatCore/memoryExtraction.ts",
"open-sse/handlers/chatCore/nonStreamingSse.ts",
"open-sse/handlers/chatCore/passthroughToolNames.ts",
"open-sse/handlers/chatCore/executorHelpers.ts",
"open-sse/handlers/chatCore/telemetryHelpers.ts",
"open-sse/handlers/chatCore/memorySkillsInjection.ts",
"open-sse/handlers/chatCore/semanticCache.ts"
],
"_ignorePatterns_comment": [
"ignorePatterns = files NOT copied into the Stryker sandbox. It does NOT scope",
"what gets mutated (that is the `mutate` array above). The test files MUST be",
"copied so the tap-runner can find covering tests, so DO NOT ignore tests/ here.",
"We only exclude heavy, mutation-irrelevant trees to keep sandbox creation fast:",
"build output, coverage, the huge docs/i18n translation tree, and other worktrees."
],
"ignorePatterns": [
".next",
"dist",
"dist-electron",
".build",
"coverage",
"playwright-report",
"test-results",
"reports",
"docs/i18n",
".worktrees",
".stryker-tmp"
],
"reporters": ["progress", "html", "json"],
"htmlReporter": {
"fileName": "reports/mutation/mutation.html"
},
"jsonReporter": {
"fileName": "reports/mutation/mutation.json"
},
"coverageAnalysis": "perTest",
"timeoutMS": 60000,
"timeoutFactor": 2.5,
"concurrency": 4,
"disableTypeChecks": true,
"checkers": [],
"thresholds": {
"high": 70,
"low": 50,
"break": null
},
"tempDirName": ".stryker-tmp",
"cleanTempDir": true,
"_tapTestFiles_comment": [
"tap.testFiles is the explicit set of node:test files that cover the 8 mutated",
"modules (union of files importing any of them), MINUS a few timing/heap/streaming-",
"sensitive integration tests that can flake under Stryker concurrent runners and",
"would break the required all-green baseline dry-run (e.g. body-timeout-integration,",
"heap-pressure, sse-heartbeat-integration, *-stream-readiness, chatcore-memory-pressure).",
"It is enumerated (not a broad glob) so the Stryker dry-run stays tractable for the",
"nightly budget — a glob over the full ~1300-file unit suite would make the per-test",
"dry-run take hours. coverageAnalysis:perTest then narrows which files run per mutant.",
"Regenerate the base union after adding/renaming covering tests, then re-prune flaky ones:",
" grep -rlE \"circuitBreaker|publicCreds|accountFallback|routeGuard|services/auth|chatCore|services/combo|utils/error|public-client|account-fallback|route-guard|circuit-breaker\" tests/unit --include=\"*.test.ts\" | sort -u"
],
"dryRunTimeoutMinutes": 30,
"_concurrency_comment": "concurrency=4 (was 1): the covering node:test files used to share SQLite/module state via the default DATA_DIR (~/.omniroute), so running them concurrently in the Stryker sandbox caused cross-file races that failed the all-green baseline. tap.nodeArgs now imports ./tests/_setup/isolateDataDir.ts, which gives each spawned test process its own temp DATA_DIR — eliminating the shared on-disk DB, so concurrency>1 is deterministic. A/B verified 2026-06-17: dry-run at concurrency=4 fails WITHOUT the isolation import (account-fallback-service tap exit 9) and passes WITH it. Raise further only if the runner has spare cores."
}