Skip to content

Commit c731940

Browse files
authored
chore: use custom jest reporter to achieve sane jest logs (#8607)
The default jest log reporter is garbage. Webstorm replaces it with their own (which is pretty good), but vscode unfortunately uses the default one. This PR does the following to the jest reporter **1. Replace the default reporter with the jest-ci-spec-reporter reporter.** The default reporter is hiding console logs and incorrectly rewriting console history. Now, logs like these: ``` [20:56:16] INFO: ---- DROPPING DATABASE ---- [20:56:17] INFO: ---- DROPPED DATABASE ---- ``` will be visible again. The default reporter was showing them for half a second, then rewrites log history and hides them. **2. add custom logger to showcase hidden error messages** Some error messages are hidden and are only displayed at the end of the test, in a very ugly way. If the test hangs, you might have to wait a long time to see those errors. This PR makes sure that errors are logged when they were intended to be logged. They will not be printed in a pretty way (Webstorm for example prints them in red and clickable, like a proper error message) - but at least they will be printed instead of leaving you in the dark **Override console global in jest setup to hide console log spam** This turns logs like ``` console.log initPayloadInt done at log (helpers/initPayloadInt.ts:27:11) ``` into `initPayloadInt done` ## CI Yes, CI logs are actually usable now. We no longer have random console logs floating around! It was horrible! Finally, you can actually see which console logs belong to which test. Before: https://github.com/payloadcms/payload/actions/runs/11241674859/job/31253918825 After: https://github.com/payloadcms/payload/actions/runs/11242035327/job/31255031760?pr=8607 **BEFORE** ![CleanShot 2024-10-08 at 21 27 23@2x](https://github.com/user-attachments/assets/7c83ced7-b4fd-4e90-95ff-2c240829c3cd) What test triggered this "ValidationError: The following field is invalid: filteredRelation" error? Who knows!! Could have been any test. We will never know... **AFTER:** Finally, clarity ![CleanShot 2024-10-08 at 21 28 15@2x](https://github.com/user-attachments/assets/a259950e-3213-4faa-9f87-e54fd970f6dc) ## Screenshots - Passing database test suite ## Passing database test suite ### Before ![CleanShot 2024-10-08 at 21 07 39@2x](https://github.com/user-attachments/assets/00a07d30-fbeb-4a52-8982-3e0bc198e278) ![CleanShot 2024-10-08 at 21 08 05@2x](https://github.com/user-attachments/assets/0bc02982-83e4-4205-a1e9-0c0277390ab2) ### After ![CleanShot 2024-10-08 at 21 06 52@2x](https://github.com/user-attachments/assets/cd1e6ac1-17c0-4859-a374-2176e698784e) ## Screenshots - Failing test ### Before - that's where it hangs ![CleanShot 2024-10-08 at 21 09 52@2x](https://github.com/user-attachments/assets/088b1074-bd57-4d9d-8de4-81f1a5edf407) ### After - that's where it hangs Actually shows me the error without having to wait 3 minutes for test to timeout: ![CleanShot 2024-10-08 at 21 13 13@2x](https://github.com/user-attachments/assets/ec91f530-2f5e-4b6d-872a-f483b9a421f4) ### Before - after waiting for 3 minutes for test to timeout: ![CleanShot 2024-10-08 at 21 12 08@2x](https://github.com/user-attachments/assets/64ac9945-3a3c-4eb5-991c-943859500bb5) (1000 lines of same error spam...) ![CleanShot 2024-10-08 at 21 19 28@2x](https://github.com/user-attachments/assets/ccd33c38-f6d9-47a8-8c5a-41c118cfe849) ### After - after waiting for 3 minutes for test to timeout: ![CleanShot 2024-10-08 at 21 14 54@2x](https://github.com/user-attachments/assets/c2240305-21da-4b4c-9e28-ee68f8b2899d) ![CleanShot 2024-10-08 at 21 15 09@2x](https://github.com/user-attachments/assets/d6f7fab6-acd4-4bcc-a560-9e86792fdbbf) (Error spam) ![CleanShot 2024-10-08 at 21 15 20@2x](https://github.com/user-attachments/assets/6be43e88-f881-4598-bb32-d7cfc90ef710)
1 parent 21606de commit c731940

7 files changed

+133
-6
lines changed

Diff for: jest.config.js

+11-1
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,12 @@ const esModules = [
1515
'uint8array-extras',
1616
].join('|')
1717

18+
import path from 'path'
19+
import { fileURLToPath } from 'url'
20+
21+
const filename = fileURLToPath(import.meta.url)
22+
const dirname = path.dirname(filename)
23+
1824
/** @type {import('jest').Config} */
1925
const baseJestConfig = {
2026
extensionsToTreatAsEsm: ['.ts', '.tsx'],
@@ -36,10 +42,14 @@ const baseJestConfig = {
3642
'^.+\\.(t|j)sx?$': ['@swc/jest'],
3743
},
3844
verbose: true,
45+
reporters: [
46+
path.resolve(dirname, './test/jest-spec-reporter.cjs'),
47+
path.resolve(dirname, './test/jestreporter.cjs'),
48+
],
3949
}
4050

4151
if (process.env.CI) {
42-
baseJestConfig.reporters = [['github-actions', { silent: false }], 'summary']
52+
baseJestConfig.reporters.push(['github-actions', { silent: false }], 'summary')
4353
}
4454

4555
export default baseJestConfig

Diff for: pnpm-lock.yaml

+2
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Diff for: test/_community/payload-types.ts

+1-1
Original file line numberDiff line numberDiff line change
@@ -273,4 +273,4 @@ export interface Auth {
273273
declare module 'payload' {
274274
// @ts-ignore
275275
export interface GeneratedTypes extends Config {}
276-
}
276+
}

Diff for: test/jest-spec-reporter.cjs

+100
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
// From https://github.com/robertbradleyux/jest-ci-spec-reporter/blob/main/src/jest-ci-spec-reporter.ts
2+
/*
3+
MIT License
4+
5+
Copyright (c) 2023 Robert Bradley
6+
7+
Permission is hereby granted, free of charge, to any person obtaining a copy
8+
of this software and associated documentation files (the "Software"), to deal
9+
in the Software without restriction, including without limitation the rights
10+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
11+
copies of the Software, and to permit persons to whom the Software is
12+
furnished to do so, subject to the following conditions:
13+
14+
The above copyright notice and this permission notice shall be included in all
15+
copies or substantial portions of the Software.
16+
17+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
18+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
19+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
20+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
21+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
22+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
23+
SOFTWARE.
24+
*/
25+
'use strict'
26+
Object.defineProperty(exports, '__esModule', { value: true })
27+
class JestCiSpecReporter {
28+
onRunStart({ numTotalTestSuites }) {
29+
console.log()
30+
console.log(`Found ${numTotalTestSuites} test suites.`)
31+
console.log()
32+
}
33+
onRunComplete(_, results) {
34+
const {
35+
numFailedTests,
36+
numPassedTests,
37+
numPendingTests,
38+
testResults,
39+
numTotalTests,
40+
startTime,
41+
} = results
42+
testResults.forEach(({ failureMessage }) => {
43+
if (failureMessage) {
44+
console.log(failureMessage)
45+
}
46+
})
47+
const testResultText = numFailedTests === 0 ? 'SUCCESS' : 'FAILED'
48+
const numNotSkippedTests = numPassedTests + numFailedTests
49+
const runDuration = this._getRunDuration(startTime)
50+
console.log()
51+
console.log(
52+
`Executed ${numNotSkippedTests} of ${numTotalTests} (skipped ${numPendingTests}) ${testResultText} (${runDuration})`,
53+
)
54+
console.log(`TOTAL: ${numNotSkippedTests} ${testResultText}`)
55+
}
56+
onTestResult(test, { testResults }) {
57+
testResults.forEach((result) => {
58+
var _a, _b
59+
const { title, duration, status, ancestorTitles } = result
60+
const { name } =
61+
(_b = (_a = test.context.config) === null || _a === void 0 ? void 0 : _a.displayName) !==
62+
null && _b !== void 0
63+
? _b
64+
: {}
65+
if (name) {
66+
ancestorTitles.unshift(name)
67+
}
68+
const breadcrumbs = `${ancestorTitles.join(' > ')} >`
69+
70+
console.log(
71+
` ${this._getTestStatus(status)} ${breadcrumbs} ${title} ${this._getTestDuration(duration)}`,
72+
)
73+
})
74+
}
75+
getLastError() {
76+
return undefined
77+
}
78+
_getRunDuration(startTime) {
79+
const deltaInMillis = new Date().getTime() - new Date(startTime).getTime()
80+
const seconds = ((deltaInMillis % 60000) / 1000).toFixed(3)
81+
return `${seconds} secs`
82+
}
83+
_getTestDuration(duration) {
84+
return `\x1b[1m\x1b[30m(${duration !== null && duration !== void 0 ? duration : 0}ms)\x1b[0m`
85+
}
86+
_getTestStatus(status) {
87+
switch (status) {
88+
case 'passed':
89+
return '\x1b[1m\x1b[32m[PASS]\x1b[0m'
90+
case 'pending':
91+
return '\x1b[1m\x1b[33m[SKIP]\x1b[0m'
92+
case 'todo':
93+
return '\x1b[1m\x1b[34m[TODO]\x1b[0m'
94+
case 'failed':
95+
default:
96+
return '\x1b[1m\x1b[31m[FAIL]\x1b[0m'
97+
}
98+
}
99+
}
100+
exports.default = JestCiSpecReporter

Diff for: test/jest.config.js

-4
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,4 @@ const customJestConfig = {
2222
},
2323
}
2424

25-
if (process.env.CI) {
26-
customJestConfig.reporters = [['github-actions', { silent: false }], 'summary']
27-
}
28-
2925
export default customJestConfig

Diff for: test/jest.setup.js

+3
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
import console from 'console'
2+
global.console = console
3+
14
import { generateDatabaseAdapter } from './generateDatabaseAdapter.js'
25

36
process.env.PAYLOAD_DISABLE_ADMIN = 'true'

Diff for: test/jestreporter.cjs

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
class CustomReporter {
2+
constructor(globalConfig, reporterOptions, reporterContext) {
3+
this._globalConfig = globalConfig
4+
this._options = reporterOptions
5+
this._context = reporterContext
6+
}
7+
8+
onTestCaseResult(test, testCaseResult) {
9+
if (testCaseResult.status === 'passed') {
10+
return
11+
}
12+
console.log('Test case result:', testCaseResult)
13+
}
14+
}
15+
16+
module.exports = CustomReporter

0 commit comments

Comments
 (0)