🚀 Feature: Replace canonical representations used in diffs with something like util.format()

[ThisIsMissEm]

Prerequisites

• Checked that your issue hasn't already been filed by cross-referencing issues with the faq label
• Checked next-gen ES issues and syntax problems by using the same environment and/or transpiler configuration without Mocha to ensure it isn't just a feature that actually isn't supported in the environment in question or a bug in your code.
• 'Smoke tested' the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, your usage of Mocha, or Mocha itself
• Ensured that there is no discrepancy between the locally and globally installed versions of Mocha. You can find them with: node node_modules/.bin/mocha --version(Local) and mocha --version(Global). We recommend that you not install Mocha globally.

Description

Due to how mocha generates the diffs of action vs expected, the results can be very confusing and misleading. There's a big difference between an array that contains undefined and the value undefined — that is, [undefined] vs undefined.

I originally thought this was an issue in Chai, but it turns out to be in Mocha:

• expect().to.deep.equal fails to handle arrays vs undefined chaijs/chai#1356
• https://github.com/mochajs/mocha/blob/v8.0.1/lib/utils.js#L307

Steps to Reproduce

Given an assertion of:

expect({
  scopes: []
}).to.deep.equal({
  scopes: undefined
})

Expected behavior: [What you expect to happen]

Output should be:

  -  "scopes": []
  +  "scopes": undefined

Actual behavior: [What actually happens]

  -  "scopes": []
  +  "scopes": [undefined]

Reproduces how often: [What percentage of the time does it reproduce?]

Every time.

Versions

• The output of mocha --version and node node_modules/.bin/mocha --version: 6.2.0, but the problem exists in v8.0.1 too
• The output of node --version: 12.8.x
• Your operating system
  • name and version: n/a
  • architecture (32 or 64-bit): n/a
• Your shell (e.g., bash, zsh, PowerShell, cmd): n/a
• Your browser and version (if running browser tests): n/a
• Any third-party Mocha-related modules (and their versions): n/a
• Any code transpiler (e.g., TypeScript, CoffeeScript, Babel) being used (and its version): n/a

Additional Information

This is due to Mocha's internal stringify function using square brackets to denote different types: https://github.com/mochajs/mocha/blob/v8.0.1/lib/utils.js#L303-L307

It looks okay when you're just doing a diff between undefined and something else, where the output will be:

- "something"
+ [undefined]

But because the stringify function is used recursively inside objects, it leads to confusing output.

[boneskull]

I agree it's weird. Function-typed values, null, undefined, Date, Buffer, and circular references ([Circular]) all display this way. Strings use double-quotes, but numbers (including Infinity) and booleans are bare.

A quick fix would just be to remove these brackets. But also I've never really been happy with how Mocha outputs representations.

It occurs to me that Node.js' util.format() using %o or %O would cover most or all of the bases here. For example, in Mocha, a Date is displayed as [Date: 1970-01-01T00:00:00.000Z]. util.format() displays it as 1970-01-01T00:00:00.000Z (which is not the result of Date.prototype.toString() nor Date.prototype.toLocaleString(); I have no idea where this comes from). This seems OK to me, because 1970-01-01T00:00:00.000Z is not in quotes; therefore it is not a string (but this may be confusing).

In Mocha, a Buffer displays as essentially the output of Array.from(someBuffer) but wrapped in [] and prefixed with Buffer:, e.g.:

let buf = Buffer.from([0x01, 0x02]);

mochaStringify(buf) // [Buffer: [1, 2]]

util.format(buf) // <Buffer 01 02>

The main thing we want to avoid is this: a failing comparison assertion must not show the expected and actual values as identical. If your assert.ok(foo === bar) fails, Mocha must display foo and bar side-by-side in such a way that the differences are obvious. The other thing is the output must be for humans.

However, util.format also takes advantage of ANSI colors, bold text, etc, which is inappropriate for diffs. There's also probably umpteen-billion JS object diff libs out there. Standards probably exist, but I'm guessing they would be aimed at machines instead of people.

[boneskull]

Anyhow, if someone from the community knows of a good library, or wants to propose a util.format-based solution, that'd be cool. Any changes here should be considered semver-major

[ThisIsMissEm]

A very light touch approach would be to just change the symbols wrapping, e.g., « and » instead of [ and ]

(Not using <> because jsx is a thing)

[boneskull]

may not be displayable on some windows terminal

[arvidOtt]

@boneskull if you pass something to util.format which is not a format string and not a string itself it uses util.inspect to print it.

And it is possible to configure the output when directly using util.inspect or util.formatWithOptions( since v10.0.0) and passing the inspectOptions parameter (e.g. ANSI colors is disabled by default). So I would propose trying solve this with a configured util.inspect.

https://nodejs.org/api/util.html#util_util_inspect_object_showhidden_depth_colors
https://github.com/nodejs/node/blob/master/lib/internal/util/inspect.js#L273

[boneskull]

@ThisIsMissEm Given that changing the output here would be a breaking change, I'd rather swap out all of it with something better rather than just tweak this one case.

[fluggo]

While you're at it, make sure the new formatter supports Map objects. Currently they always display as {}.

A quick test shows:

> console.log(util.inspect(new Map([['thing', 2], ['place', 'five']]), {compact: false, sorted: true}))
Map(2) {
  'place' => 'five',
  'thing' => 2
}

If util.inspect() works, I guess the question is what's to be done for the browser? The implementation is pretty involved.

Edit: I have now monkeypatched Mocha to show better diffs:

const mochaUtils = require('mocha/lib/utils');
import { inspect } from 'util';
mochaUtils.stringify = (val: unknown) => inspect(val, {compact: false, sorted: true});