git-node: `git node bisect`

[joyeecheung]

https://twitter.com/targos89/status/982158028878196736

Outline:

• git node bisect --from 20180101 [--to 201810406] /path/to/test.js
• Detect the right arch, then download correct binaries from https://nodejs.org/download/nightly/ , decompress it, run the test with bin/node, bisect until it finds two consecutive nightlies where the exit code of the test becomes different
  • Let the user configure where the files are downloaded via ncu-config set tmpdir, default can be .ncu/tmp or /tmp
• Grab the sha from the nightlies, suggest the user to run:
  • git bisect start <nightly1-sha> <nightly2-sha> --
  • git bisect run "make -j8 && ./node /path/to/test.js"

[joyeecheung]

cc @targos

[joyeecheung]

Also we can be more generic about the mirror: e.g. allow other variants like https://nodejs.org/download/v8-canary/ and support mirrors like https://cnpmjs.org/mirrors/node-nightly/ (this one is much faster if you are in China). Probably a config with ncu-config would be enough.

[joyeecheung]

Also we may be more flexible about the command being run...for instance, if the test case must involve an addon, it should allow the user to specify the --node-dir or run whatever they need (e.g. some configuration command run during preinstall) to get the test case ready. Maybe provide an environment variable containing the path to the distribution to the command supplied by the user and let them script the test would be enough.

[mmarchini]

This would be useful to help find which V8 commit broke llnode (I just did a "manual bisect" with nvm and v8-canary nightlies while trying to find what broke llnode on V8 6.4, and it was really painful 😞).

I can work on it this weekend. Any suggestions on how to set the mirror and how to have a custom command? Maybe --mirror=nightly|v8-canary|url (nightly by default) and --script=/path/to/script.js | --command=[custom command]?

[joyeecheung]

@mmarchini --mirror/--script/--command SGTM. It would be nice to support them (at least mirror, probably makes less sense for --script/--command) in ncu-config so there is no need to type those things out every time the tool is used

[mmarchini]

It would also be interesting to bisect Node.js versions:

git node bisect --from v7.0.0 --to v8.0.0 /path/to/test.js

[watson]

In case anybody is interested, I've created this script that essentially does the same: https://gist.github.com/watson/493f359315e7542ddcad37b3408b9f5e

[github-actions]

This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.

[codebytere]

We could probably copy https://chromium.googlesource.com/chromium/src/+/master/tools/bisect-builds.py to some extent and make it a nice interactive flow

[github-actions]

This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.