Workshopper testing

[llkats]

Let's discuss setting up a CI-type solution to test workshoppers comprehensively before release.

jk sorta, please see this comment below.

[martinheidegger]

I am not sure CI is possible. The issue that usually occurs is that on windows the tty doesn't work correctly. Testing it would mean to actually have a tty tool (that has different timings) and that is executed (per hand?) to make sure that the tty works.

Workshops can have other kinds of tests. This test ensures all the recommended solutions pass: https://github.com/workshopper/browserify-adventure/blob/master/test/solutions.js

[llkats]

I should try this again, since I misunderstood the initial case that @martinheidegger meant. Is there a way to test workshoppers against new and upcoming versions of node/npm? For example, TTY no longer working in Windows.

[martinheidegger]

@substack there is workshopper-adventure-test that should do that even further. But those are automated tests.

The real problem, that we can't tackle easily, are integration tests. Depending on Node-version there are various things that can fail. i.e.:

• TTY bugs (the interaction with the terminal is not-working, sometimes-working, `conditionally working ー this is by far the most annoying and difficult issue to deal with ー; also, the control characters could mingled resulting in wrong output)
• Colors displayed in the terminals with too low contrast ("black on black", etc.)
• Interaction might be "slow", i.e. the cursors work but there is a long delay
• The installation of npm packages doesn't work properly.
• The environment paths after installing Node.js are not set properly (Please can't simple run npm i sometimes on windows)
• File System operation bugs (problems loading files with delay, etc.)
• Home environment variable lookup might change or break (i.e. where the storage is being kept)
• If the encoding handshake goes wrong it might be resulting in scrambled international characters.

So: In short: I think the tests that need interaction with other parties are of a nature to prevent bugs like above. I am not sure how many of those can be well automated considering that this my test matrix:

Platform	Win 7	Win 8	Win 10	Mac	Linux
Terminal				☑️	☑️
iTerm2				☑️
Hyperterm	☑️	☑️	☑️	☑️	☑️
Command Prompt	☑️	☑️	☑️
Powershell	☑️	☑️	☑️
Cygwin	☑️	☑️	☑️
Github Shell	☑️	☑️	☑️	☑️
Node Command Prompt	☑️	☑️	☑️
Cmder	☑️	☑️	☑️