passing args to exercises

[timoxley]

As args to exercises are passed on cmdline,

Specifically, I wanted to pass functions I worked around this in functional-javascript-workshop with my own function serialisation and serializing some wrapper code which would re-construct any complex objects (e.g. objects with prototypes setup).

How do you envision such a thing could be integrated into workshopper?

Possible solutions:

1. Use something like mixu/snapshot to serialise functions etc + use wrapped exec for constructing complex objects. Downside here is an extra step is required if you want to add any random element to the complex objects, it has to read the random seeds from cmdline then use that to reconstruct the objects.
2. Use vm to sandbox

[rvagg]

Use https://github.com/rvagg/workshopper-wrappedexec/

Call exercise.wrapModule() to give it a full path to a module to require() prior to solution execution.

Set properties with exercise.wrapSet(key, value) and they get passed to the module if the module exports a function. There's serialisation that happens here but your module also gets to set properties back on that object and they are serialised back into the parent process so you can use them for validation.

What I'd actually recommend though is that instead of serialising functions you just inject them with your wrappedexec module. Put them in a module and pull them in.

[rvagg]

that's all very cryptic, I understand, I'll try and do up an example for you if you can't figure it out.

[timoxley]

@rvagg hm, ok, can you do an example of how you might setup something like:

https://github.com/timoxley/functional-javascript-workshop/blob/master/problems/hello_world/solution.js

[timoxley]

This is pretty basic, doesn't require serialisation of anything at this point, but I'm not exactly clear on how I'd go about requiring their exported module to run tests on it.

[rvagg]

Don't even use the exec and comparestdout processors. You have the ful path to their moule in this.args[0], perhaps require it and run tests on it directly, in process. Emit 'pass' and 'fail' events on exercise to show what you're testing and then return true or false to the processor callback. You'd only need to register a processor with exercise.addVerifyProcessor() to do the tests but you probably need to make something happen when they use run as well.

[timoxley]

Don't even use the exec and comparestdout processors.

Yep.

This is what I've got thus far, seems to work not bad.

(Goal is to call their function and solution with some lorem, and compare outputs.)

var lorem = require('lorem-ipsum')
var exercise = require('workshopper-exercise')()
var assert = require('assert')
var path = require('path')

exercise = exercise.use(require('workshopper-boilerplate'))
exercise.addBoilerplate(require.resolve('./boilerplate/hello_world.js'))

exercise.addSetup(function(mode, callback) {
  this.submission = this.args[0]
  if (!this.solution) this.solution = path.join(this.dir, './solution/index.js')

  this.submissionModule = require(process.cwd() + '/' + this.submission)
  this.solutionModule = require(this.solution)
  this.inputs = lorem().split(' ')
  process.nextTick(callback)
})

exercise.addProcessor(function(mode, callback) {
  this.inputs.forEach(function(input) {
    console.log(this.submissionModule(input))
  }, this)

  process.nextTick(callback)
})

exercise.addVerifyProcessor(function(mode, callback) {
  this.inputs.forEach(function(input) {
    assert.equal(this.submissionModule(input), this.solutionModule(input))
  }, this)

  process.nextTick(callback)
}) 

module.exports = exercise

[ajcrites]

I'm struggling with this (writing the exercise) that works with this solution: https://github.com/ajcrites/generators-adventure/blob/master/problems/on_thunks/solution.js

The solution is to export a thunk of fs.readFile. Using Workshopper 0.7 it was pretty simple, but things seem to have changed quite a bit. I'm using a similar solution to the one in the previous comment with addVerifyProcessor. This somewhat works, but there are problems -- specifically if the solution has errors or doesn't trigger the callback since it receives one.

Another thing is that the "actual" and "expected" output can't be compared since I'm using addVerifyProcessor. Ideally I would be able to do whatever I wanted with the solution module and compare the stdout output like before.

Using the old method: https://github.com/ajcrites/generators-adventure/blob/master/problems/on_thunks/setup.js

Attempt at current method: https://gist.github.com/ajcrites/3c420d86967b9a6776fb

[rvagg]

Check out learnyounode to see the comparestdout stuff in action, since you're not executing it at all you're not doing any comparison of output (which is completely optional now fwiw). The danger with directly invoking user supplied code is that it will often behave in unpredictable ways so you'll need to do things like try/catch and run parallel timers to check if your callback has been called or not. You have a lot of power to inspect their code but you have to be prepared for all sorts of crazy. Have a look at the make_it_modular exercise in learnyounode for one of the most complicated examples of this: https://github.com/rvagg/learnyounode/tree/master/exercises/make_it_modular

Note in this case I'm getting them to make a separate module file so it's not their submission file that it's verifying, we do a comparestdout but also do some inspection to find out the name of their module file and then require() that and step through it with verify.js. Note all the safety checks and the checking of every assumption, even things like "did they pass me an Array?", each time you do a check like this emit a 'pass' or 'fail' event on exercise and they'll get feedback about what they did right/wrong.

In your case it sounds like you don't need to be running comparestdout, you really just want to inspect their code. Do the require() stuff but move that down into the VerifyProcessor so you can provide feedback if the require() fails (remember it can throw). Don't take anything for granted, check everything they do to make sure they did what you ask, even typeof foo == 'function' and give them a 'pass' / 'fail' for that. The more feedback you get the fewer support requests you'll get for your workshopper because you give them the ability to narrow down where they are going wrong.

Feel free to point me at your work-in-progress code if you're needing help and you have it in a branch. I don't have a whole lot of time but I might be able to offer pointers.