Mocha not running xState/test example because of test ordering.

[WarrenWilkinson]

Hi, I was running this from the documentation

describe('toggle', () => {
  const testPlans = toggleModel.getShortestPathPlans();

  testPlans.forEach((plan) => {
    describe(plan.description, () => {
      plan.paths.forEach((path) => {
        it(path.description, async () => {
          // do any setup, then...

          await path.test(page);
        });
      });
    });
  });

  it('should have full coverage', () => {
    return toggleModel.testCoverage();
  });
});

But my test coverage was always failing.. The problem appears to be that the "should have full coverage" test was run first, because the other tests are nested in a describe grouping and it is not.

I'm no expert on Javascript or Mocha, so maybe I'm doing it wrong, but if I comment out the describe block, or add a describe block around the full coverage test, then it successfully runs (all code coverage passes). Smoking gun in my book.

    "dependencies": {
        "auth0": "^2.31.0",
        "ejs": "^3.1.5",
        "express": "^4.17.1",
        "express-session": "^1.17.1",
        "mongoose": "^5.11.8",
        "passport": "^0.4.1",
        "passport-auth0": "^1.4.0",
        "xstate": "^4.15.2"
    },
    "devDependencies": {
        "@xstate/test": "^0.4.1",
        "@xstate/viz": "^0.3.4",
        "chai": "^4.2.0",
        "mocha": "^8.2.1",
        "nodemon": "^2.0.6"

[Andarist]

It seems like a Mocha quirk that it executes its before describes at the same level - if that's the case then it might be worth adding a describe block around that coverage test in the example.

[davidkpiano]

The coverage test is dependent on all the tests running... and if I recall correctly, Jest doesn't (yet) do dependent tests. You can make tests run in order via --runInBand (I think), but they say it's not recommended for that purpose.

I'm going to rethink this for the next version of @xstate/test. The simplest way to reliably check for coverage would be, in a single test, to run through all of the paths and then check coverage. This approach would also make using @xstate/test much simpler, basically a one-liner:

it('should reach all paths', async () => {
  // TENTATIVE FUTURE API
  await assertPaths(someTestModel.getShortestPaths());
  
  someTestModel.testCoverage();
});

The "downside" is that you won't get individual tests for each path tested, which only really matters in reporting. Maybe there is a way to programmatically hook into Jest's API for this.