the dataset in test/fixtures/data/ames_housing.csv has everything we could want. It's huge. It has weird columns and missing data.
We should set up a testsuite that does an exhaustive run through it and compare js and R, to make sure there aren't any places where we diverge a lot from R's results, and also that there's nowhere where our models straight-up crash.
So let's do:
For all models
- Each numeric pair of numeric variables as X and Y
Additionally, for parametric models
- For each X,Y pair, each other numeric variable as a nuisance
- The same thing with one other (say, the middle) numeric variable as another nuisance
I believe this is something on the order of 70k test runs (40 numeric columns, so 40^3 plus a bit) so:
- It should not be part of the routine test suite
- We should parallelize running them
the dataset in test/fixtures/data/ames_housing.csv has everything we could want. It's huge. It has weird columns and missing data.
We should set up a testsuite that does an exhaustive run through it and compare js and R, to make sure there aren't any places where we diverge a lot from R's results, and also that there's nowhere where our models straight-up crash.
So let's do:
For all models
Additionally, for parametric models
I believe this is something on the order of 70k test runs (40 numeric columns, so 40^3 plus a bit) so: