Skip to content

Add exhaustive testing with Ames dataset #11

Description

@njvack

the dataset in test/fixtures/data/ames_housing.csv has everything we could want. It's huge. It has weird columns and missing data.

We should set up a testsuite that does an exhaustive run through it and compare js and R, to make sure there aren't any places where we diverge a lot from R's results, and also that there's nowhere where our models straight-up crash.

So let's do:

For all models

  • Each numeric pair of numeric variables as X and Y

Additionally, for parametric models

  • For each X,Y pair, each other numeric variable as a nuisance
  • The same thing with one other (say, the middle) numeric variable as another nuisance

I believe this is something on the order of 70k test runs (40 numeric columns, so 40^3 plus a bit) so:

  • It should not be part of the routine test suite
  • We should parallelize running them

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions