Run tests with range of package verions #3
Description
Dart's guidelines for package versions recommend having an opened ended version no. for package dependencies:
e.g.
^1.0.0
When source is downloaded and pub get is run for the first time dart will obtain the most recent version of all packages available and then update the pubspec.lock file to reflect those versions.
Unless the developer runs pub upgrade they will continue to use those same versions.
The problem with this is that an end user and the test environment may well end up running with an entirely different set of packages to what the development and test environments reflect.
This case is exacerbated by the fact that new versions of dependency packages will be released after the package is released.
What really needs to happen is that tests are run with every combination of package version and whenever a new dependency is released the tests are re-run. This is probably not practical.
Note: it may be practical to have a process that monitors the release of dependency packages on pub dev and re-runs unit tests each time a new package is released.
The middle ground would be to run tests with the lowest package version and the highest package version for each package that are available at the time the unit tests run.
This would double the unit test run time but is probably acceptable and will identify problems at the extreme ends the range of package versions which will hopefully be fairly indicative of problems occuring in the in between versions.