Atlas native test runner

[elcritch]

Adds support for atlas test, atlas test --parallel, and atlas test tests/mytest.nim. Runs tests/t*.nim when no args are provided.

Note: no care is taken to separate test output in parallel mode. Use serial mode if you want serial output.

[Araq]

Why do we need this? It's beyond Atlas's scope and doesn't fit in IMO.

[elcritch]

Why do we need this? It's beyond Atlas's scope and doesn't fit in IMO.

It’s not beyond the original scope of Atlas. Narrowing down atlas’s scope has made sense IMO but not in this case.

This is needed so Atlas can function as a Nimble replacement since many repos rely on nimble test to run tests.

[elcritch]

Hmmm though I just remembered there is testament. I just recall testament kinda “being meh” for some reason. As in confusing to use. Maybe that was the koch thing.

Honestly that opinion is vague and could be because I don’t favor having to remember too many of randomly named tools. Pretty sure I’m not unique in that. I like namespacing like nim test which could call testament or similar.

Perhaps having Atlas just call Testament instead?

[elcritch]

On the note of a nimble replacement, IMO Atlas should also support build again as well. Most of that is because adding atlas install && atlas build is far easier for the mayfly types who just want to quickly test a project or see it run or pass tests. It’s a terrible habit, but to be fair there’s so many options nowadays that a slight bit of friction at that stage and people will write off a project. I have done it myself.

[elcritch]

See, I might be too dumb to use Testament: https://gist.github.com/elcritch/2cae045ce132dfd34c703b6653560d71

Ugh I'll have to read the docs instead of running my tests. :P

[Araq]

Testament is not good indeed, Nimony uses a new tool named "hastur" instead. So if Atlas provides a test command, it indeed makes sense to copy Nimble here.

The problem here is that Atlas's design says "it won't call Nim for you", it merely sets up a nim.cfg. atlas test and atlas doc and atlas build violate this.

If we treat Atlas as a Nimble replacement all these things make perfect sense, of course.

[elcritch]

The problem here is that Atlas's design says "it won't call Nim for you", it merely sets up a nim.cfg. atlas test and atlas doc and atlas build violate this.

True, and I do like that. I also want a simple way to run tests.

Perhaps adding a nim test to Nim itself? Adding it to Nim itself could make sense, but gets contaminates separation of concerns there as well.

[elcritch]

What if test was a sort of plugin? Essentially Atlas could support plugins for running cmake, or running nim tests, etc.

Then technically it's not Atlas invoking Nim, but another tool/plugin.

[Araq]

Maybe a separate atlasrun tool? (Part of this repo, of course.)

[mratsim]

The problem here is that Atlas's design says "it won't call Nim for you", it merely sets up a nim.cfg. atlas test and atlas doc and atlas build violate this.

If we treat Atlas as a Nimble replacement all these things make perfect sense, of course.

Having something that deals with all the devops is quite valuable, similar to go build (hiding cgo) or cargo build (hiding rustc).

[Araq]

Atlas is a package manager, not a build tool. If it becomes a build tool it needs to support every option that Nim supports, this can be done in some half-assed broken manner as Nimble does it or via a new --forward:switch:arg command line argument.

[elcritch]

Having something that deals with all the devops is quite valuable, similar to go build (hiding cgo) or cargo build (hiding rustc).

A separate tool like atlas-run or atlas-build could fill that role perhaps.

Though @mratsim it's surprisingly effective to add tasks like test or build in a top level config.nims. Then you can do nim test or nim build.

If it becomes a build tool it needs to support every option that Nim supports, this can be done in some half-assed broken manner as Nimble does it or via a new --forward:switch:arg command line argument.

A standard way in many programs is with a --. As in atlas-build test -- -d:release or some such. So much simpler than some --forward. Alternatively, atlas-build would just build and any args would be passed along.

[Araq]

atlas-build sounds very good to me.

[ryanwalklin]

Seconding an atlas build command, I've got a bunch of subprojects as atlas dependencies (Vulkan engine for a visualisation app I'm building) that themselves need to run steps (download fonts, build C libraries, generate Futhark FFI etc), that I'm currently duk-taping config.nims scripts together for, and a more robust build system would be awesome.