Switch Tests to Library Tests

[guygastineau]

@kotp and I have discussed before the possibility of running our tests against functions from sourced libs instead of testing an executable script.

With the new mandate for stubs I think this is a good time to discuss whether or not we would prefer switching to this model.

---

Pros:

• Allow us to standardize the API of exercises.
• Allow easier static code analysis for the auto-mentor project (I've been meaning to work on that for us).
• Encourage students to modularize code into functions (perhaps leading to better bashers)
• Might make it easier for newcomers.

Cons:

• Robs autonomy from the students.
• More difficult to debug. Less savvy students might have trouble realizing how to write an interim implementation script that uses their lib, or they may not realize that they can source it into their interactive environment to play with their work. (That should be in docs or the readme's)
• it could make things slightly more complicated for some students.

---

Well, this is a big question/decision.

I welcome viewpoints from any and all of our maintainers, mentors, and students 😀

[glennj]

As well, functions can open up exercises that require some state to be kept between tests, such as List Operations.

[guygastineau]

Good point @glennj
I have wanted to hack that exercise into our track for a while. Been busy though ;)

Does that mean you are on board?

[kotp]

I would have thought that tests should be atomic, and not rely on "state" between tests.

[guygastineau]

I think he might mean state between sub tests of multiple functions. That is the kind of state that based flow we can't currently test with our model.

[guygastineau]

There would likely be setup and teardown functions for each of these series of subtests.

[guygastineau]

So, after #370 is merged I will make a demo PR for two-fer that treats it like a library.

This will prove that - and show how - it works.

@glennj and @kotp it seems like you all are in favor of this change being applied to the track.

[guygastineau]

@IsaacG I would like to know where you stand too.

We seem like the only 4 people who are currently active here, so I think it is completely safe to proceed if we are in agreement with each other.

[IsaacG]

So much for abstaining :)

I don't have a strong preference. bash isn't exactly build around APIs and importing functions feels a bit weird. Generally, in bash the only API you have is the execve() of the script. On the other hand, an API does make life loads easier to use and still provides a fairly similar API to an exec(). So I'm fine either way.

[guygastineau]

Thank you for weighing in @IsaacG

I agree that bash has a lot of flexibility. I often write simple scripts with no functions. Especially wrapper scripts (like my /usr/bin/vi is emacsclient -nw "$@")

Like @glennj pointed out, it should actually make previously impossible to-test-exercises viable for our track. Also, enforcing the "public" interface will allow us to write static code analysis programs much more easily. It's not like we are getting smashed in the mentor queues, but how many times should any of us have to good solutions to the simpler exercises.

Anyway, thanks again for sharing your opinion. I think the other two are on board, but the first PR will be like an example where everybody can weigh in how the test file should look and what-not; so I wont hand down mandates or anything like I'm Zeus 🤣

[guygastineau]

We might even be able to figure out how to do atomic benchmarks this way.