Template for tests

[tasxatzial]

So far, tests are written and updated manually. In this issue, I'll be posting all the possible ways we've seen tests being written so far. I'll briefly mention the cons of each method and propose a template that could be used instead. Why do we need a template?

• To solve issues that arise from the inconsistency in writing tests.
• To create a semi-automatic way of writing and updating tests.
• To provide users with a consistent experience.

Method 1

(deftest empty-sentence
  (is (false? (pangram? ""))))

The main drawback of this method is that it's unclear what the test does. The name of the function can give a clue; however, function names cannot always be reliably used to describe a test, as they can often become overly long, making them hard to read:

(deftest does-not-detect-anagram-if-original-word-is-repeated
  (is (= [] (anagram/anagrams-for "go" ["goGoGO"]))))

Method 2

(deftest score-test
  (is (= 50 (yacht/score [5 5 5 5 5] "yacht")))
  (is (= 0 (yacht/score [1 3 3 2 5] "yacht"))))

The main problem here is that it shows as one test being run, when in fact it's two.

Method 3

(deftest empty-strands
  (testing "Empty strands"
    (is (= 0 (hamming/distance "" "")))))

This is arguably the best method since it allows us to use the description that comes from the problem specs, but it's still unclear how to choose an appropriate function name.

Also, test descriptions from problem specs are often very similar. It's incredibly easy to end up with duplicate names when someone reuses a previous test to create a new one, often through copy-pasting.

---

Taking all the above into account here's a proposed template:

(deftest test-2e6db04a-58a1-425d-ade8-ac30b5f318f3
  (testing "Test description"
    (is (= expected_value actual_value))))

with the following rules:

• One deftest per test from the problem specifications.
• The order of deftests follows the order of the corresponding tests in the problem specifications.
• The name of the function is the UUID of the test, prepended with test- or uuid-. This allows to quickly match an implemented test with the problem specs.
• The "test description" is the test description from the .toml file, with the first letter capitalized.
• The expected_value goes first, and the actual_value goes second. This is a common convention.
• If the expected_value is a collection, a vector is used.
• The actual_value should be a call to the tested function. If the function argument is a collection, a vector is used.

This isn't a perfect method—it still has some issues:

• Users can see the function name. This isn't a major issue, considering the existing function names are often not very useful.
• The test description may mention words like "lists," which can be confusing since the implementation uses vectors. This will be discussed in #671.

Now on to some examples:

(ns hamming-test
  (:require [clojure.test :refer [deftest testing is]]
            hamming))

(deftest test-54681314-eee2-439a-9db0-b0636c656156
  (testing "Single letter identical strands"
    (is (= 0 (hamming/distance "A" "A")))))

Note that the namespace is required without referring to any functions. This approach helps avoid issues where a function name might clash with a function name from clojure.core.

If the test line ends up being very long, splitting into multiple lines or using let is preferable:

(deftest test-54681314-eee2-439a-9db0-b0636c656156
  (testing "Create robot at origin facing north"
    (let [molly (robot-simulator/robot {:x 0 :y 0} :north)]
      (is (= {:bearing :north :coordinates {:x 0 :y 0}}
             molly)))))

I might edit this post later if I’ve missed anything.

[ErikSchierboom]

@tasxatzial I think we can close this now, right?

[tasxatzial]

I think we can close this now, right?

It's no longer strictly necessary, but I'd still like to gather everything about test templates into a new issue to document the decisions made when creating a template or updating tests manually.

[tasxatzial]

Once done, we can close #670, #671 and this one.