Tester improvements (refactor, report, arbitrary validators) (#163)

- feature: eucalypt's test mode (`eu -T`) now generates an HTML test report including results and stdout / stdin of the test execution

- feature: HTML test reports generated by `eu -T` can now be automatically opened in a browser using the `-O` flag

- fix: error message fix - no longar panics on invalid yaml or json

- feature: The experimental test mode now supports arbitrary validations applied to the output. Single argument functions in the test subject file can be listed in the `:validations` key of the test target metadata. These run against the evidence data generated by the test and have access to `stdout`, `stderr`, `result`, `stats`

Author: gmorpheme

Cargo.lock

@@ -38,6 +38,8 @@ toml = { version = "0.5.8", features = ["preserve_order"] }
 dirs = "3.0.2"
 html5ever = "0.25.1"
 lru = "0.6.5"
+uuid = { version = "0.8.2", features = ["serde", "v4"] }
+webbrowser = "0.5.5"
 
 [[bench]]
 harness = false

Cargo.toml

@@ -63,7 +63,7 @@ new-version: [version.major,
 
 ` {target: :build-meta
    doc: "build metadata to embed as resource in binary"}
-main: {
+build-meta: {
   version: new-version
   commit: build.commit
   url: build.url

Eufile

@@ -42,6 +42,7 @@ tests: {
     t5: ({ k: :k } lookup(:k)) = (:k lookup-in({ k: :k }))
     t6: ({ k: :k } lookup-or(:x, 1234)) = lookup-or-in({ k: :k }, :x, 1234)
     t7: ({ c: 22 } lookup-alts([:a, :b, :c], 0)) = ({ a: 22 c: 18 } lookup-alts([:a, :b, :c], 0))
+    t8: [{}, {}, {foo: 66}] lookup-across(:foo, 77) //= 66
 
     pass: [t1, t2, t3, t4, t5, t6, t7] all-true?
   }

harness/test/010_prelude.eu

@@ -0,0 +1,35 @@
+{
+  title: "049 tester"
+  test-formats: [:yaml, :json]
+}
+
+(x ⊕ y): x + y
+
+` { target: :'test-⊕'
+    doc: "`⊕` correctly adds two numbers"
+    verify: [:all-values-true, :beta-specifically-is-true] }
+tests: {
+  α: 2 ⊕ 3 //= 5
+  β: -1 ⊕ -1 //= -2
+}
+
+` "all equalities evaluate to true"
+all-values-true(context): context.result values all-true? then(:PASS, :FAIL)
+
+` "beta calculation evaluates to true"
+beta-specifically-is-true(context): context.result.β then(:PASS, :FAIL)
+
+` { target: :test-fail
+    verify: [:all-values-false] }
+failures: {
+  α: 2 ⊕ 2 //= 5
+  β: 1 ⊕ 1 //= 3
+}
+
+` "all equalities evaluate to false"
+all-values-false(context): context.result values all(= false) then(:PASS, :FAIL)
+
+` { target: :test-default-expectation }
+test-default-expectation: {
+  RESULT: :PASS
+}

harness/test/049_tester.eu

@@ -1,4 +1,4 @@
-# Eucalypt standard prelude
+"eucalypt standard prelude"
 
 ##
 ## Prelude versioning and run metadata
@@ -128,9 +128,17 @@ lookup-or(s, d, b): __LOOKUPOR(s, d, b)
 ` "`lookup-or-in(b, s, d)` - look up symbol `s` in block `b`, default `d` if not found."
 lookup-or-in(b, s, d): lookup-or(s, d, b)
 
-` "`lookup-alts(syms, d)` - look up symbols `syms` in turn in block `b` until a value is found, default `d` if none."
+` "`lookup-alts(syms, d, b)` - look up symbols `syms` in turn in block `b` until a value is found, default `d` if none."
 lookup-alts(syms, d, b): foldr(lookup-or-in(b), d, syms)
 
+` "`lookup-in-alts(bs, s, d)` - look up symbol `s` in turn in each of blocks `bs` until a value is found, default `d` if none."
+lookup-across(s, d, bs): {
+  f(b, acc): lookup-or(s, acc, b)
+}.(foldr(f, d, bs))
+
+` "`lookup-path(ks, b)` - look up value at key path `ks` in block `b`"
+lookup-path(ks, b): foldl(lookup-in, b, ks)
+
 ##
 ## Boolean
 ##
@@ -607,13 +615,13 @@ reverse(l): foldl(cons flip, [], l)
 ` "`all-true?(l)` - true if and only if all items in list `l` are true."
 all-true?(l): foldl(and, true, l)
 
-` "`all-true?(l)` - true if and only if all items in list `l` satisfy predicate `p?`."
+` "`all(p?, l)` - true if and only if all items in list `l` satisfy predicate `p?`."
 all(p?, l): l map(p?) all-true?
 
 ` "`any-true?(l)` - true if and only if any items in list `l` are true."
 any-true?(l): foldr(or, false, l)
 
-` "`any-true?(l)` - true if and only if any items in list `l` satisfy predicate `p?`."
+` "`any(p?, l)` - true if and only if any items in list `l` satisfy predicate `p?`."
 any(p?, l): l map(p?) any-true?
 
 ` "`window(n, step, l)` - list of lists of sliding windows over list `l` of size `n` and offest `step`"