Add assertion for cpu and memory budget when testing via attributes

[nemo83]

What is your idea? Provide a use case.

Script costs execution are an important aspect of writing contracts on cardano.

In every validator script I've been working on, one of the phases was to optimise costs.

Currently tests already report on their costs, but, afaik, there is no way to assert against them.

Once optimised for resources consumption, it should be possible to ensure that further code modifications, won't unexpectedly blow up resources used during execution. This is even more relevant when migrating from a version to another of aiken. Even for aiken development itsels.

Something like rust attributes could work very nicely as adding assertion within the test could alter cpu/memory used.

Example:

#resources[eq{mem: 493718, cpu: 145378658}]
test marshal_for_connection_end_cost() {
  let bytes =
    marshal_for_connection_end(
      ConnectionEnd(
        #"0001",
        [Version(#"0101", [#"0201", #"0202"])],
        UNINITIALIZED,
        new_counterparty(#"0301", #"0401", MerklePrefix(#"0501")),
        10,
      ),
    )
  bytes == #"0a020001120c0a0201011202020112020202220e0a020301120204011a040a020501280a"
}

In the example the attribute requires resources to be equal to some specific memory and cpu. It could be nice to have to have option to chose between exact values or less than values.

Why is it a good idea?

Coz it allows devs to set strong budget constraints for execution costs of specific part of the code, and prevent changes to the code to blow up costs unexpectedly.

What is the current alternative and why is it not good enough?

Check costs in test reports breakdown and compare with previous ones.

[KtorZ]

Currently tests already report on their costs, but, afaik, there is no way to assert against them.

Note that a possible work-around here is to write your test outside of Aiken, leveraging the JSON output from aiken check to conduct assertions with whatever test runner you like. For example, in plain bash:

#!/bin/bash

set -e

# Declare tests to verify here with format "<title>:<max mem>:<max cpu>".
suite=(
  "verify_bitcoin_block_845999:188451:58800269"
  "insert_bitcoin_block_845602:188451:58800269"
  "example_kumquat:0:0"
)

ko="    \033[1;31mx\033[0m"
ok="    \033[1;32m✓\033[0m"

for scenario in ${suite[@]}; do
  scenario=(${scenario//:/ })
  title=${scenario[0]}
  echo -e "\033[1;95m$title\033[0m"
  result=$(aiken check -m $title 2>/dev/null | jq ".modules[].tests[] | select(.title == \"$title\")")
  status=$(jq -r '.status' <<< $result)
  if [ $status == "pass" ]
  then
      mem=$(jq -r '.execution_units.mem' <<< $result)
      cpu=$(jq -r '.execution_units.cpu' <<< $result)
      if [ $mem -gt ${scenario[1]} ]
      then
        echo -e "$ko mem is too large (actual: \033[31m$mem\033[0m, maximum: \033[32m${scenario[1]}\033[0m)"
      fi

      if [ $cpu -gt ${scenario[2]} ]
      then
        echo -e "$ko cpu is too large (actual: \033[31m$cpu\033[0m, maximum: \033[32m${scenario[2]}\033[0m)"
      fi

      if [ $mem -le ${scenario[1]} -a $cpu -le ${scenario[2]} ]
      then
        echo -e "$ok mem: $mem, cpu: $cpu"
      fi
  else
      echo -e "$ko test execution failed"
  fi
done

[nemo83]

Thank you very much for your answer, didn't know this was possible.

It's good to see that there are alternatives and that asserting against cpu/mem costs is an actual necessity.

It would be nice to have a more straightforward way of doing it.