Build test for architectural impact on performance using testcases/benchmarks

[kthakore]

As we architecture new changes to the Rune runtime and codegen components we need to understand the tradeoffs.

We need to make some benchmark tests that help keep our eye on the impact of our architectural decisions.

[Michael-F-Bryan]

@kthakore, what sort of things are you looking for in a benchmark test? Can you provide a couple example use cases so we know what to code for?

[Michael-F-Bryan]

As discussed with @saidinesh5 on Slack and Google Meets, In particular, things we want to measure are:

• Building (rune build)
  • Build the noop Rune which just outputs the RAW data you give it (our "base" build time)
  • Build something fairly representative like the style_transfer Rime
  • Making a trivial change (e.g. adding an argument to a proc block) and rebuild to see what our incremental build times are like
• Runtime (rune run, but from Rust instead of the command-line)
  • Run the noop or sine runes to get our baseline overhead
  • Run something fairly representative like style_transfer or microspeech
    • As a Rune
    • In Rust by using the tflite crate directly (the theoretical fastest configuration)
    • In Python with the TensorFlow library (what people would normally do)