TTC 2018: Case 1 "Quality-based Software-Selection and Hardware-Mapping as Model Transformation Problem"

Getting started

In order to get the case working, perform the following steps:

• Install GLPK to get the reference implementation working (otherwise the test, and therefore build, will fail)
  • Go to https://www.gnu.org/software/glpk/ (Linux) or http://winglpk.sourceforge.net/ (Windows) and follow the instructions
  • Tested with latest version 4.64 of GLPK
• Clone the repository: git clone https://git-st.inf.tu-dresden.de/stgroup/ttc18.git && cd ttc18
• Verify, that the path to GLPK is correct in gradle.properties (and change it, if necessary)
• Build it: ./gradlew build (or gradlew.bat build on Windows)
• Run the benchmark: ./gradlew benchmarkFull
  • As this might take long, running a set of scenarios is possible with ./gradlew benchmarkFull -Pscenario=0,small (comma separated list of ids and/or names)
  • Alternatively, the timeout can be set to a smaller value in jastadd-mquat-benchmark/src/main/resources/scenarios.json
  • Please do not alter the definition of the scenarios, instead use the Custom Benchmark

Overview over the repository structure

All modules are prefixed with jastadd-mquat, as this is an implementation of MQuAT (Multi-Quality AutoTuning) based on JastAdd. There are 5 modules:

• base: Contains the specifications for grammar and attributes, (de-)serializers and the model generator
• benchmark: Benchmark infrastructure and settings
• solver: Interfaces for solvers, and a small testsuite
• solver-ilp: Reference implementation using ILP
• solver-simple: Naïve, brute-force solver written in Java

Creating a solution

A new solution should be created using a new module (or multiple, if necessary). You can use the simple-solver module as an example. The following steps need to be completed:

1. Create an implementation of de.tudresden.inf.st.mquat.solving.BenchmarkableSolver (which extends the Solver interface). The main method here is public Solution solve(Root model) throws SolvingException, which takes a model as input an returns a solution
2. Add an include of your project to settings.gradle
3. Optional step: Create a test case by extending the HandwrittenTestSuite
4. Add a compile dependency to your project in build.gradle of the project jastadd-mquat-benchmark
5. Update [de.tudresden.inf.st.mquat.benchmark.SolverFactory.createAvailableSolversIfNeeded] (https://git-st.inf.tu-dresden.de/stgroup/ttc18/blob/master/jastadd-mquat-benchmark/src/main/java/de/tudresden/inf/st/mquat/benchmark/SolverFactory.java#L22) to create a new instance of your solver
6. Add the name of your solver to the benchmark settings
   • Use jastadd-mquat-benchmark/src/main/resources/scenarios.json for the Gradle task benchmarkFull
   • Use jastadd-mquat-benchmark/src/main/resources/local-benchmark-settings.json for the Gralde task benchmarkCustom (see Custom Benchmark for details)
7. Run the benchmark, either ./gradlew benchmarkFull or ./gradlew benchmarkCustom

Custom Benchmark

To test your solution, the Gradle task benchmarkCustom can be used. This task generates a custom set of models and runs a benchmark for them. All default parameters are specified in the file benchmark-settings.json within the directory jastadd-mquat-benchmark/src/main/resources. To change them, create a new file in this directory named local-benchmark-settings.json. In this local version, all parameter values override the default settings, but are ignored when committing.

To test your solver with the name fancy-solver along with the reference implementation using a model with 10 and 15 requests and a timeout of 50 seconds, the file local-benchmark-settings.json would be as follows.

{
  "solvers": [
    "ilp-direct",
    "fancy-solver"
  ],
  "basic": {
    "verbose": true,
    "minRequests": 10,
    "maxRequests": 15,
    "stepRequests": 5,
    "timeoutValue": 50,
    "timeoutUnit": "SECONDS",
    "total": 2
  }
}

The value total is used to constrain the total number of models to be generated. Set this to null (the default) to generate all value for the defined parameter ranges. Refer to de.tudresden.inf.st.mquat.generator.ScenarioDescription for a description of the possible parameters.

Notes and Troubleshooting

• Please use the gradle wrapper script, as different version of Gradle might not work with the setup
  • The wrapper script uses version 3.3
• If anything is not working as expected, feel free to contact on of the authors of the TTC case