Skip to content

Home

Jump to bottom
Giuliano Casale edited this page Jul 27, 2016 · 78 revisions

Enhancement Tools: FG Module (DICE-FG)

Introduction

The Filling-the-Gap (FG) module of the DICE Enhancement Tool is designed to achieve the following objectives:

  • Provide statistical estimation algorithms to infer resource consumption of an application.
  • Provide fitting algorithms to match monitoring data to parametric statistics distributions.
  • Use the above algorithms to parameterize UML models annotated with the DICE profile.
  • Acquire data via JSON and the DICE monitoring platforms (DMON).
  • Provide support for parallel estimation, in order to reduce the computational cost of inference.

A high-level overview of the tool architecture is given in the figure below:

The logical components of the DICE-FG tool are the Analyzer and the Actuator:

  • DICE-FG Analyzer: The DICE-FG Analyzer executes the statistical methods necessary to obtain the estimates of the performance models parameters, relying on the monitoring information available on the input files.
  • DICE-FG Actuator: The DICE-FG Actuator updates in the UML model the parameters of the models, e.g., resource demands, think times, etc., which are obtained from the DICE-FG Analyzer.

Prerequisites

  • Windows or Linux. (DICE-FG has been tested on Windows 10 and Ubuntu 12.04.)
  • Matlab Compiler Runtime (MCR) R2016a. This is a royalty-free runtime that does not require owning a Matlab license.

For users interested in running the source scripts within MATLAB, additional requirements are:

  • MATLAB Statistics and Machine Learning toolbox
  • JSONlab toolbox

Getting started

Check out DICE-FG and get into the main directory:

svn co https://github.com/dice-project/DICE-Enhancement-FG
cd DICE-Enhancement-FG/trunk/

We can launch a demo of the estimation routines using

./bin/run_dicefg.sh MCR_FOLDER ./regression/test1/configuration_test_est.xml

where MCR_FOLDER needs to be replaced with the installation folder of the MCR; more details can be found in the readme.txt file under the _bin_ folder.

Similarly, a demo for the fitting routines is run using

./bin/run_dicefg.sh MCR_FOLDER ./regression/test1/configuration_test_fit.xml

Defining a new analysis

DICE-FG analyses are entirely specified using a XML configuration file provided as an command line parameter to run_dicefg.sh script. The general structure of the configuration file is as follows (call it config.xml)

<?xml version="1.0" encoding="utf-8"?>
<fg>   
    <configuration>...</configuration>
    <dataset>...</dataset>
    <resource>...</resource>
</fg>

where '...' needs to be replaced by an element tree. A detailed description can be found in the Configuration XML Manual.

Once the XML configuration file is created, the run_dicefg.sh script can be run as shown in the previous section to carry out the FG analysis. For example, if the configuration is stored in config.xml we can simply type

./bin/run_dicefg.sh MCR_FOLDER config.xml

Creating an input data file

Input data files are specified either using JSON files (.json extension) or MATLAB's files (.mat extension). Two kind of datasets can be provided in input to FG:

  • Resource dataset: measurements associated to a given resource, e.g., utilization at VM 1.
  • System dataset: measurements for a collection of resources, e.g., end-to-end latency in a distributed application, aggregate number of jobs in two servers.

The easiest way to specify each dataset is a MATLAB cell array. However, users without a MATLAB license can either use GNU octave or specify the dataset using JSON. A detailed description of the requires data structure is provided in the Input Data Manual.

Contact

Maintainer: Giuliano Casale, Imperial College London: g.casale AT imperial.ac.uk

Clone this wiki locally