Skip to content

Home

Jump to bottom
Giuliano Casale edited this page Jul 28, 2016 · 78 revisions

Enhancement Tools: FG Module (DICE-FG)

Introduction

The Filling-the-Gap (FG) module of the DICE Enhancement Tool is designed to achieve the following objectives:

  • Provide statistical estimation algorithms to infer resource consumption of an application.
  • Provide fitting algorithms to match monitoring data to parametric statistical distributions.
  • Annotate results in UML models.
  • Acquire data via JSON files, either user-specified or obtained from the DICE monitoring platforms (D-MON).

A high-level overview of the tool architecture is given in the figure below:

DICE-FG Architecture

The logical components of the DICE-FG tool are as follows:

  • DICE-FG Analyzer: Executes the statistical methods necessary to obtain the estimates of the performance models parameters, relying on the monitoring information available on the input files.
  • DICE-FG Actuator: Annotates in the UML model the parameters of the models, e.g., resource demands, think times, etc., which are obtained from the DICE-FG Analyzer.

Prerequisites

Binary distribution:

  • Windows 10, Ubuntu 12.04, or later.
  • Matlab Compiler Runtime (MCR) R2016a. This is a royalty-free runtime that does not require owning a Matlab license.

Source distribution:

  • MATLAB 2012a or later
  • MATLAB Statistics and Machine Learning toolbox
  • MATLAB Optimization toolbox
  • JSONlab toolbox

Getting started

Check out DICE-FG and get into the main directory:

svn co https://github.com/dice-project/DICE-Enhancement-FG
cd DICE-Enhancement-FG/trunk/

The following command runs a demo of the estimation algorithms:

./bin/run_dicefg.sh MCR_FOLDER ./regression/test1/configuration_test_est.xml

where MCR_FOLDER needs to be replaced with the installation folder of the MCR. See bin\readme.txt for additional information.

Similarly, the following command runs a demo of the fitting routines:

./bin/run_dicefg.sh MCR_FOLDER ./regression/test1/configuration_test_fit.xml

Defining a new analysis

DICE-FG analyses are entirely specified using a XML configuration file provided as a command line parameter to run_dicefg.sh script. The general structure of the configuration file is as follows (call it config.xml)

<?xml version="1.0" encoding="utf-8"?>
<fg>   
    <configuration>...</configuration>
    <dataset>...</dataset>
    <analysis algorithm="..." flags="...">...</analysis>
	...
    <analysis algorithm="..." flags="...">...</analysis>
</fg>

where '...' needs to be replaced by an element tree. The configuration tree specifies the general FG configuration, the dataset tree specifies the input data, and each analysis tree(s) defines a run of FG with a specific estimation or fitting algorithm.

More details can be found in the Configuration XML Manual.

Once the XML configuration file is created, the run_dicefg.sh script can be run as shown in the previous section to carry out the FG analysis. For example, if the configuration is stored in config.xml we can simply type

./bin/run_dicefg.sh MCR_FOLDER config.xml

Creating an input data file

Input data files are specified either using JSON files (.json extension) or MATLAB's files (.mat extension). Two kind of datasets can be provided in input to FG:

  • Resource dataset: measurements associated to a given resource, e.g., utilization at VM 1.
  • System dataset: measurements for a collection of resources, e.g., end-to-end latency in a distributed application, aggregate number of jobs in two servers.

The easiest way to specify each dataset is a MATLAB cell array. However, users without a MATLAB license can either use GNU octave or specify the dataset using JSON. A detailed description of the requires data structure is provided in the Input Data Manual.

Contact

Maintainer: Giuliano Casale, Imperial College London: g.casale AT imperial.ac.uk

Clone this wiki locally