Home

D-SPACE4Cloud Plugin

Welcome to the DICE Optimisation plugin wiki!

This repository contains the DICE Optimisation plugin, which is a component of the DICE Optimization Tool, so we will first give an overview of the whole project.

Optimization tool overview

D-SPACE4Cloud (DICE Optimization Tool) is the optimization tool integrated in the DICE IDE. It consists of three main components: an Eclipse Plug-in, a frontend and a backend service. Details on the overall architecture and on the internals of each component can be found in Section 4. Suffice to say that the Eclipse plug-in allows the ARCHITECT to specify the input models and performance constraints and transforms the input UML diagrams into the input performance models for the performance solver (GreatSPN, JMT or dagSim). The frontend exposes a graphical interface designed to facilitate the download of the optimization results (which are computed through batch jobs) while the backend implements a strategy aimed at identifying the minimum cost deployment.

Multiple DTSMs are provided as input, one for each VMs considered as a candidate deployment. VMs can be associated with different cloud providers. D-SPACE4Cloud will identify the VM type and the corresponding number which fulfill performance constraints and minimize costs. The tool takes as input also a DDSM model which is updated with the final solution found and can be automaticaly deployed through the DICER tool (see DICE Deliverable D2.4).

Moreover, the tool requires as input a description of the execution environment (list of providers, list of VM types or a description of the computational power available in house) and the performance constrains. Input files and parameters can be specified by the ARCHITECT through a wizard.

Usage scenarios

Two main analyses can be conducted using D-SPACE4Cloud:

• Public Cloud Analysis - In this scenario the ARCHITECT wants to analyze the case in which the whole Big Data cluster is provisioned on a public cloud. The first consequence of this choice is that the virtualized resources (i.e., VMs) that can be used to provision the cluster can be considered practically infinite for our purposes. This also means that, under the common assumption that rejecting a job has a much higher cost than the VM leasing costs, it will not apply any job rejection policy in this case. Consequently, the concurrency level for each job, i.e., the number of concurrent users running the job (see Section 4.1) can be set arbitrarily high being always (theoretically) possible to provision a cluster able to handle the load.
  In this scenario, the ARCHITECT may want to know which machine type to select and how many of them in order to execute the application with a certain level of concurrency, meaning considering several similar applications running at the same time in the cluster. She/he might also like to know which cloud provider is cheaper to choose, considering that providers have also different pricing models. For this reason, she/he has to feed the tool with the DTSM diagrams but also with a list of providers, and a list of VM types for each provider. Note that, the information required in the profile of the previous release of D-SPACE4Cloud (see DICE Deliverable D3.8), including, e.g., the number of maps, number of reducers, average map time, average reduce time, etc., are now available within the DTSMs (in particular, every DTSM includes such information for each candidate VM type).
• Private Cloud Analysis - In this case the cluster is provisioned in house. This choice in some sense changes radically the problem. In fact, the resources usable to constitute a cluster are generally limited by the hardware available. Therefore, the resource provisioning problem has to contemplate the possibility to exhaust the computational capacity (memory and CPUs) before being able to provision a cluster capable of satisfying a certain concurrency level and deadlines. In such a situation the ARCHITECT could consider two sub-scenarios:
  • Allowing job rejection, that is consider the possibility to reject a certain number of jobs (lowering consequently the concurrency level), i.e., introducing an Admission Control (AC) mechanism. In this case, since the overall capacity is limited, the system reacts to the excess of load by rejecting jobs; this has an impact on the execution costs as it seems fair to believe that pecuniary penalties can be associate to rejection.
  • Denying job rejection, that is imposing that a certain concurrency level must be respected. This translates into a strong constraint for the problem that may not be satisfiable with the resources at hand.

Another aspect not to be underestimated is that data-centers are made of physical machines each with a certain amount of memory and cores. D-SPACE4Cloud considers the linear relaxation of the problem and estimates the total capacity available as the sum of the memory and number of CPUs.