We use proprietary and third party's cookies to improve your experience and our services, identifying your Internet Browsing preferences on our website; develop analytic activities and display advertising based on your preferences. If you keep browsing, you accept its use. You can get more information on our Cookie Policy
Cookies Policy
FIWARE.OpenSpecification.Cloud.SDC R3 - FIWARE Forge Wiki

FIWARE.OpenSpecification.Cloud.SDC R3

From FIWARE Forge Wiki

Jump to: navigation, search
Name FIWARE.OpenSpecification.Cloud.SDC
Chapter Cloud,
Catalogue-Link to Implementation Sagitta
Owner Telefónica I+D, Henar Muñoz Frutos

Contents

Preface

Within this document you find a self-contained open specification of a FIWARE generic enabler, please consult as well the FIWARE Product Vision, the website on http://www.fiware.org and similar pages in order to understand the complete context of the FIWARE platform.

FIWARE WIKI editorial remark:
This page corresponds to Release 3 of FIWARE. The latest version associated to the latest Release is linked from FIWARE Architecture


Copyright

Copyright © 2012 by Telefónica I+D. All Rights Reserved.

Legal Notice

Please check the following FI-WARE Open Specifications Legal Notice to understand the rights to use these specifications.

Overview

This specification describes the Software Deployment and Configuration (SDC) GE, which is the key enabler used to support automated deployment (installation and configuration) of software on running virtual machines. As part of the complete process of deployment of applications, the aim of SDC GE is to deploy software product instances upon request of the user using the SDC API or through the Cloud Portal, which in turn uses the PaaS Manager GE. After that, users will be able to deploy artifacts, that are part of the application, on top of the deployed product instances

The following diagram shows the main components of the Service Deployment and Configuration GE (SDC GE)


SDC Architecture Overview


SDC architecture specification


In order to deploy product instances and artifacts, the SDC GE operates on SDC-enabled virtual machines (also referred as Runtime Execution Containers, RECs) that include a preinstalled SDC client. The SDC GE relies on the existence of a Configuration Management Engine, like the Opscode Chef or Puppet system integration frameworks, to automate software remote installation processes. Those Configuration Management engines are used in a client/server mode: a central Configuration Management Engine manages all the nodes (RECs), keeping information about their location and the operations that have to be executed on every node in a given time. These operations are described in form of recipes that describe how the software to be placed on a node (such as Apache Tomcat, MySQL, or HAProxy) has to be deployed. They describe a series of resources that should be configured to start running in a particular state: packages that should be installed, services that should be running, or files that should be copied. The Configuration Management Engine makes sure each resource is properly configured, and provides a safe, flexible, easily-repeatable mechanism that ensures all the nodes are always running exactly the way they were expected. Recipes belonging to a single product are aggregated in a cookbook for that product, which is stored and managed from the central Configuration Management Engine. Elements in the Product Catalogue must be synchronized with the cookbooks that are available in the central Configuration Management Engine.

All the available recipes needed for the management of a given product are aggregated in a cookbook following a naming convention. Every product supported by the SDC must have a cookbook with all the required recipes. Some instructions about how to build cookbooks and recipes can be found in Recipes for GEs.


Target Usage

The SDC GE automates the (un)-installation, configuration and management of product instances on a REC and the installation, configuration and management of artifacts on a previously installed product instance. The primary usage of the SDC GE is to provide an execution layer to be used by the PaaS Manager GE which ultimately deals with all the processes associated to software management. Nevertheless, the recipes managed by the SDC GE can be easily defined for complete products. Therefore, the SDC GE can be used for the deployment and configuration of complete applications on virtual machines, using the SDC GE APIs directly or through an on-demand application deployment portal.

Main Concepts

Following the previous FMC diagram of the SDC, this section introduces the main concepts related to this GE through the definition of their interfaces and components, and by means of an example of their use.

The SDC is responsible for carrying out the provisioning and maintenance of the software in virtual machines. The key concepts visible to the cloud user could be differentiated between the interfaces and the components. All of them are described below.

Entities

To use the SDC effectively, the following concepts have to be explained, which are illustrated in next figure. Take into account that these concepts are just a subset of the concepts defined for the PaaS Manager GE.

  • Software Product. A software product is a software piece to be installed in an operating system that can work by itself (it does not need another software to run). Both Tomcat and MySQL are example of software products.
  • Software Product Release. A software product release involves a concrete release (version) of a software product. Tomcat 7.0 or Tomcat 5.5 are examples of two releases of the tomcat product. Both software product and product release information are stored in the Product Catalogue.
  • Software Product Instance. It is the instantiation of a software product release in a VM or server.
  • Application. An application is a set of configured artifacts, supporting a concrete set of functions, deployed over one or more VMs or servers and software product instances deployed on top of that VMs and servers.
  • Application Release. An application release represents the specification of a given version of an application, through the specification of all the artifacts and attributes that are required for its deployment.
  • Artifact. An artifact refers to each one of the application components that made up an application. Both a war or a properties file are examples of artifacts.
  • Application Instance. An application instance is an actual deployment of all the application artifacts of a given Application Release on one or more RECs.
  • Attributes. The attributes associated to an application or a software product release that are used to help in their configuration during the deployment process.
  • VM. A virtual machine or server represents the infrastructure where a REC is created.
SDC Class Diagram
SDC Class Diagram

Interfaces

The SDC GE is accessible through the Service Deployment and Configuration Interface (SDCI). This interface is a REST interface that allows End Users or Developers to cope with the deployment and lifecycle management of software products and applications defined to run in virtual machines. In addition, it offers to Software Providers the possibility to add recipes about their software in the SDC catalogue. The SDCI API is specified at SDC Open RESTful API Specification (PRELIMINARY)

In addition, the SDC interacts with other components through well-defined interfaces:

  • The SDC Client Interface, which is used to obtain information from a VM or server and request Configuration agent executions.
  • The Configuration Engine Interface, which is the API for interacting with the configuration engine, whose API (for Chef server) is found at [1]

Components

The SDC GE internal components are:

  • Communication Service. It implements the REST API interface (SDCI) and manages the asynchronous tasks status.
  • ProductManager. It manages the catalogue of products.
  • ProductInstanceManager. It is in charge of managing and executing the installation of products and artifacts through the Configuration Engine.
  • ConfigurationManager. This component is in charge of managing the communication with the Configuration Engine for the assignation of new recipes to the REC execution queue, and to trigger the actual execution through the SDC client in the REC.
  • Repositories. There is a repository for task statuses, a catalogue of product releases (and applications) and an inventory of product instances.

Used by the SDC GE but separated from it, three components have to be highlighted:

  • Central Configuration Engine, which manages the execution of recipes on the nodes (RECs). One example is the Chef server.
  • Configuration agent, which executes the recipes in the local RECs.
  • SDC client, which serves as support and intermediary between the SDC and the configuration agent.

Example Scenario

Apart from querying information about the defined or installed products, there are two main usages of the SDC GE: installing products and installing artifacts. Installing a product involves to specify either a VM or server details and credentials, and a product (previously defined and with a proper cookbook). Installing an artifact requires identifying the product instance over which it has to be installed and configured, and to trigger the execution of the proper recipe. Both scenarios run asynchronously, and when they are requested, the SDC GE returns a task identifier that can be used to track the status of the operation.

Main Interactions

Although the scenarios of the SDC are simple, the interactions are complex due to the number of components involved. The following figure depicts how it works.

SDC interactions.


SDC interactions


Usually, the SDC GE will receive a request for installing a product from the PaaS Manager (1), although it may be also be used by a client application or from a web portal. The SDC GE knows the IP address of the host where the product has to be installed, but does not know the hostname and host domain, which are required by the Configuration Engine. Therefore, it obtains that information from the SDC Client running in the REC (2).

After that, the SDC GE assigns the execution of a recipe to the corresponding node (3) and notifies the SDC client (4) to request the Configuration Agent in the same VM or server (5) to execute the pending recipes for that node, as informed by the Configuration Engine (6). With that information, the Configuration Agent executes the recipe needed to install the software product instance (7). Finally, the SDC GE requests the Configuration Engine to unassign the recipe to the node (8).

The second scenario starts when the PaaS Manager requests to install an artifact (9). The SDC GE receives information about the product instance and therefore, knows the node where the recipe has to be executed. The process to install the artifact from that moment on is exactly as described above, except because the name of the recipe for installing an artifact will be different from the one in the recipe used to install a software product.

Basic Design Principles

Design Principles

The SDC GE has to comply with the following technical requirements:

  • Decouple the management of the catalogue (specifications of what can be deployed) from the inventory (instances of what has been already deployed).
  • Support an Asynchronous interface with polling mechanism to obtain information about the deployment status.
  • Deal with ad-hoc interfaces definition, given that no real or de-facto standard exists for PaaS.

Resolution of Technical Issues

When applied to SDC GE, the general design principles outlined above can be translated into the following design approaches:

  • REST based automated self-provisioning interface for both products and artifacts.
  • Dependency on the Configuration Management Engine (Opscode Chef or Puppet) to automate the installation processes; the virtual machines have to be based on images that include the SDC Client and the Configuration Agent to be usable by the SDC.
  • Multitenancy is not covered by the SDC GE. The entire lifecycle of software products and applications is managed at a higher level, by the PaaS Manager or by an administrator.
  • SDC GE does not manage the installation and configuration of the infrastructure required for monitoring or measuring the service; these must be provided by the cookbooks or the VM or server images used.

Detailed Specifications

Following is a list of Open Specifications linked to this Generic Enabler. Specifications labeled as "PRELIMINARY" are considered stable but subject to minor changes derived from lessons learned during last interactions of the development of a first reference implementation planned for the current Major Release of FI-WARE. Specifications labeled as "DRAFT" are planned for future Major Releases of FI-WARE but they are provided for the sake of future users.

Open API Specifications

Re-utilised Technologies/Specifications

The SDC GE is based on RESTful Design Principles. The technologies and specifications used in this GE are:

  • RESTful web services
  • HTTP/1.1 (RFC2616)
  • XML data serialization formats.

Terms and definitions

This section comprises a summary of terms and definitions introduced during the previous sections. It intends to establish a vocabulary that will be helpful to carry out discussions internally and with third parties (e.g., Use Case projects in the EU FP7 Future Internet PPP). For a summary of terms and definitions managed at overall FI-WARE level, please refer to FI-WARE Global Terms and Definitions

  • Infrastructure as a Service (IaaS) -- a model of delivering general-purpose virtual machines (VMs) and associated resources (CPU, memory, disk space, network connectivity) on-demand, typically via a self-service interface and following a pay-per-use pricing model. The virtual machines can be directly accessed and used by the IaaS consumer (e.g., an application developer, an IT provider or a service provider), to easily deploy and manage arbitrary software stacks.
  • Platform as a Service (PaaS) -- an application delivery model in which the clients, typically application developers, follow a specific programming model to develop their applications and or application components and then deploy them in hosted runtime environments. This model enables fast development and deployment of new applications and components.
  • Project is a container of virtual infrastructure that has a set of virtual resources (e.g., computing capacities, storage capacities) to support the former. In other words, a VDC is a pool of virtual resources that supports the virtual infrastructure it contains.
  • Service Elasticity is the capability of the hosting infrastructure to scale a service up and down on demand. There are two types of elasticity -- vertical (typically of a single VM), implying the ability to add or remove resources to a running VM instance, and horizontal (typically of a clustered multi-VM service), implying the ability to add or remove instances to/from an application cluster, on-demand. Elasticity can be triggered manually by the user, or via an Auto-Scaling framework, providing the capability to define and enforce automated elasticity policies based on application-specific KPIs.
  • Service Level Agreement (SLA) is a legally binding contract between a service provider and a service consumer specifying terms and conditions of service provisioning and consumption. Specific SLA clauses, called Service Level Objectives (SLOs), define non-functional aspects of service provisioning such as performance, resiliency, high availability, security, maintenance, etc. SLA also specifies the agreed upon means for verifying SLA compliance, customer compensation plan that should be put in effect in case of SLA incompliance, and temporal framework that defines validity of the contract.
Personal tools
Create a book