We use proprietary and third party's cookies to improve your experience and our services, identifying your Internet Browsing preferences on our website; develop analytic activities and display advertising based on your preferences. If you keep browsing, you accept its use. You can get more information on our Cookie Policy
Cookies Policy
How to create recipes for the GEs - FIWARE Forge Wiki

How to create recipes for the GEs

From FIWARE Forge Wiki

Jump to: navigation, search

GE owner partner need to distribute their GEs installation and configuraion software in a automatic way in terms of recipes. This tutorial tries to help GE owners which some guidelines to start to work with these recipes.


Contents

Motivation

FIWARE partners provide generic enablers implementations to be used by the UCs. The process to instantiate a GE in the testbed can be executed by two processes:

  • Each GE can be offered as Openstack image, so that, the UC needs just to deploy this image in the testbed.
  • The UC can specify exactly which software needs to be installed where and the platform will instantiate the required infrastructure.

The first approach is valid for simpler GEs which only requires one VM, for advances functionalities and for integration of several uses The idea for instantiate what the UCs wants. It is by using simple scripts which make all the process install, uninstall, configure, update of the required software and it is taken care of the configuration among different VMs.

Chef, cookbooks and recipes

Chef is an open-source systems integration framework, to automate the software remote installation processes. It gives flexibility from rapid provisioning and deployment of servers to the automated delivery of applications and services at any scale. Chef provides a flexible model for reuse by enabling users to model infrastructure as code to easily and consistently configure and deploy infrastructure across any platform. There it is possible to specify any kind of instruction about the software. Concretely, Chef automation instructions are writeen in Ruby code and allows to describe how machines should be set up.

Chef is used in a client/server mode: a Chef server manages all the nodes that are managed, keeping information about their location and the operations that have to be executed on every node in a given time. These operations are described in the recipes, which describes how a part of a node (such as Apache Tomcat, MySQL, or HAProxy) has to be configured. These recipes describe a series of resources that should be in a particular state: packages that should be installed, services that should be running, or files that should be written. Chef makes sure each resource is properly configured, and provides a safe, flexible, easily-repeatable mechanism for making sure all the nodes are always running exactly the way they were foreseen. The recipes are written in Ruby and all the recipes belonging to a single product are aggregated in a cookbook, which is stored and managed from the Chef Server.

In addition, a Chef agent is installed in each node (VM). Whenever a new request of product or artifact installation is received, it is configured in the Chef server and executed by the Chef client through the execution of a recipe. All the recipes available for the management of a given product are aggregated in a cookbook with a naming convention.

Fiware and Chef

Cookbooks in FIWARE are going to be managed by the Chef server. However, in order to help to cookbooks developers, FIWARE is going to provide them a VM with chef-solo and chef-client installed, so that, they can code and test cookbooks there. In this way:

  • A VM with chef-solo to do testings
  • GE owners have to create cookbooks
  • GE owners have to provides GE RPM and debian packages

Deploying a VM for testing Chef cookbooks/recipes

There is a testbed for FIWARE people. It is in http://130.206.80.61. You need to have a user/password for accessing. If you don't have you can contact to testbed people. You can deploy an image for testing chef, it is called paas-chef-solo-templ and you can find in the images tab. Before it, be sure you have created a keypair (public/private key for accessing to the VM) and assign it when you deploy the VM

Chef solo and cookbooks

You will have a Openstack image with the chef solo installed and configured. It is the paas-chef-solo, which needs a keypair to access to it. So firstly, they a VM from this image, access to it. In order to check that chef is installed, just click

 chef-solo
 >Starting Chef Client, version 11.4.0
 Compiling Cookbooks...
 FATAL: No cookbook found in "/var/chef/cookbooks/", make sure cookbook_path is set correctly.
 ERROR: Running exception handlers
 ERROR: Exception handlers complete
 Chef Client failed. 0 resources updated

This means that there isn't a cookbook to be executed. So that the next step should be to create a cookbook.

Create a cookbook

The first thing to do when writing a cookbook is to create the cookbook directory structure

 knife cookbook create "cookbookexample"

All the coobooks created will be stored in the folder /var/chef/cookbooks. The resulting directory structure will be created:

 % tree cookbooks/cookbookexample
 cookbooks/cookbookexample
 ├── README.md
 ├── attributes
 ├── definitions
 ├── files
 │   └── default
 ├── libraries
 ├── metadata.rb
 ├── providers
 ├── recipes
 │   └── default.rb
 ├── resources
 └── templates
   └── default

Afterwards, if we execute again chef-solo, we will obtain

 chef-solo
 Starting Chef Client, version 11.4.0
 Compiling Cookbooks...
 Converging 0 resources
 Chef Client finished, 0 resources updated

Node configuration

Since chef-solo doesn't have any interaction with a Chef Server, you'll need to specify node-specifc attributes in a JSON file. This can be located on the target system itself, or it can be stored on a remote server. To execute is


 chef-solo -j node.json

Within the JSON file, you'll specify the recipes that Chef should run in the "run_list". An example JSON file, which sets a resolv.conf:

 ~/node.json
 {
   "resolver": {
     "nameservers": [ "10.0.0.1" ],
     "search":"int.example.com"
   },
   "run_list": [ "recipe[resolver]" ]
 }

Chef recipes

The information about the different Chef resources can be found in []. Here it is possible to check the main and basic ones to start working.

Install the package

There is a Chef resource for installing RPM or debían packages. This is the package resource . If you can use the following actions (:install, :remove, : :upgrade) First, as I’ve installed the package, I clearly need a package. I’m pretty confident that this particular package will not break backwards compatibility, and can be safely upgraded to the latest version if necessary.

 package "cookbookpackage" do
   action :install
 end

Concretely, the sintaxis for the package is

 package "name" do
   some_attribute "value" # see attributes section below
   ...
   action :action # see actions section below
 end

Manage the service

The software needs to be installed as a service. That is, there’s an init script that can be enabled and started to monitor disk devices. This is actually the whole point of the cookbook. For managing the service, is used service resource is used to manage a service. Concretely, it is possible to start|stop|restart|reload|force-reload|status a service. In its sintaxis is required to specify the service name, some attributes (in case required) and an action.

 service "name" do
   some_attribute "value" # see attributes section below
   ...
   action :action # see actions section below
 end

As an example

 service "smartmontools" do
   supports :status => true, :reload => true, :restart => true
   action [:enable,:start]
 end

In this resource, it’s possible to use meta parameter supports for the service. It is possible to do it what the init script can do by simply running it with no options on thetest system.

 % /etc/init.d/smartmontools
  • Usage: /etc/init.d/smartmontools {start|stop|restart|reload|force-reload|status} The various options passed to the init script manage it in the familiar way. Telling Chef about it has a specific effect on the way the service provider functions when Chef manages it.
  • status: Chef will use /etc/init.d/SERVICENAME status to determine if the service is running. If the return code is 0, its running. Otherwise, Chef checks the process table for a process running with the name of the service.
  • reload: Chef can only take the reload action for a service if it actually supports reload.
  • restart: When the restart action is sent to a service, if it supports restart then Chef will use/etc/init.d/SERVICENAME restart. Otherwise, Chef will use stop and start.

Earlier when we wrote the template resources, we notified the service to reload. Since the resource supports reload, we can do this. Also note that the resource has an array of actions. Each of these actions will be taken if necessary when Chef manages it. In this case, it will be enabled at boot time, and then started if it is not already running.

Configuration files are templates

The template resource is used to manage file contents with an embedded Ruby (erb) template. This resource includes actions and attributes from the file resource. Template files managed by the template resource follow the same file specificity rules as the remote_file and file resources. The syntax for using the template resource in a recipe is as follows. The action can involve :create or :create_if_missing and template

 template "name" do
   some_attribute "value" # see attributes section below
   ...
   action :action # see actions section below
 end

Configuration files are generatically dinamically by templates. These templates Templates are dynamically generated using ERB. For instance, the next example crea a configuration file based on a template with some permissions and after that it reload the service.

 template "/etc/default/smartmontools" do
   source "smartmontools.default.erb"
   owner "root"
   group "root"
   mode 0644
   notifies :reload, "service[smartmontools]"
 end

Templates are dynamically generated using ERB, and they can use Node attributes. It is possible to use the automatically detected attributes, or to set new attributes for the node in the cookbook. The attributes go in the attributes/default.rb file in the cookbook. This attributes are definitely something to document in the README. For instance

 default['smartmontools']['devices']      = []

In templates/default/smartd.conf.erb, I check if there’s a list of devices to monitor, and if so iterate over the list passing in the default options (device_opts). The cookbook doesn’t at this time support per-device options – the same ones are applied to all devices. If devices is empty, then the configuration will use DEVICESCAN.

 <% if node['smartmontools']['devices'].length > 0 -%>
 <%   node['smartmontools']['devices'].each do |device| -%>
 <%=    "/dev/#{device} #{node['smartmontools']['device_opts']}" %>
 <%   end -%>
 <% else -%>

Recipes structure

All cookbooks used as software to be installed in FIWARE, needs to have the following recipes (where version is the software version):

  • version_install: the recipe to install the sotfware
  • version_uninstall: the recipe to uninstall the sotfware
  • version_configure:
  • version_start:
  • version_stop:

There is a svn with the cookbooks, you can use your forge account. You can have a look at the contextbroker GE cookbook.

1. When you have created your cookbok, please upload it into the svn. Your cookbooks will be uploaded automatically to the chef server for testing: https://130.206.81.105. There you can check that your cookbook is there. For accessing to the GUI, you should ask for an user in the GUI.

2. You can deploy a VM with a chef-client installed (this chef-client is connected to the chef server). There is an image which is called chef-client-test which you can use

3. Once you have deployed the VM, enter in the VM, and execute chef-client. This process will register your VM into the chef-server. Thus, you can see your VM in the node tag of chef-server. You can go to the edit tab and add the recipe you want to test.

4. Go to your VM and execute chef-client, you will see how the recipe is being executed.

Chef client / Chef-server

Once the recipes have been testbed with the chef-solo, it's time for using the chef-client /chef-server configuration. FI-WARe provides an experimentation chef-server (https://130.206.81.105). To work with it, firstly, you need to ask for a valid admin user for the chef-server to the testbed team. Once you have it, you will be able to:

  • Execute chef-client in your VM, so that, you VM will be registered in the chef-server.
  • Access the GUI by the user/password
  • Acces the chef-server by CLI. To do that, in the chef-client node:
    • just copy the private key (the testbed team provide to you) to /root/.chef/youruser.pem
    • then execute knife configure, and specify https://130.206.81.105 as the chef-server and /etc/chef/validation.pem as the chef-validator certificate
  • You can execute knife node list and you can query the existing nodes
  • If you execute knife node show yournode, you will have information about it
  • you can add a recipe to execute with knife node run_list add yournode yourcookbook::yourecipe
Personal tools
Create a book