We use proprietary and third party's cookies to improve your experience and our services, identifying your Internet Browsing preferences on our website; develop analytic activities and display advertising based on your preferences. If you keep browsing, you accept its use. You can get more information on our Cookie Policy
Cookies Policy
Materializing Data/Context Management in FI-WARE - FIWARE Forge Wiki

Materializing Data/Context Management in FI-WARE

From FIWARE Forge Wiki

Jump to: navigation, search

Introduction

Contents

Following is a description of the assets that have been adopted as baseline for building a reference implementations of the GEs in the Data/Context Management chapter of FI-WARE. The reference implementation of a Generic Enabler is typically based on the evolution and integration of a number of assets, some being open source, therefore publicly available, while others being provided by partners of the FI-WARE project. A Backlog of Themes, Epics, Features and User-Stories followed for the evolution and integration of assets linked to the reference implementation of a Generic Enabler is also included.

Finally, a list of topics still being addressed at a high level follows the description of assets in this chapter. They are mapped into Themes and Epics in the Chapter Backlog. Features and User-Stories, derived from refined of these Theme and Epics will be allocated to Backlogs linked to GEs in the future.

For a comprehensive vision on the FI-WARE Data/Context Management chapter architecture, you can go here. We highly recommend you to read it before analyzing how reference implementations of GEs are being materialized.

The Roadmap of the Data/Context Management chapter presents a description of the Technical Roadmap planned for the chapter, which will be developed through subsequent Releases of the FI-WARE Platform. Please also check the Releases and Sprints numbering, with mapping to calendar dates.


Publish/Subscribe Broker - Orion Context Broker

Baseline Assets

  • Orion Context Broker is TID's context management and brokerage platform based on the producer-consumer model.

Epics

Features

Unit Testing Plan

Product guides

Complex Event Processing

Baseline Assets

  • Proactive Technology Online - Proton - is a complex event processing (CEP) engine. The Proactive Technology Online is a platform to support the development, deployment, and maintenance of event-driven applications. While standard reactive applications are based on reactions to single events, the Proactive Technology Online engine component reacts to situations rather than to single events. A situation is a condition that is based on a series of events that have occurred within a dynamic time window called a processing context. The Proactive Technology Online is the next generation of a former asset called AMIT - IBM Active Middleware Technology™

Following is the compilation of entries in the Backlog followed for materialization of this Generic Enabler.

Epics

Features

Unit Testing Plan

Product guides

Cloud Messaging

Baseline Assets

Epics

FIWARE.Epic.Data.AeonCloudMessaging.Dashboard

FIWARE.Epic.Data.AeonCloudMessaging.Backend

FIWARE.Epic.Data.AeonCloudMessaging.Events-Manager

FIWARE.Epic.Data.AeonCloudMessaging.SDK

Features

FIWARE.Feature.Data.AeonCloudMessaging.Dashboard.DemoApps

FIWARE.Feature.Data.AeonCloudMessaging.Dashboard.ResourceManagement

FIWARE.Feature.Data.AeonCloudMessaging.Backend.RestAPI

FIWARE.Feature.Data.AeonCloudMessaging.Backend.Security

FIWARE.Feature.Data.AeonCloudMessaging.Events-Manager.PubSubSystemRecovery

FIWARE.Feature.Data.AeonCloudMessaging.Events-Manager.Security

FIWARE.Feature.Data.AeonCloudMessaging.SDK.PubSubOps

Unit Testing Plan

Product guides

BigData Analysis - Cosmos

Baseline Assets

  • Hadoop is a heterogeneous MapReduce platform that is mainly use for ad-hoc data exploration of large sets of data.
  • MongoDB is a scalable, high-performance, open source, document-oriented database.
  • Cosmos is Telefónica's platform for big data based on Hadoop and MongoDB, and some other plugins on top of MapReduce core.

Following is the compilation of entries in the Backlog followed for materialization of this Generic Enabler.

Epics

Features

Unit Testing Plan

Product guides

CKAN

Baseline Assets

  • CKAN is an open data management platform. It is designed to make publication, access and use of open data easier.

Epics

Features

Unit Testing Plan

Product guides

Social Semantic Enricher

Baseline Assets

Epics

Features

Unit Testing Plan

Product guides

Social Data Aggregator

Baseline Assets

Over time all generations have come to embrace the changes social network has brought about. Nowadays online social media have gained astounding worldwide growth and popularity. They are generating a huge amount of records about users’ activities and also attracting attention from variety of researchers and companies worldwide. Every day data collected by social networks are gathered and feed a variety of analytics (users behavior, habits, topic trends..) which are capable of extracting significant patterns and further analisys. The aim of Social Data Aggregator (SDA) GE is to retrieve data from different Social Networks and provide different analytics depending on user needs. The GE relies on Apache Spark for computation on data. The SDA comes with different built-in modules but custom modules can be added as well.

Epics

Features

Unit Testing Plan

Product guides

Metadata Store Management Platform

Baseline Assets

Epics

Features

Unit Testing Plan

Product guides

Stream-oriented Generic Enablers

A number of Epics related to Stream-oriented Storage, Processing and Distribution will be addressed under the umbrella of a dedicated topic within the 2nd FIWARE Open Call, leading to identification of a number of Generic Enablers and potential selection of baseline assets contributed by new partners to the FIWARE consortium.

Transmission of content and data streams over a network can be performed in a variety of ways. The type of content and data to be transferred and the underlying networks conditions determine the methods for communicating. From the common end user view point, there is today the desire to access unlimited amounts of content/data, including highly accurate timing dependencies between the different sources. But there are also other needs for another category of applications in which the challenge is to transmit and store temporarily a huge number of streams coming from hundreds of moving stream sources (e.g., mobile device video cameras).

Initially, content streaming was mostly done over managed network using specific packaging of the payloads and specific transport protocols to guarantee the Quality of Service (QoS) requirements. This led to solutions such as DVB standards, piling RTP-RTSP over MEPG-TS or MPEG coding. But nowadays, things are changing and unmanaged network are utilized with HTTP protocols to perform adaptive streaming. More, streaming is not limited to unidirectional delivery but can be envisaged within interactive applications including complex processing operations. The Epics proposed here for streaming should provide all the means to set up services based on the distribution or exchange of content and data in a streaming manner. A final target for this enabler could be to offer “Streaming as a Service”. Streaming thus should apply not only to media contents but also to streams of sensor data and many other forms of information within the FI-PPP.

Baseline Assets

Epics

Features

Unit Testing Plan

Product Guides


Data Context Streams (bundle)

The bundle Data Context Streams provides an instalable set of GEris packaged together. These GEris are offered in an integrated way so that they can be easily deployed and used by developers:

  • Orion Context Broker: It is the key element and mandatory GEri of this bundle. It allows to publish and consume context information through the NGSI APIs. Other GEs in the bundle will publish or consume context information from Orion Context Broker.
  • Cygnus storage: Although it is not a generic enabler, Cygnus is a complementary piece of software for Orion Context Broker that can be used to store historical information from the Orion Context Broker both in Cosmos Big Data storage and Open Data portal.
  • PROTON Complex Event Processing: This GE is able to receive Context information through NGSI subscriptions, detect complex situations based on preconfigured rules, and generate new context information to be published in the Context Broker or other types of output.
  • Cosmos Big Data: This GE is the destination of the Context Broker data through Cygnus component. All the context historical information will be stored in the GE for later analysis through map & reduce applications or Hive based queries.
  • Kurento Streams-Oriented GE: This GE is offered optionally as part of the bundle. There are scenarios where the result of analysis of media can generate context information to the Context Broker, as detecting objects in an area of an image (fences), or analyzing a given situation from the video (e.g.: people detection).

The Data Context Streams bundle allows to generate, distribute, store, analyze and use context information. Context information is generated in many situations from IoT Generic Enablers (data from sensors), IT systems integrations and context data publication through NGSI APIs. This context data can be consumed directly by applications, but there are several use cases where it is useful to further exploit and process this data by using the functionality provided as part of the FIWARE GEris and Cygnus component mentioned before.

Bundle guides

General

The following Epics include cross GE functionality that is further defined in the respective GE backlogs:

Personal tools
Create a book