We use proprietary and third party's cookies to improve your experience and our services, identifying your Internet Browsing preferences on our website; develop analytic activities and display advertising based on your preferences. If you keep browsing, you accept its use. You can get more information on our Cookie Policy
Cookies Policy
FIWARE.ArchitectureDescription.Data.MetadataPreprocessing R3 - FIWARE Forge Wiki

FIWARE.ArchitectureDescription.Data.MetadataPreprocessing R3

From FIWARE Forge Wiki

Jump to: navigation, search
FIWARE WIKI editorial remark:
This page corresponds to Release 3 of FIWARE. The latest version associated to the latest Release is linked from FIWARE Architecture



  • Copyright © 2012-2014 by SIEMENS

Legal Notice

Please check the following Legal Notice to understand the rights to use these specifications.


Target usage

Target users are all stakeholders that need to convert metadata formats or need to generate objects (as instantiation of classes) that carry metadata information. The requirements to transform metadata typically stem from the fact that in real life various components implementing different metadata formats need to inter-work. However, typically products from different vendors are plugged together. In this case, the Metadata Preprocessing GE acts as a mediator between the various products.

Example scenarios and main services exported

The Metadata Preprocessing GE is typically used for preparing metadata coming from a data-gathering device for subsequent use in another device (i.e., another GE or an application or another external component). The data-gathering device can be a sensor, e.g., the analytics component of a surveillance camera. Depending on the manufacturer of the camera, different metadata schemes are used for structuring the metadata. The Metadata Preprocessing GE generally transforms the metadata into a format that is expected by a subsequent component, e.g., a storage device. In addition to performing the transformation of the metadata format (e.g., defined by XML Schema), also some elements of the metadata can be removed from the stream by a filtering component. This is especially useful in case these elements cannot be interpreted by the receiving component. For example, in the Use Case project OUTSMART (more specifically, in the Santander cluster), coma-separated sensor metadata is transformed into XML metadata by the Metadata Preprocessing GE for further processing and storage. The transformation task is described in more detail in the following. OUTSMART's advanced measure & managing system (AMMS) is constantly producing sensor data for the following format:


The format can be interpreted as follows: AMMS identifier; date; hour; quarter (of hour); type of measurement; data measured; error code. The Metadata Proprocessing GE transforms this data into XML for further processing.

    <obs from="urn:outsmart:06E1E5A2108003821">
        <what href="phenomenon:activepower"/>
        <param href="identifier:UniversalIdentifierOfLogicalHub"><text>sc.poc.outsmart.eu</text></param>
        <data><quan uom="uom:watts/h">107</quan></data>
    <obs from="urn:outsmart:06E1E5A2108003821">
        <what href="phenomenon:reactivepower"/>
        <param href="identifier:UniversalIdentifierOfLogicalHub"><text>sc.poc.outsmart.eu</text></param>
        <data><quan uom="uom:Varh">12</quan></data>

Note that a special transformation unit (implemented by the GE owner) was necessary to realize this task. (The transformation could not be specified by an XSLT stylesheet, since the input data was not in XML format.)

Basic Concepts

Functional components of the Metadata Preprocessing GE

The following figure depicts the components of the Metadata Preprocessing Generic Enabler. These functional blocks are the Control Interface, the Metadata Interface for inbound streams, the Metadata Transformation & Filtering component, and the Metadata Interface for outbound (processed) streams. The mentioned methods are described in more detail in Section Main Interactions.

Functional components of the Metadata Preprocessing GE

The functionality of the components is described in the following bullet points.

  • Control Interface: The control interface is the entity for configuring and controlling the metadata processing engine. The algorithms used for transformation and filtering as well as the metadata source are configured, i.e., connected using the configureInstance method. Sinks receiving the outbound streams can connect using the information provided in the getMetadata method. More details on the APIs are described in the Section "Main Interactions".
  • Metadata Interface (for inbound streams): Different interchange formats (such as the ones for streaming or for file access) can be realized (i.e., configured or programmed into this interface at a later stage). At the current stage, the Real-time Transport Protocol (RTP) as standardized in RFC 3550 [RFC3550] is supported as interchange format. Different packetization formats for the contained/packetized payload data (i.e., the XML metadata) depending on the application might be used. Usually, the Inbound Metadata Interface" as a (RTSP/RTP) streaming client connected to a (RTP/RTSP) streaming server (i.e., the Streaming Device).
  • Metadata Transformation & Filtering: The Metadata Transformation & Filtering component is the core component of this Generic Enabler. Based on an XML Stylesheet Language for Transformations (XSLT) [XSLT] and a related stylesheet, the processing of the metadata is performed. In principle, other kinds of transformations (other than XSLT) can also be applied (e.g., text-to-XML transformation); however, dedicated changes to the enabler are needed for this. Metadata filtering is an optional step in the processing chain. The filtering can be used, e.g., for thinning and aggregating the metadata, or simple fact generation (i.e., simple reasoning on the transformed metadata). Filtering is usually done during transformation by using XSLT technology. The output of this step is a new encapsulation/formatting of the metadata received.
  • Metadata Interface (for outbound streams): Through this interface, the transformed (and possibly filtered) metadata or metadata stream is accessed. For example, the "Device" connected to the Outbound Metadata Interface can be a (RTSP/RTP) streaming client. In this case, the Outbound Metadata Interface acts as a (RTSP/RTP) streaming server.

Realization by MetadataProcessor asset

The MetadataProcessor asset realizes a specific implementation of the Metadata Preprocessing GE. Timed metadata (i.e., metadata elements with associated timing information, e.g., a timestamp) is received over an RTSP/RTP interface (as specified in RFC 2326 [RFC2326] and RFC 3550 [RFC3550], respectively), which implements the metadata interface for inbound data/streams. Different RTP sessions can be handled; therefore metadata streams can be received from several components/devices (e.g., cameras or other type of sensors). The target in such a realization could be the provision of metadata as facts to a metadata broker, which would be the receiver of the outbound stream.

Main Interactions

The external API is a RESTful API that permits easy integration with web services or other components requiring metadata access and transformation services (e.g., other GEs or external applications/components). The following interface will be supported:

  • getVersion: The version of the Metadata Preprocessing GE is returned.
  • listInstances: All instances (i.e., processing units) of the Metadata Preprocessing GE are listed.
  • createInstance: An instance for processing metadata streams/events is created.
  • getInstanceInfo: The information about a specific instance (i.e., processing unit) is returned.
  • destroyInstance: An existing metadata processing instance is destroyed.
  • startInstance: The metadata processing (e.g., transformation and/or filtering) is started.
  • stopInstance: The metadata processing is stopped/halted.
  • getConfig: The configuration of a specific processing unit is returned.
  • configureInstance: A metadata source (e.g., another GE) is connected to the enabler and/or the metadata processing (e.g., the XSLT stylesheet for the conversion of metadata formats and filtering of metadata streams/events) is configured for a specific instance (i.e., processing unit).
  • getMetadata: The URI (i.e., RTSP URL)of the metadata output stream is returned.

The following figure explains the main interactions in a (very general) example usage. In the first step, a new instance for metadata processing is created. The ID of the instance is returned to the calling application/component. In a second step the processing of the Metadata Preprocessing GE is configured (e.g., by providing an XSLT stylesheet). In a third and fourth step the source and the sink of the metadata processing are configured. Note that the order of the configuration steps (i.e., configureInstance, getMetadata) is arbitrary. Note further that more than one sink can be added as receiving component, but only one source can be configured. (However, additional processing units for metadata transformation can be created using createInstance.) In a fifth step, the processing is started.

Example usage

After the processing is done, the specific instance of the GE is stopped. Note that the instance could be started again afterwards by re-using its instance ID. (A list of all existing instances can be retrieved using the listInstances request.) Also the processing of the source could be reconfigured and sinks can be added or removed. As a final step in this example usage, the specific metadata preprocessing instance is destroyed. Note that it is not necessary to stop the instance before destroying it, since this will be done automatically. A simple but concrete example for metadata transformation and metadata filtering can be found in the Open RESTful API Specification of this GE.

Basic Design Principles

The following basic design principles apply:

  • The Metadata Preprocessing GE realizes a generic metadata transformation approach, which is not restrictive to specific metadata schemes.
  • Encapsulation of transport and metadata transformation is implemented as-a-service, usable from other web applications or components.
  • Transformation is based on standardized and commonly used XML Stylesheet Language for Transformations (XSLT).


[RFC2326] H. Schulzrinne, A. Rao, and R. Lanphier, "Real Time Streaming Protocol (RTSP)", RFC 2326, Apr. 1998.
[RFC3550] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP: A transport protocol for real-time applications", RFC 3550, Jul. 2003.
[XSLT] W3C / M. Kay (editor), "XSL Transformations (XSLT) Version 2.0", http://www.w3.org/TR/xslt20/, Jan. 2007.
Personal tools
Create a book