We use proprietary and third party's cookies to improve your experience and our services, identifying your Internet Browsing preferences on our website; develop analytic activities and display advertising based on your preferences. If you keep browsing, you accept its use. You can get more information on our Cookie Policy
Cookies Policy
FIWARE.OpenSpecification.Apps.Repository R4 - FIWARE Forge Wiki

FIWARE.OpenSpecification.Apps.Repository R4

From FIWARE Forge Wiki

Jump to: navigation, search
Name FIWARE.OpenSpecification.Apps.Repository
Chapter Apps,
Catalogue-Link to Implementation Repository-RI
Owner UPM, Francisco de la Vega

Contents

Preface

Within this document you find a self-contained open specification of a FIWARE generic enabler, please consult as well the FIWARE Product Vision, the website on http://www.fiware.org and similar pages in order to understand the complete context of the FIWARE platform.

Copyright

  • Copyright © 2014 by SAP
  • Copyright © 2015 by UPM

Legal Notice

Please check the following FIWARE Open Specification Legal Notice (essential patents license) to understand the rights to use this open specification. This scheme differs from the one chosen for the rest of open specifications that make up the FIWARE platform since it is inherited from the one chosen by SAP between the two FIWARE license schemes allowed in Release 3.

To illustrate this open specification license from our perspective:

  • The specifications of this Generic Enabler are made available under IPR rules that allow for a exploitation and sustainable usage both in Open Source as well as proprietary, closed source products to maximize adoption.
  • This Open Specification is exploitable for proprietary 3rd party products and is exploitable for open source 3rd party products, including open source licenses that require patent pledges.
  • If the owner of this GE spec holds a patent that is essential to create a conforming implementation of the GE spec (i.e. it is impossible to write a conforming implementation without violating the patent) then a license to that patent is deemed granted to the implementation.

Note: SAP provides the software associated to the Repository - RI (Reference Implementation) according to the initial specification. UPM provides the software associated to the Repository - RI according to the current version of the specification as reflected in these pages. Both implementations are provided as open source under the BSD License. Please check the specific terms and conditions linked to this BSD open source license at https://github.com/conwetlab/Repository-RI/blob/master/license.txt. You can also obtain the Repository GE software using the FIWARE Catalogue at http://catalogue.fiware.org/enablers/repository-repository-ri

Overview

Together with the Store GE and the Marketplace GE, the Repository GE is a core enabler of the FIWARE Business Framework. The Repository GE provides a consistent uniform API to access USDL service descriptions and associated media files for applications of the business framework. A service provider can use the Repository GE to publish the description of various aspects of the service according to a unified description language.

USDL is used in its Linked Data version "Linked USDL". Documentation can be found at <http://linked-usdl.org/> . Information about the FIWARE Platform is available at http://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php . USDL describes services on a metadata level and can refer to supplemental resources of any media type. Therefore the Repository GE must be able to store resources in arbitrary formats. The RDF datamodel of USDL allows referring to entities of the service description via the resource URL. Therefore Linked-USDL is already well prepared to allow the distribution of service descriptions all over the Internet.

Moreover, the Repository GE provides a uniform API to perform smart searches over the published descriptions. This API is oriented to service descriptions and media files serialized in RDF formats (e.g a Linked-USDL document) and allows the usage of SPARQL as query language.

Target usage

The Repository GE is a place to store service models, especially USDL descriptions but also other models required by components of the overall FIWARE business framework (e.g. technical models for service composition and mashup). The Repository GE provides a common location for storage (centrally or distributed and replicated), reference and/or safety.

The Repository GE contains published descriptions, which can be utilized by any component in respect to privacy and authorization constraints imposed by the business models. Usually a repository is under control of an authority and usually is keeping track of versions, authenticity and publication dates.

User roles

  • The Provider creates services describing basic service information as well as technical information. He needs to upload and publish service descriptions on the repository in order to make them available to other components of the platform, such as the Shops/Stores, Aggregators, etc.
  • The Aggregator will use for example technical and service-level information of existing services in the Repository in order to perform a composition of a value added service or application. So for example, in order to give a valid statement about the availability of the new composed service. The availability of each contained service needs to be considered. The Aggregator also needs information about the technical interfaces of a service in order to develop code to call them correctly . Service descriptions for the newly created composite service can be uploaded and published to the repository again.
  • The Marketplace Provider enables consumers to make comparisons between services therefore needs all kind of business relevant descriptions of services, such as general descriptions, business partners, service-levels, and pricing, to be presented in the store or within the marketplace. Also technical information can be required, on a level to be able to do comparisons between services for the consumer.
  • The Channel Maker. The internet consists of various channels and heterogeneous devices for consuming information such as Web (browser), Android, iOS but also global as well as local social networking and community platforms such as Facebook, LinkedIn, MySpace, Xing, KWICK! The Channel Maker needs detailed information about the channel to ensure the proper channel creation or selection. Further a channel may require embedding or wrapping the service so it can be accessed by the user through the specific channel.
  • The Hoster requires information on service-level descriptions, deployment and hosting platform requirements to provide the necessary infrastructure needed to host applications/services in a reliable and scalable way.

Basic Concepts

Web Citizen

The Repository GE is relying on Web principles:

  • URI to identify resources
  • Consistent URI structure based on REST style protocol
  • HTTP content negotiation to allow the client to choose the appropriate data format supporting HTML, RDF, XML, RSS, JSON, Turtle, ...
  • Human readable output format using HTML rendering ('text/html' accept header) including hyperlinked representation
  • Use of HTTP response codes including ETags (proper caching)
  • Linked Data enablement supporting RDF input and output types

Open Distributed Architecture

The Repository GE Open Specification has to be seen as a specification of the repository abstract functionality. There can be many technologies used to implement the functionality. Often the repository protocol is implemented on top of a Web content management system.

Also we envision a large number of repositories containing service descriptions, which also might to refer to descriptions in other repositories. Repositories can be hosted by the provider or a provider may use repository services of other platform providers. The latter might be an alternative for small sized providers, which don't want to provide an own infrastructure.

The service descriptions in a repository are typically used by different other components of the platform, such as service stores or marketplaces, by extracting information needed for the specific functionality.

Data Model

The Repository GE is structured into core objects, which are resources and collections. These objects constitute also the granularity of access control. Collections are containers for storing resources, which are typically used to maintain all resources belonging to a certain service description in one place.

Resources

The resources are mainly the USDL service descriptions themselves as well as complementary media files that are used within the service descriptions.

Collections

A collection is a container for collecting resources. Multiple collections can be used on the repository for various purposes. Collections can be nested and may provide versioning of the resources. Collections are used to keep all content that is locally referred from the service descriptions together in one place. For example a service description often has additional documentation, depictions and other collateral information, which can be bundled together in one collection.

Recipes

Recipes are virtual containers selecting resources from different collections.

Content Negotiation

For optimal interoperability and flexible use, the Repository GE should be able to deliver the results of an operation in multiple formats. HTTP content negotiations should be used to let the client choose an appropriate content type. Basic content types (mime-type) are

  • HTML - to deliver the results in hyper-linked HTML that can be rendered directly in a Web Browser
  • RDF - various RDF serializations for processing in applications
  • JSON - Javascript Object Notation for easy processing in a mashup environment.

Repository Architecture

The Repository GE is used by various other GEs within the FIWARE platform. Namely Marketplace GE, Store GE, Application Mashup GE as well as the Revenue Settlement and Sharing System GE can access the Repository GE to retrieve detailed information about a service or application. In general, the different services and applications offered in the FIWARE Business Framework are described in Linked-USDL documents that are published in the Repository GE. These descriptions can be later used to discover and compare offerings.


Repository in the context of the Business Framework

Besides the FIWARE platform, Future Internet applications or composite services on top of the FIWARE platform can also use the Repository GE as a service for their own purpose. An example of the inner architecture of the Repository is shown in the following diagram.

Example of high-level architecture of a repository implementation

The architecture shown here is only a blueprint for possible implementations of the Repository GE and depicts the functional components, necessary to realize this functionality. There are many technology options for a concrete implementation, depending on the context and application domain and its nonfunctional requirements. Since the requirements according to repository size, workload and other parameters can be quite different, there is no obvious all-encompassing implementation solution. The implementations can span very simple ones, which provide only few extensions to a standards Web service to very sophisticated ones that utilizes enterprise content management systems (e.g. based on the "CMIS - Content Management Interoperability Services" standard).

The Repository GE only stores and provides access to service descriptions. Since there is no common standard for versioning, and the requirement according to versioning may vary depending on the use case scenario, we do not require version control from a repository implementation, although a real implementation can provide versioning models and mechanism (e.g. using the capabilities of the underlying CMS system).

Also there is no requirement regarding consistency checking of the service descriptions in the Repository GE. The applications themselves have to ensure that the descriptions are consistent. All clients of a repository have to cope with incomplete and inconsistent information by default. This reflects the architecture of the Web, where also no consistency commitment of the pages on different Web servers can be made. To ensure integrity additional measures have to be taken.


Technical Interfaces

Main Interactions

The Repository GE operation protocol is kept very simple. It basically provides operations to get and put resources, such as service descriptions and media content. Additional operations are used to structure the repository into collections of resources.

Managing Resources (mandatory)

The core functionality of the Repository GE is to store resources and retrieve them when necessary. Further resources sometimes need to be updated and eventually deleted. The following diagram shows an example sequence of resource management operations of the Repository GE.

Example sequence of resource management operations

The Get Resource operation can be used to retrieve a resource from the Repository GE. This operation delivers the actual content of the resource and/or metadata about the resource, such as the media type, creator, or modification date, depending on the used technical interface. The following parameters need to be exchanged:

  • resource identifier - Resource identifier of the resource to be returned.

If only information about collections is requested, the collection identifier is used instead of the resource identifier.

  • collection identifier - Identifier for the collection, which contains the resource.
  • resource - Resource which will be returned
  • media type - Media type of the resource which will be returned.


The Post Resource operation is used to store a new resource in the Repository GE. The repository should take precautions to provide inconsistent changes due to concurrent access. The following parameters need to be exchanged:

  • resource identifier - Identifier which contains the resource.
  • collection identifier - Identifier which denotes the collection which the resource will be put. The collection can be a part of the resource identifier. For example, when URL paths are used to identify a resource.
  • resource - the content of the resource to be stored in the repository.

The Put Resource operation is used to store a new resource into the repository or update an existing resource with the same resource identifier. The repository should take precautions to provide inconsistent changes due to concurrent access. The following parameters need to be exchanged:

  • resource identifier - Identifier which contains the resource.
  • collection identifier - Identifier which denotes the collection into which the resource will be put. The collection can be a part of the resource identifier, if for example URL paths are used to identify a resource.
  • resource - the content of the resource to be stored into the repository


In order to delete a resource irrevocably from the repository the Delete Resource operation is used. The following parameters are exchanged:

  • resource identifier - Resource identifier of the resource to be deleted.


Managing Collections

Collections are used to put a structure into the Repository GE. In order to easily access parts of the Repository GE, it allows clients to get information about the contents of individual collections. The following diagram shows an example sequence of collection management operations of the Repository GE.

Example sequence of collection management operations

The Create Collection operation creates a new collection in the Repository GE, containing the necessary details such as owner, policies, and other metadata attributes. It requires the parameter:

  • description - Description of the collection to be created, which contains the location path within the repository and administrative data such as creator and access policies.


To get the details and contents of a collection the Get Collection operation is used. The collection information contains information such as owner, policies, textual descriptions, dates, versions, number of resources, and more. The level of detail of the description may depend on the authorization level of the requester. The following parameters can be involved in the operation:

  • collection identifier - Collection identifier for which a description is to be returned.
  • filter - Optional filter expression to select the properties to be filtered.
  • description - Returned collection description containing information according to the filter expression.


A collection in the Repository GE can be deleted with the Delete Collection operation. This operation can only be successful for a requester that has the appropriate authorization. The delete operation requires the identifier of the collection as input. After this operation the collection is no longer accessible for clients. Only one parameter is necessary:

  • collection - Collection identifier for the collection to be deleted


Listing Content

The List operation lists collections and/or resources contained in the Repository GE, which are accessible by the user. This operation usually is needed for a repository browser and maintenance tool as well as an editor tool in order to select the resource to be maintained.

The operations using the following parameters: No input parameters are required. However, for practical reasons it might be useful to restrict the list of collections and resources by specifying the number of results and the starting offset.

  • collection - Collection for which the list is restricted to.
  • index - Index of the first element of the result set to be returned.
  • limit - Maximal number of results to be returned.
  • filter - A repository implementation might also offer the possibility to filter the list of collections according to specific criteria. An optional filter expression can be used to reduce the number of delivered results. A repository may support different criteria to filter the output
  • result list - The operation results in a list of collections/resources that is returned to the client and contains resource descriptions according to the collection, filter, index, and limit expressions.


List the additional services

Besides the operations described above, a repository might provide additional services, such as search, backup, etc. A repository should list and describe the additional services to the clients when the List Services operation is invoked. If additional services are offered only for specific collections of the Repository, the collection identifier can be used to list the actual available functionalities for this collection. The required parameters for this operation are:

  • collection - Collection identifier for which the additional services will be listed.
  • result list - List of additional services are returned to the client.


Searching the Repository

A Repository might provide searching service, to search service descriptions according to the occurrence search terms in properties of the description and media content. It is desirable that in compliance to OpenSearch the repository provides an OpenSearch description to the search API.

Querying the Repository

The Repository GE also provides a more complex querying service in a specific query language. As an example a query service based on SPARQL [ REP2 ]would allow to execute complex queries on the Linked Data RDF model [ REP1 ].

The following diagrams shows an example sequence of operations to perform queries using SPARQL:

Example sequences of smart query operations

The Get Query operation is a service used to obtain Linked USDL data of the Repository GE by using a query that has less than 2054 characters. The following parameters can be involved in the operation:

  • query - Query to be executed.
  • media type - Media type of the Linked Data will be returned.
  • result RDF - Result obtained of executing the query.


The Post Query operation is a service used to obtain Linked USDL data of the Repository GE by using a query without size restriction.The following parameters can be involved in the operation:

  • query - Query to be executed.
  • media type - Media type of the Linked Data will be returned.
  • result RDF - Result obtained of executing the query.

Basic Design Principles

Rationale

There are many proprietary solutions implementing repository functionality and also many standards for various types of repositories. Within FIWARE we try to abstract this functionality into a Generic Enabler.

Implementation agnostic

The API abstracts from the concrete implementation technology. Implementations using various kinds of databases should be possible. Although the main goal is to store services descriptions in a distributed environment, any implementation of the repository GE can be used as long as the technical interfaces comply with the GE operation protocol and can be mapped (mediated) to the FIWARE preferred REST-based reference implementation.

References

REP1 RDF Primer W3C Recommendation 10 February 2004 (http://www.w3.org/TR/2004/REC-rdf-primer-20040210/)
REP2 SPARQL 1.1 Overview, W3C Working Draft 01 May 2012 (http://www.w3.org/TR/2012/WD-sparql11-overview-20120501/9


Re-utilised Technologies/Specifications

The Repository GE is based on RESTful Design Principles. The technologies and specifications used in this GE are:

  • RESTful web services
  • HTTP/1.1
  • JSON and XML data serialization formats
  • Linked USDL

Detailed Specification

Terms and definitions

This section comprises a summary of terms and definitions introduced during the previous sections. It intends to establish a vocabulary that will be help to carry out discussions internally and with third parties. For a summary of terms and definitions managed at overall FIWARE level, please refer to FIWARE Global Terms and Definitions


  • Aggregator (Role): A Role that supports domain specialists and third-parties in aggregating services and apps for new and unforeseen opportunities and needs. It does so by providing the dedicated tooling for aggregating services at different levels: UI, service operation, business process or business object levels.
  • Application: Applications in FIWARE are composite services that have a IT supported interaction interface (user interface). In most cases consumers do not buy the application, instead they buy the right to use the application (user license).
  • Broker (Role): The business network’s central point of service access, being used to expose services from providers that are delivered through the Broker’s service delivery functionality. The broker is the central instance for enabling monetization.
  • Business Element: Core element of a business model, such as pricing models, revenue sharing models, promotions, SLAs, etc.
  • Business Framework: Set of concepts and assets responsible for supporting the implementation of innovative business models in a flexible way.
  • Business Model: Strategy and approach that defines how a particular service/application is supposed to generate revenue and profit. Therefore, a Business Model can be implemented as a set of business elements which can be combined and customized in a flexible way and in accordance to business and market requirements and other characteristics.
  • Business Process: Set of related and structured activities producing a specific service or product, thereby achieving one or more business objectives. An operational business process clearly defines the roles and tasks of all involved parties inside an organization to achieve one specific goal.
  • Business Role: Set of responsibilities and tasks that can be assigned to concrete business role owners, such as a human being or a software component.
  • Channel: Resources through which services are accessed by end users. Examples for well-known channels are Web sites/portals, web-based brokers (like iTunes, eBay and Amazon), social networks (like Facebook, LinkedIn and MySpace), mobile channels (Android, iOS) and work centers. The access mode to these channels is governed by technical channels like the Web, mobile devices and voice response, where each of these channels requires its own specific workflow.
  • Channel Maker (Role): Supports parties in creating outlets (the Channels) through which services are consumed, i.e. Web sites, social networks or mobile platforms. The Channel Maker interacts with the Broker for discovery of services during the process of creating or updating channel specifications as well as for storing channel specifications and channeled service constraints in the Broker.
  • Composite Service (composition): Executable composition of business back-end MACs (see MAC definition later in this list). Common composite services are either orchestrated or choreographed. Orchestrated compositions are defined by a centralized control flow managed by a unique process that orchestrates all the interactions (according to the control flow) between the external services that participate in the composition. Choreographed compositions do not have a centralized process, thus the services participating in the composition autonomously coordinate each other according to some specified coordination rules. Backend compositions are executed in dedicated process execution engines. Target users of tools for creating Composites Services are technical users with algorithmic and process management skills.
  • Consumer (Role): Actor who searches for and consumes particular business functionality exposed on the Web as a service/application that satisfies her own needs.
  • Desktop Environment: Multi-channel client platform enabling users to access and use their applications and services.
  • Front-end/Back-end Composition: Front-end compositions define a front-end application as an aggregation of visual mashable application pieces (named as widgets, gadgets, portlets, etc.) and back-end services. Front-end compositions interact with end-users, in the sense that front-end compositions consume data provided by the end-users and provide data to them. Thus the front-end composition (or mashup) will have a direct influence on the application look and feel; every component will add a new user interaction feature. Back-end compositions define a back-end business service (also known as process) as an aggregation of backend services as defined for service composition term, the end-user being oblivious to the composition process. While back-end components represent atomization of business logic and information processing, front-end components represent atomization of information presentation and user interaction.
  • Gateway (Role): The Gateway role enables linking between separate systems and services, allowing them to exchange information in a controlled way despite different technologies and authoritative realms. A Gateway provides interoperability solutions for other applications, including data mapping as well as run-time data store-forward and message translation. Gateway services are advertised through the Broker, allowing providers and aggregators to search for candidate gateway services for interface adaptation to particular message standards. The Mediation is the central generic enabler. Other important functionalities are eventing, dispatching, security, connectors and integration adaptors, configuration, and change propagation.
  • Hoster (Role): Allows the various infrastructure services in cloud environments to be leveraged as part of provisioning an application in a business network. A service can be deployed onto a specific cloud using the Hoster’s interface. This enables service providers to re-host services and applications from their on-premise environments to cloud-based, on-demand environments to attract new users at much lower cost.
  • Marketplace: Part of the business framework providing means for service providers, to publish their service offerings, and means for service consumers, to compare and select a specific service implementation. A marketplace can offer services from different stores and thus different service providers. The actual buying of a specific service is handled by the related service store.
  • Mashup: Executable composition of front-end MACs. There are several kinds of mashups, depending on the technique of composition (spatial rearrangement, wiring, piping, etc.) and the MACs used. They are called application mashups when applications are composed to build new applications and services/data mash-ups if services are composed to generate new services. While composite service is a common term in backend services implementing business processes, the term ‘mashup’ is widely adopted when referring to Web resources (data, services and applications). Front-end compositions heavily depend on the available device environment (including the chosen presentation channels). Target users of mashup platforms are typically users without technical or programming expertise.
  • Mashable Application Component (MAC): Functional entity able to be consumed executed or combined. Usually this applies to components that will offer not only their main behaviour but also the necessary functionality to allow further compositions with other components. It is envisioned that MACs will offer access, through applications and/or services, to any available FIWARE resource or functionality, including gadgets, services, data sources, content, and things. Alternatively, it can be denoted as ‘service component’ or ‘application component’.
  • Monetization: Process or activity to provide a product (in this context: a service) in exchange for money. The Provider publishes certain functionality and makes it available through the Broker. The service access by the Consumer is being accounted, according to the underlying business model, and the resulting revenue is shared across the involved service providers.
  • Premise (Role): On-Premise operators provide in-house or on-site solutions, which are used within a company (such as ERP) or are offered to business partners under specific terms and conditions. These systems and services are to be regarded as external and legacy to the FIWARE platform, because they do not conform to the architecture and API specifications of FIWARE. They will only be accessible to FIWARE services and applications through the Gateway.
  • Prosumer: A user role able to produce, share and consume their own products and modify/adapt products made by others.
  • Provider (Role): Actor who publishes and offers (provides) certain business functionality on the Web through a service/application endpoint. This role also takes care of maintaining this business functionality.
  • Registry and Repository: Generic enablers that able to store models and configuration information along with all the necessary meta-information to enable searching, social search, recommendation and browsing, so end users as well as services are able to easily find what they need.
  • Revenue Settlement: Process of transferring the actual charges for specific service consumption from the consumer to the service provider.
  • Revenue Sharing: Process of splitting the charges of particular service consumption between the parties providing the specific service (composition) according to a specified revenue sharing model.
  • Service: We use the term service in a very general sense. A service is a means of delivering value to customers by facilitating outcomes customers want to achieve without the ownership of specific costs and risks. Services could be supported by IT. In this case we say that the interaction with the service provider is through a technical interface (for instance a mobile app user interface or a Web service). Applications could be seen as such IT supported Services that often are also composite services.
  • Service Composition: in SOA domain, a service composition is an added value service created by aggregation of existing third party services, according to some predefined work and data flow. Aggregated services provide specialized business functionality, on which the service composition functionality has been split down.
  • Service Delivery Framework: Service Delivery Framework (or Service Delivery Platform (SDP)) refers to a set of components that provide service delivery functionality (such as service creation, session control & protocols) for a type of service. In the context of FIWARE, it is defined as a set of functional building blocks and tools to (1) manage the lifecycle of software services, (2) creating new services by creating service compositions and mashups, (3) providing means for publishing services through different channels on different platforms, (4) offering marketplaces and stores for monetizing available services and (5) sharing the service revenues between the involved service providers.
  • Service Level Agreement (SLA): A service level agreement is a legally binding and formally defined service contract, between a service provider and a service consumer, specifying the contracted qualitative aspects of a specific service (e.g. performance, security, privacy, availability or redundancy). In other words, SLAs not only specify that the provider will just deliver some service, but that this service will also be delivered on time, at a given price, and with money back if the pledge is broken.
  • Store: Part of the Business Framework, offering a set of services that are published to a selected set of marketplaces. The store thereby holds the service portfolio of a specific service provider. In case a specific service is purchased on a service marketplace, the service store handles the actual buying of a specific service (as a financial business transaction).
  • Unified Service Description Language (USDL): USDL is a platform-neutral language for describing services, covering a variety of service types, such as purely human services, transactional services, informational services, software components, digital media, platform services and infrastructure services. The core set of language modules offers the specification of functional and technical service properties, legal and financial aspects, service levels, interaction information and corresponding participants. USDL is offering extension points for the derivation of domain-specific service description languages by extending or changing the available language modules.


Personal tools
Create a book