We use proprietary and third party's cookies to improve your experience and our services, identifying your Internet Browsing preferences on our website; develop analytic activities and display advertising based on your preferences. If you keep browsing, you accept its use. You can get more information on our Cookie Policy
Cookies Policy
Advanced Middleware and Web UI Architecture - FIWARE Forge Wiki

Advanced Middleware and Web UI Architecture

From FIWARE Forge Wiki

Jump to: navigation, search


FIWARE WIKI editorial remark:
This page corresponds to Release 3 of FIWARE. The latest version associated to the latest Release is linked from FIWARE Architecture


Note: Parts of Advanced Middleware and WebUI continue in the scope of different chapters since FIWARE Release 4. Please refer to the updated architecture pages here:

The Advanced Middleware and Web User Interface (UI) Architecture chapter (aka. as MiWi) of FIWARE offers Generic Enablers from two different but related areas:

Advanced Middleware Architecture
The high-performance middleware that is backward compatible with traditional Web services (e.g. REST) but offers advanced features and performance and dynamically adapts to the communication partners and its environment. A novel API separates WHAT data needs to be communicated from WHERE the data come from within the native data structures of the application, and HOW the data should be transmitted to the target. Additionally, the middleware offers "Security by Design" through a declarative API, where the application defines the security requirements and policies that apply to its data, which are then automatically enforced by the middleware. The middleware uses of the security functionality offered by the Security chapter of FIWARE.
Advanced Web-based User Interface (Web UI) Architecture
In order to become widely visible and adopted by end users, the FIWARE Future Internet platform must not only offer server functionality but must also offer much improved user experiences. The objective is to significantly improve the user experience for the Future Internet by adding new user input and interaction capabilities, such as interactive 3D graphics, immersive interaction with the real and virtual world (Augmented Reality), virtualizing and thus separating the display from the (mobile) computing device for ubiquitous operations, and many more. The technology is based on the Web technology stack, as the Web is quickly becoming THE user interface technology supported on essentially any (mobile) device while already offering advanced rich media capabilities (e.g. well-formatted text, images, video). First devices are becoming available that use Web technology even as the ONLY user interface technology. The Web design and programming environment is well-known to millions of developers that allow quick uptake of new technology, while offering a proven model for continuous and open innovation and improvement.

These two areas have been combined because highly interactive (2D/3D) user interfaces making use of service oriented architectures have strong latency, bandwidth, and performance requirements regarding the middleware implementations. An example is the synchronization service for real-time shared virtual worlds or the machine control on a factory floor that must use the underlying network and computing hardware as efficiently as possible. Generic Enables are provided to make optimal use of the underlying hardware via Software Defined Networking in the Middleware (SDN, using the GEs of the Interface to Networks and Devices (I2ND) Architecture chapter) and the Hardware Support in the 3D-UI GE (see FIWARE.ArchitectureDescription.MiWi.3D-UI) that provides access to GPU and other parallel compute functionality that may be available.

Architecture Overview

The following diagram shows the main components (Generic Enablers) that comprise FIWARE Advanced Middleware and Web-based User Interfaces chapter architecture.

MiWi Architecture Overview
Advanced Middleware and Web UI Architecture Overview

Advanced Middleware

The Advanced Middleware GE provides a software layer between the application and the communication network and allows application to abstract from the intricacies of how to send a piece of data to a service offered by a another application and possibly return results. The middleware offers functionality to find and establish a connection to a service, negotiate the best wire and transport protocols, access the applications native data structures and encode the necessary data in a format suitable for the chosen protocol, and finally send that data and possibly receive results in return. In a similar way an application can use the middleware also to offer services to other applications by registering suitable service functionality and interfaces, which can then be used as targets of communication.

The Advanced Middleware GE architecture offers a number of key advantages over previous middleware implementation:

  • High-Level Service Architecture: It offers applications a high-level architecture that can shield them from the complexities and dangers of network programming. When applications declare services and data structures they can annotate them with the QoS, security, and other requirements while the Advanced Middleware GE automatically implements them. Thus applications can exclusively focus on the application functionality.
  • Security: The network is the main security threat to most applications today but existing middleware has offered only limited security functionality that has often been added as an afterthought and requires the application developer (who are often not security experts) to configure the security functionality. Instead, the Advanced Middleware GE offers Security by Design where security has been designed into the architecture from the start. Applications can simply declare their security needs in the form of security policies (security rules) and apply them to data structures and service at development time or even later during deployment definitions. It then makes sure that these requirements are met before any communication takes place and applies any suitable security measures (e.g. encryption, signatures, etc.) during the communication.
  • High-performance: The Advanced Middleware API has been designed as to allow for the highest possible communication performance. Besides the common networking technologies like TCP/IP, this also includes the option of doing Remote DMA (RDMA) directly between in-memory data structures thereby completely eliminate the OS and network stack from the data communication on some modern networks (e.g. Infiniband), the use of shared memory on the same machine, or even the use of direct function calls for services within the same address space (e.g. via plug-ins).
  • Dynamic Multi-Protocol support: The Advanced Middleware GE can select at run-time the best way to communicate with a remote service. Thus, an application can simultaneously talk with legacy services via their predefined protocols (e.g. DDS, REST) while able to take advantage of higher performance functionality when talking to other services using the Advanced Middleware GE.
  • QoS and Software Defined networking: Where possible the QoS annotations are also used to configure the network using modern Software Defined networking functionality, e.g. to reserve bandwidth.

Advanced Web UI Architecture

The Advanced Web-based User Interface (Advanced Web-UI) architecture consists of four main groups of GEs that together provide a comprehensive set of services for application to implement advanced, highly-interactive 2D and 3D user interfaces.

The entire Advanced Web-UI components are based on HTML and other standard web technologies as implemented by any modern Web browser. With the fast development over the last few years, the web has become the main application and UI platform. We leverage this development and extend HTML with a number of new capabilities that can be used to significantly enhance the UI capabilities and improve the user experience.

  • Client Core: The Client Core module provides the core functionality for creating HTML based user interfaces. These GEs run within the Web browser but can make use of external Web services where necessary (e.g. through the adv. middleware GE).
    • The 2D-UI GE enhances HTML with some additional functionality that improves the development of advanced user interfaces. It makes use of WebComponents and ShadowDOM technology to encapsulate functionality and provide user interface elements in an object-oriented form. In addition it will provide ways to handle a larger variety of input devices as necessary for 3D functionality.
    • The 3D-UI GE adds new HTML elements (using a polyfill implementation) for describing 3D scenes, including geometry, material, textures, lights, and cameras. A new functional dataflow mechanism is used to enable interactive animations, image processing, augmented reality, and other dynamic elements to a scene.
  • Server Core: The Server Core module mainly provides a scalable synchronization server.
    • The Synchronization GE allows multiple Advanced Web-UI instances running on different clients to synchronize in real-time. While this is mandatory for providing shared 3D environments (e.g. for games) it can also be applied for 2D functionality. A key element will be the use of a highly flexible and highly scalable design of the synchronization server architecture that will enable the dynamic repartitioning of the 3D environment for different server functionality like scripting , physics, and others. The AMi middleware will be used to provide optimal network performance for such demanding applications. A SceneAPI is used to offer remote services to connect to a 3D environment and modify it in real-time.
  • Supporting Services: This module provides services that are commonly being used when creating Web-based user interfaces.
    • The Cloud Rendering GE is a service that connects to a synchronization server renders the scene based on a chosen camera view and streams the results using common video streaming functionality (e.g. in FIWARE).
    • The GIS Data Provider GE offers access to 3D GIS data via geo-location queries which can be used by any application to render content in a virtual real world scenario.
    • The POI Data Provider GE provides access to advanced Point of Interest data that can be used to position 2D and 3D content in the context of a 3D scene. Besides the usual meta data the POI data specification allows for defining (dynamic) 3D scene elements relative to the POI, attach Augmented Reality markers relative to the POI and the 3D content, and other advanced functionality.
    • The 2D-3DCapture GE allows for capturing data from the real world in the form of images, augmenting them with additional data, and providing the results to other services including those that create 3D structures from these images.
  • Application-oriented Services: This module provides high-level GEs that operate more on the application level in the form of tools.
    • The Augmented Reality (AR) GE builds on top of the 3D-UI GE which provides the hardware-accelerated basic image and computer vision processing that is necessary for augmented reality and provides higher order AR functionality that can be used by applications and other services. This includes the registration and tracking with various forms of markers or features, the setup of such tracking based on data from various sources and in various formats, and finally the setup of the rendering of such tracked features.
    • The Real-Virtual Interaction GE combines AR with the Internet of Things (IoT) in order to monitor the state of sensors and other information sources from the real world, visualizes this state in a 3D world, allows user to interact with these visual elements, and then uses services to act on the real world.
    • The Virtual Characters GE allows application to add and control animated virtual characters in 3D scenes.
    • The Interface Designer GE is an interactive, web-based composition and editing tools that allows users to interactively edit 3D world in the same browser environment that is also used to run an application. It is not a fully 3D content creation tools but depends on pre-existing content that can be added to a scene, can be positioned, and whose parameters and animations can be modified.

Inter-dependencies and Interaction between GEs

Advanced Middleware GE

The Advanced Middleware provides basic Web service-oriented communication services to all other services. It does not depend on any other FIWARE GEs except for implementing its security functionality, where it uses the GEs from the security chapter to e.g. verify identities and obtain security related information (such as security policies) that is then used to enforce the required security automatically within the middleware.

The Advanced Web UI GEs may make use of the middleware GEs for implementing high-performance and secure communication with other FIWARE or other services.

Advanced Web-based User Interface GEs:

These GEs have been grouped into four modules that share different dependencies

  • The Client Core GEs provides the mechanisms to do interactive 2D and 3D rich-media graphics rendering and interaction based on HTML-5 and the Web. These GEs can run completely independently of any other GEs in any modern Web browser.
  • The Server Core GEs provide functionality for real-time synchronization between multiple client instances. It can also provide additional functionality to the clients.
  • The Supporting Service GEs provides additional services mostly to the Server Core GEs but some services may also be used directly from a client without a server for cases where the server may not otherwise be needed.
  • The Application-oriented GEs provide additional functionality to applications or the user directly. They require at least the client GEs but may offer additional features when the client is connected to a server (e.g. persistent scene editing).

The Web UI GEs will typically be used to implement the user interface of Web applications. Most of them will make also use of other FIWARE services either directly from within the client or through the server or any server tier that is running there.

Special Services

Some of the GEs in this chapter will typically not be running on a remote server or cloud setup. Instead they will commonly be deployed directly on an end users (mobile) device (e.g. through Javascript). This is true for example for the 3D-UI GE or the Augmented Reality GE as they need to operate on local data (3D scene data or local camara input) and provide very low-latency, real-time feedback to the user (image display or sound output). While these GEs could in principle also be deployed on remote servers, this would not provide the necessary user experience for interactive applications.

Architecture description of GEs

Middleware GE:

Basic Client GEs:

Basic Server GE:

Supporting Web Service GEs:

Application-oriented GEs:

Interfaces to other Chapters

Cloud Hosting

Advanced Middleware and Web-based UI assets deployment and operations will take benefit of the dynamic resources allocation and monitoring functions supported in the Cloud chapter. In particular, recipes for automated deployment of several GEs in the Advanced Middleware and web-based UI chapter are going to be provided (see PaaS and SDC GEs in the Cloud chapter).

Internet of Things

The middleware may be used to more efficiently communicate with many data sources, sources that transfer frequently or that transfer large amounts of data. It also allows offloading much of the security burden to the middleware.

The Advanced Web UI components are very well suited to dynamically visualize the data from the Internet of Things in user-friendly ways and by taking the context into account, e.g. by embedding the data in its Geographic context GISDataProvider or by using Augmented Reality to visualize the data and interact with it in the context of the real environment of the user.

Data / Context Management

The Advanced Web UI components are very well suited to visualize the data processed by that chapters GEs (e.g. via the BigData or ContextBroker GEs). It also provides the 2D and 3D UI functionality to present and interact in novel ways with the Streaming GE.

The Advanced Middleware GE in turn can be used to more efficiently implement the high-bandwidth or latency-sensitive data transfers often needed by GE from this chapter.

Applications and Services Ecosystem and Delivery Framework

The Advanced Web UI GEs extend the capabilities provided by the Service Mashup and in particular the Application Mashup GEs but can in turn be used by these GEs as building blocks for novel user experiences (e.g. 3D functionality).

Interface to Networks and Devices

The Advanced Middleware GE uses the functionalities provided by the GEs this chapter to implement functionality, like optimal routing, QoS guarantees, or security.


The security chapter of FIWARE provides the functionality for implementing the advanced security features of the Advanced Middleware GE, including for example the necessary identity management (see Access Control Generic Enabler).

Personal tools
Create a book