We use proprietary and third party's cookies to improve your experience and our services, identifying your Internet Browsing preferences on our website; develop analytic activities and display advertising based on your preferences. If you keep browsing, you accept its use. You can get more information on our Cookie Policy
Cookies Policy
FIWARE.OpenSpecification.WebUI.2D-UI - FIWARE Forge Wiki

FIWARE.OpenSpecification.WebUI.2D-UI

From FIWARE Forge Wiki

Jump to: navigation, search
Name FIWARE.OpenSpecification.WebUI.2D-UI
Chapter Advanced Web-based User Interfaces,
Catalogue-Link to Implementation 2D-UI
Owner Adminotech Oy, Antti Kokko

Contents

Preface

Within this document you find a self-contained open specification of a FIWARE generic enabler, please consult as well the FIWARE Product Vision, the website on http://www.fiware.org and similar pages in order to understand the complete context of the FIWARE platform.


Copyright

Legal Notice

Please check the following Legal Notice to understand the rights to use these specifications.

Overview

We will be researching the current state of possible input devices in a modern web browser environment. These input devices range from traditional keyboard and mouse to touch, gamepads and sensors. One of the key parts in this GE is to implement a Input API for the 3D client and scripts utilizing it, to register events and get notified when user interaction happens on the 3D scene and its objects. The research will include APIs that may not be included in browser standards yet but can be found in nightly builds or such beta releases.

The second part of this GE, the UI, we apply the same research for modern technologies and example implementation that will make creating and managing common widgets easier for your 3D application. The research will heavily include the use of new/upcoming web standard Web Components.

Basic Concepts

Input API

Enables registering/unregistering to receive input events fired by e.g. touch pad, gamepad, keyboard and mouse. It also enables registering new input events runtime to extend the functionality. These events must be possible to send to used event system so applications/scripts using the event system can listen the events.

Input Abstraction

Enables registering/unregistering or updating existing input context/state which defines keyboard and mouse conditions that fire an event. E.g. pressing 'w' is forward or pressing 'w' and 'mouse left' is jump. End user is free to decide and name own input states. Within the input state end user can define following bindings or conditions:

  • Name
  • 0-n - keyboard bindings
  • Mouse - left down, middle down, right down
  • Time slot - within all conditions must be true
  • Priority - in which order the registered input states are handled
  • Multiplier - how many times either keyboard or mouse conditions must be repeated within the given time slot.

Web component

Web Components bring a new way to encapsulate your UI, look and feel and your JavaScript functionality into a single component. These components are easy to include and add to any web page without including scripts and dealing with the usual initialization steps for different libraries. Additionally you are interacting with a DOM element that the Web Component defines, not a JavaScript library directly, the Web Component defined its DOM interface and implements the needed JavaScript behind it.

The component model for the Web aka Web component can be understood as a reusable widget/HTML module on screen.

It consists of 5 pieces:

  • Templates - which define chunks of markup that are inert but can be activated for use later.
  • Decorators - which apply templates based on CSS selectors to affect rich visual and behavioral changes to documents.
  • Custom Elements - which let authors define their own elements, with new tag names and new script interfaces.
  • Shadow DOM - which encapsulates a DOM subtree for more reliable composition of user interface elements.
  • Imports - which defines how templates, decorators and custom elements are packaged and loaded as a resource.

Each of these pieces is useful individually. When used in combination, Web Components enable Web application authors to define widgets with a level of visual richness and interactivity not possible with CSS alone, and ease of composition and reuse not possible with script libraries today.

http://www.w3.org/TR/2013/WD-components-intro-20130606/.

Shadow DOM

Shadow DOM encapsulates DOM tree from the main document DOM tree. In other words a widget or web component can be encapsulated from the rest of the page and all e.g. javascript library updates or style changes won´t affect the Shadow DOM tree.

https://dvcs.w3.org/hg/webcomponents/raw-file/tip/spec/shadow/index.html.

Polymer

Polymer implements the concept of the Web Components standard. It is built for modern browsers that support web platform APIs. Support for old browsers is done by polyfills which is a Javascript library that enables the use of the new web platform APIs if they are not yet present in the browser itself. Polymer leverages Web Components, a new set of standards designed to provide reusable components for the web. With Polymer users can create own custom elements and reduces writing the boilerplate markup or code all over again.

http://www.polymer-project.org/

Gamepad API

Gamepad API enables using JavaScript to read the state of any gamepad controller attached to your computer. Please refer to the W3C page for a full description of the Gamepad API specification

Generic Architecture

The 2D-UI Generic Enabler consists of two main building blocks: The Input API on one hand, which will provide an interface to abstract from different input devices, and the Web Component interface on the other, which simplifies the integration of graphic 2D user interfaces into the Web application. While both, input and graphic user interface, are part of this GE, it is explicitly desired that input is also compatible with other GE of the chapter, as for example 3D-UI.

The 2D-UI is implemented as web page in a browser. The web page provides graphical interface for the user and provides support for input devices and input abstraction via Javascript libraries. Polymer is implemented by using Javascript libraries and Polymer components which are either placed on a different html -file and imported on the web page, dynamically loaded or vulcanized as one file where all components, html and javascript are combined in one html -file.

InputAPI

Provides methods to create and compose input events of different devices to create more complex input patterns. Supported devices are:

  • Keyboard
  • Mouse
  • Touch
  • Gamepad (XBOX, Playstation etc.)

Web Components

Web Components, implemented and provided by the Polymer library, are used to predefine templates of 2D Graphic User Interface patterns that can be used to assemble the Web page. The resulting application will provide the Graphic User interface, with which the user interacts in terms of abstract input events, which are realized by the Input API.

Main Interactions

Research and implement simple and easy way to use Input API and Web Components in browser environment. InputAPI should be extendable by 3rd party developers. Once new input devices or UI libraries are exposed to the browser environment we should provide a clear way of extending the InputAPI. To start with the InputAPI we will provide modules/components described in Generic Architecture chapter. To start with Web components we will provide example implementation of a web component.

  • As a test case for InputAPI there will be a simple html page that includes these libraries and print the user input on the screen.
  • As a test case for Input Abstraction there will be a simple html page where user can create own input state/context and see how InputAPI reacts to the created state/context printing the user input on the screen.
  • As a test case for Web Component there will be html pages with different implementation to place web component on the screen.

Basic Design Principles

  • Clear Javascript libraries with samples and documentation
  • Clear html and Javascript files with documentation

References


Detailed Specifications

Following is a list of Open Specifications linked to this Generic Enabler. Specifications labeled as "PRELIMINARY" are considered stable but subject to minor changes derived from lessons learned during last interactions of the development of a first reference implementation planned for the current Major Release of FI-WARE. Specifications labeled as "DRAFT" are planned for future Major Releases of FI-WARE but they are provided for the sake of future users.

Open API Specifications


Re-utilised Technologies/Specifications


Terms and definitions

This section comprises a summary of terms and definitions introduced during the previous sections. It intends to establish a vocabulary that will be help to carry out discussions internally and with third parties (e.g., Use Case projects in the EU FP7 Future Internet PPP). For a summary of terms and definitions managed at overall FI-WARE level, please refer to FIWARE Global Terms and Definitions

Annotations
Annotations refer to non-functional descriptions that are added to declaration of native types, to IDL interface definition, or through global annotations at deployment time. The can be used to express security requirements (e.g. "this string is a password and should be handled according the security policy defined for password"), QoS parameters (e.g. max. latency), or others.
AR
AR → Augmented Reality
Augmented Reality (AR)
Augmented Reality (AR) refers to the real-time enhancement of images of the real world with additional information. This can reach from the rough placement of 2D labels in the image to the perfectly registered display of virtual objects in a scene that are photo-realistically rendered in the context of the real scene (e.g. with respect to lighting and camera noise).
IDL
IDL → Interface Definition Language
Interface Definition Language
Interface Definition Language refers to the specification of interfaces or services. They contain the description of types and function interfaces that use these types for input and output parameters as well as return types. Different types of IDL are being used including CORBA IDL, Thrift IDL, Web Service Description Language (WSDL, for Web Services using SOAP), Web Application Description Language (WADL, for RESTful services), and others.
Middleware
Middleware is a software library that (ideally) handles all network related functionality for an application. This includes the setup of connection between peers, transformation of user data into a common network format and back, handling of security and QoS requirements.
PoI
PoI → Point of Interest
Point of Interest (PoI)
Point of Interest refers to the description of a certain point or 2D/3D region in space. It defines its location, attaches meta data to it, and defines a coordinate system relative to which additional coordinate systems, AR marker, or 3D objects can be placed.
Quality of Service (QoS)
Quality of Service refers to property of a communication channel that are non-functional, such a robustness, guaranteed bandwidth, maximum latency, jitter, and many more.
Real-Virtual Interaction
Real-Virtual Interaction refers to Augmented Reality setup that additionally allow users to interact with real-world objects through virtual proxies in the scene that monitor and visualize the state in the real-world and that can use services to change the state of the real world (e.g. switch lights on an off via a virtual button the the 3D scene).
Scene
A Scene refers to a collection of objects, which are be identified by type (e.g. a 3D mesh object, a physics simulation rigid body, or a script object.) These objects contain typed and named data values (composed of basic types such as integers, floating point numbers and strings) which are referred to as attributes. Scene objects can form a hierarchic (parent-child) structure. A HTML DOM document is one way to represent and store a scene.
Security
Security is a property of an IT system that ensures confidentiality, integrity, and availability of data within the system or during communication over networks. In the context of middleware, it refers to the ability of the middleware to guarantee such properties for the communication channel according to suitably expressed requirements needed and guarantees offer by an application.
Security Policy
Security Policy refers to rules that need to be fulfilled before a network connection is established or for data to be transferred. It can for example express statements about the identity of communication partners, properties assigned to them, the confidentiality measures to be applied to data elements of a communication channel, and others.
Synchronization
Synchronization is the act of transmitting over a network protocol the changes in a scene to participants so that they share a common, real-time perception of the scene. This is crucial to implementing multi-user virtual worlds.
Type Description
Type Description in the context of the AMi middleware refers to the internal description of native data types or the interfaces described by an IDL. It contains data such as the name of a variable, its data type, the hierarchical relations between types (e.g. structs and arrays), its memory offset and alignment within another data type, and others. Type Description are used to generate the mapping of native data types to the data that needs to be transmitted by the middleware.
Virtual Character
Virtual Character is a 3D object, typically composed of triangle mesh geometry, that can be moved and animated and can represent a user's presence (avatar) in a virtual world. Typically supported forms of animation include skeletal animation (where a hierarchy of "bones" or "joints" controls the deformation of the triangle mesh object) and vertex morph animation, where the vertices of the triangle mesh are directly manipulated. Virtual character systems may support composing the character from several mesh parts, for example separate upper body, lower body and head parts, to allow better customization possibilities.
WebGL
WebGL → (Web Graphics Library) is a JavaScript API for rendering 3D and 2D computer graphics in web browser.
Personal tools
Create a book