We use proprietary and third party's cookies to improve your experience and our services, identifying your Internet Browsing preferences on our website; develop analytic activities and display advertising based on your preferences. If you keep browsing, you accept its use. You can get more information on our Cookie Policy
Cookies Policy
StreamOriented Open API Specification - FIWARE Forge Wiki

StreamOriented Open API Specification

From FIWARE Forge Wiki

Jump to: navigation, search

Contents

Legal Notice

Please check the Legal Notice to understand the rights to use FIWARE Open Specifications.

Introduction

The Stream Oriented API is a resource-oriented API accessed via HTTP/HTTPS that uses JSON-RPC V2.0 based representations for information exchange. This document describes the API exposed by the Application Server as defined in the GE Architecture Description.

Intended Audience

This specification is intended for both software developers and implementors of this GE. For developers, this document details the API to build interactive multimedia applications compliant with the GE Architecture Description. Implementors can build their GEi APIs based on the information contained in this specification.

Before reading this document it is recommended to read first the GE Architecture Description and the Programmers Guide. Moreover, the reader should be also familiar with:

API General Features

Authentication

Stream-oriented GE Open API can be offered in two modes: with or without authentication. When authentication is enabled, each request requires the inclusion of an access token. This access token will be granted to the client browser by the IdM GE, as described in the IdM GE Specification. To obtain the token, the client application will redirect the client browser to the IdM GE to perform the authentication and authorization steps of OAuth2. The token thus obtained must be included when stabilising Open API websocket connection to server.

Representation Transport

Resource representation may be transmitted between client and server by using directly HTTP 1.1 protocol, as defined by IETF RFC-2616 or through a WebSocket transport, as defined by IETF RFC-6455. Each time an HTTP request contains payload, a Content-Type header shall be used to specify the MIME type of wrapped representation. In addition, both client and server may use as many HTTP headers as they consider necessary.

Representation Format

Stream Oriented API supports JSON as representation format for request and response parameters following the recommendations in the proposal JSON-RPC over HTTP.

When using HTTP 1.1 transports, the format of the requests is specified by using the Content-Type header with a value of application/json-rpc and is required for requests containing a body. The format required for the response is specified in the request by setting the Accept header to the value application/json-rpc, that is, request and response bodies are serialized using the same format.

Request object

An RPC call is represented by sending a Request object to a server. The Request object has the following members:

  • jsonrpc: a string specifying the version of the JSON-RPC protocol. It must be exactly "2.0".
  • method: a string containing the name of the method to be invoked.
  • params: a structured value that holds the parameter values to be used during the invocation of the method.
  • id: an identifier established by the client that contains a string or number. The server must reply with the same value in the Response object. This member is used to correlate the context between both objects.

Successful Response object

When an RPC call is made the server replies with a Response object. In the case of a successful response, the Response object will contain the following members:

  • jsonrpc: a string specifying the version of the JSON-RPC protocol. It must be exactly "2.0".
  • result: its value is determined by the method invoked on the server. In case the connection is rejected, it's returned an object with a rejected attribute containing an object with a code and message attributes with the reason why the session was not accepted, and no sessionId is defined.
  • id: this member is mandatory and it must match the value of the id member in the Request object.

Error Response object

When an RPC call is made the server replies with a Response object. In the case of an error response, the Response object will contain the following members:

  • jsonrpc: a string specifying the version of the JSON-RPC protocol. It must be exactly "2.0".
  • error: an object describing the error through the following members:
    • code: an integer number that indicates the error type that occurred.
    • message: a string providing a short description of the error.
    • data: a primitive or structured value that contains additional information about the error. It may be omitted. The value of this member is defined by the server.
  • id: this member is mandatory and it must match the value of the id member in the Request object. If there was an error in detecting the id in the Request object (e.g. Parse Error/Invalid Request), it equals to null.

Limits

Media processing is very CPU intensive and therefore the developer should be aware that the creation of multiple simultaneous sessions can exhaust server resources. Thus, limits are not specified for 1.0.0 version of the Stream Oriented GE.

Extensions

Querying extensions is not supported in current version of the Stream Oriented GE.

API Specification

The Stream Oriented GE Open API provides full control of the Media Server through Media Elements, which are the building blocks providing a specific media functionality. They are used to send, receive, process and transform media. This API provides a toolbox of Media Elements ready to be used. It also provides the capability of creating Media Pipelines by joining Media Elements of the toolbox.

The API requires full-duplex communications between client and server infrastructure. For this reason, is is based on WebSocket transports.

Previous to issuing commands, the Stream Oriented GE Open API client requires establishing a WebSocket connection with the server infrastructure.

Verb URI Description
NA ws://{SERVER_IP}:{SERVER_PORT}/kurento Establishment of WebSocket connection for the exchange of full-duplex JSON-RPC messages.

Once the WebSocket has been established, the Stream Oriented GE Open API offers five different types of request/response messages:

  • create: Instantiates a new pipeline or media element in the media server.
  • invoke: Calls a method of an existing media element.
  • subscribe: Creates a subscription to a media event in a media element.
  • unsubscribe: Removes an existing subscription to a media event.
  • release: Explicit termination of a media element.

The Stream Oriented GE API allows to servers send requests to clients:

  • onEvent: This request is sent from server to clients when a media event occurs.

Create

Create message requests the creation of an element of the toolbox. The parameter type specifies the type of the object to be created. The parameter creationParams contains all the information needed to create the object. Each object type needs different creationParams to create the object. These parameters are defined later in this document. Finally, a sessionId parameter is included with the identifier of the current session. The value of this parameter is sent by the server to the client in each response to the client. Only the first request from client to server is allowed to not include the sessionId (because at this point is unknown for the client).

The following example shows a Request object requesting the creation of an object of the type PlayerEndPoint within the pipeline 6829986 and uri http://host/app/video.mp4 in the session c93e5bf0-4fd0-4888-9411-765ff5d89b93:

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "create",
  "params": {
    "type": "PlayerEndPoint",
    "creationParams": {
      "pipeline": "6829986",
      "uri": "http://host/app/video.mp4"
    },
    "sessionId": "c93e5bf0-4fd0-4888-9411-765ff5d89b93"
  }
}  

The Response object contains the object id of the new object in the field value. This object id has to be used in other requests of the protocol (as we will describe later). As stated before, the sessionId is also returned in each response.

The following example shows a typical response to a create message:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "value": "442352747",
    "sessionId": "c93e5bf0-4fd0-4888-9411-765ff5d89b93"
  }
}

Invoke

Invoke message requests the invocation of an operation in the specified object. The parameter object indicates the id of the object in which the operation will be invoked. The parameter operation carries the name of the operation to be executed. Finally, the parameter operationParams has the parameters needed to execute the operation. The object specified has to understand the operation name and parameters. Later in this document it is described the valid operations for all object types.

The following example shows a Request object requesting the invocation of the operation connect on the object 442352747 with parameter sink 6829986. The sessionId is also included as is mandatory for all requests in the session (except the first one).

{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "invoke",
  "params": {
    "object": "442352747",
    "operation": "connect",
    "operationParams": {
      "sink": "6829986"
    },
    "sessionId": "c93e5bf0-4fd0-4888-9411-765ff5d89b93"
  }
}

The Response object contains the value returned while executing the operation invoked in the object or nothing if the operation doesn’t return any value.

The following example shows a typical response while invoking the operation connect (that doesn’t return anything):

{
  "jsonrpc": "2.0",
  "id": 2,
  "result": {
    "sessionId": "c93e5bf0-4fd0-4888-9411-765ff5d89b93"
  }
}

Release

Release message requests the release of the specified object. The parameter object indicates the id of the object to be released.

{    
  "jsonrpc": "2.0",
  "id": 3,
  "method": "release",
  "params": {
    "object": "442352747",
    "sessionId": "c93e5bf0-4fd0-4888-9411-765ff5d89b93"
  }
}

The Response object only contains the sessionID. The following example shows the typical response of a release request:

{
  "jsonrpc":"2.0",
  "id": 3,
  "result":
  {
    "sessionId":"c93e5bf0-4fd0-4888-9411-765ff5d89b93"
  }
}

Subscribe

Subscribe message requests the subscription to a certain kind of events in the specified object. The parameter object indicates the id of the object to subscribe for events. The parameter type specifies the type of the events. If a client is subscribed for a certain type of events in an object, each time an event is fired in this object, a request with method onEvent is sent to the client. This kind of request is described few sections later.

The following example shows a Request object requesting the subscription of the event type EndOfStream on the object 311861480. The sessionId is also included.

{
  "jsonrpc":"2.0",
  "id": 4,
  "method":"subscribe",
  "params":{
    "object":"311861480",
    "type":"EndOfStream",
    "sessionId":"c93e5bf0-4fd0-4888-9411-765ff5d89b93"
  }
}

The Response object contains the subscription identifier. This value can be used later to remove this subscription.

The following example shows the response of subscription request. The value attribute contains the subscription id:

{
  "jsonrpc":"2.0",
  "id": 4,
  "result":
  {
    "value":"353be312-b7f1-4768-9117-5c2f5a087429",
    "sessionId":"c93e5bf0-4fd0-4888-9411-765ff5d89b93"
  }
}

Unsubscribe

Unsubscribe message requests the cancelation of a previous event subscription. The parameter subscription contains the subscription id received from the server when the subscription was created.

The following example shows a Request object requesting the cancelation of the subscription 353be312-b7f1-4768-9117-5c2f5a087429.

{
  "jsonrpc":"2.0",
  "id": 5,
  "method":"unsubscribe",
  "params":{
    "subscription":"353be312-b7f1-4768-9117-5c2f5a087429",
    "sessionId":"c93e5bf0-4fd0-4888-9411-765ff5d89b93"
  }
}

The Response object only contains the sessionID. The following example shows the typical response of an unsubscription request:

{
  "jsonrpc":"2.0",
  "id": 5,
  "result":
  {
    "sessionId":"c93e5bf0-4fd0-4888-9411-765ff5d89b93"
  }
}

onEvent

When a client is subscribed to a type of events in an object, the server send an onEvent notification each time an event of that type is fired in the incumbent object. This is possible because the API is implemented with websockets and there is a full duplex channel between client and server. The notification that server send to client has all the information about the event:

  • data: Information about this specific of this type of event.
  • source: the object source of the event.
  • type: The type of the event.
  • subscription: subscription id for which the event is fired.

The following example shows a notification sent for server to client to notify an event of type EndOfStream in the object 311861480 with subscription 353be312-b7f1-4768-9117-5c2f5a087429.

{
  "jsonrpc":"2.0",
  "id": 6,
  "method":"onEvent",
  "params":{              
    "value":{
      "data":{
        "source":"311861480",
        "type":"EndOfStream"
      },          
      "object":"311861480",
      "subscription":"353be312-b7f1-4768-9117-5c2f5a087429",
      "type":"EndOfStream",
    },
    "sessionId":"4f5255d5-5695-4e1c-aa2b-722e82db5260"      
  }
}

The Response object does not contain any information. Is only a form of acknowledge message. The following example shows the typical response of an onEvent request:

{
  "jsonrpc":"2.0",
  "id":6,
  "result": ""
}

Error responses

If errors arise processing a request, there is a generic error response, in which an error code and a description message in sent, as follows:

{
  "jsonrpc": "2.0",
  "id": 7,
  "error":
  {
    "code": -32601,
    "message": "Error description"
  }
}

Media Element toolbox

The Media Element toolbox provided by Stream Oriented GE Open API is divided into Endpoints, Filters and Hubs.

  • Endpoint offers capabilities to work with protocols and codecs (HttpPostEndpoint, RtpEndpoint and WebRtcEnpoint) and also media repository handling (PlayerEndpoint and RecorderEnpoint).
  • Filters are responsible of media processing, such as computer vision and augmented reality (Face detection, Bar and QR code reading, generic GStreamer filter).
  • Hubs offer capabilities to connect several inputs and outputs and create different types of group communication between them (Composite, Dispatcher, DispatcherOneToMany).

Therefore, the Open API protocol specification provides capabilities to create and handle these Media Elements. The following table shows a description at a glance of the Media Elements provided by Stream Oriented GE Open API.

Type

Capability

Name

Icon

Description

Endpoint

Protocols and Codecs

WebRtcEndpoint

WebRtcEdnpoint

This Endpoint offers media streaming using WebRTC.

RtpEndpoint

RtpEndpoint

Endpoint that provides bidirectional content delivery capabilities with remote networked peers through RTP protocol. It contains paired sink and source MediaPad for audio and video.

HttpPostEndpoint

HttpPostEndpoint

This type of Endpoint provide unidirectional communications. Its MediaSource are related to HTTP POST method. It contains sink MediaPad for audio and video, which provide access to an HTTP file upload function.

Media Repository

PlayerEndpoint

PlayerEndpoint

It provides function to retrieve contents from seekable sources in reliable mode (does not discard media information) and inject them into KMS. It contains one MediaSource for each media type detected.

RecorderEndpoint

RecorderEndpoint

Provides function to store contents in reliable mode (doesn't discard data). It contains MediaSink pads for audio and video.

Filter

Computer Vision & Augmented Reality

FaceOverlayFilter

FaceOverlayFilter

It detects faces in a video feed. The face is then overlaid with an image.

ZBarFilter

ZBarFilter

This Filter detects QR and bar codes in a video feed. When a code is found, the filter raises a CodeFound.

GStreamerFilter

GStreamerFilter

This is a generic Filter interface, that creates GStreamer filters in the media server.

Hub

Group Communication

Composite

Composite

A Hub that mixes the audio stream of its connected sources and constructs a grid with the video streams of its connected sources into its sink.

Dispatcher

Dispatcher

A Hub that allows routing between arbitrary port pairs.

DispatcherOneToMany

DispatcherOneToMany

A Hub that sends a given source to all the connected sinks.

Personal tools
Create a book