We use proprietary and third party's cookies to improve your experience and our services, identifying your Internet Browsing preferences on our website; develop analytic activities and display advertising based on your preferences. If you keep browsing, you accept its use. You can get more information on our Cookie Policy
Cookies Policy
Unstructured Data Analysis Open RESTful API Specification - FIWARE Forge Wiki

Unstructured Data Analysis Open RESTful API Specification

From FIWARE Forge Wiki

Jump to: navigation, search

Contents

Introduction to the Unstructured Data Analysis API

Please check the FI-WARE Open Specification Legal Notice (implicit patents license) to understand the rights to use FI-WARE Open Specifications.

Unstructured Data Analysis API Core

The Unstructured Data Analysis API is a RESTful, resource-oriented API accessed via HTTP that uses JSON-based representations for information interchange. This API provides the means to effectively manage unstructured data analysis projects and can be used to integrate these functionalities into third party applications.

Intended Audience

This specification is intended for data analysis and visualization developers that need to analyze unstructured web resources. This document provides a full specification of how to interoperate with this GE. To use this information, the reader should firstly have a general understanding of the [1].

API Change History

This version of the Unstructured Data Analysis API Guide replaces and obsoletes all previous versions. The most recent changes are described in the table below:

Revision Date Changes Summary
March 04, 2014 Initial API version

How to Read this Document

All FI-WARE RESTful API specifications will follow the same list of conventions and will support certain common aspects. Please check Common aspects in FI-WARE Open Restful API Specifications. For a description of some terms used along this document, see [2].

Aditional Resources

For more details about the Unstructured Data Analysis GE, please refer to [3] and also to the Architecture description mentioned before.

General Ontology Registry API Information

Resources Summary

Representation Format

The Unstructured Data Analysis API supports JSON based representation formats.

Representation Transport

Resource representation is transmitted between client and server by using HTTP 1.1 protocol, as defined by IETF RFC-2616. Each time an HTTP request contains payload, a Content-Type header shall be used to specify the MIME type of wrapped representation. In addition, both client and server may use as many HTTP headers as they consider necessary.

Resource Identification

This section must explain which would be the resource identification used by the API in order to identify unambiguously the resource. For HTTP transport, this is made using the mechanisms described by HTTP protocol specification as defined by IETF RFC-2616.

Links and References

No additional links or references are provided.


API Operations

The following section provides the detail for each RESTful operation giving the expected input and output for each URI.

Unstructured Data Analysis API Operations

Create/Retrieve/Delete Project configutation

Verb URI Description
POST /{project name} Creates a new Unstructured Data Analysis project
GET /{project name} Get the current configuration of a unstructured data analysis project (name, description, sources) encoded in a JSON format
DELETE /{project name} Delete an unstructured data analysis project

Response codes:

  • HTTP/1.1 200 - The operation has been executed
  • HTTP/1.1 500 - If there are some unidentified error.

Request example (Create Project):

POST /Project HTTP/1.1
Host: localhost:8080
Content-Type: application/x-www-form-urlencoded

Response example:

HTTP/1.1 200 OK
{"message":"OK","success":true}

Request example (Get Project):

GET /Project HTTP/1.1
Host: localhost:8080

Response example:

HTTP/1.1 200 OK
{"id":1,"description":"test channel","name":"Test","sources":[{"id":"bbcnews","name":"bbcnews","url":"http://feeds.bbci.co.uk/news/rss.xml"}]}

Request example (Delete Project):

GET /Project HTTP/1.1
Host: localhost:8080

Response example:

HTTP/1.1 200 OK
{"message":"OK","success":true}

Get Project Sources

Verb URI Description
GET /{Project name}/sources Get the current RSS sources (name, URL, description) of the project encoded using JSON

Response codes:

  • HTTP/1.1 200 - If the sources are successfully retrieved
  • HTTP/1.1 500 - If there are some unidentified error.

Request example:

GET /TestProject/sources HTTP/1.1
Host: localhost:8080

Response example:

HTTP/1.1 200 OK
[{"id":"1","name":"bbcnews","url":"http://feeds.bbci.co.uk/news/rss.xml"}]

Add/Get/Remove RSS sources in a project

Verb URI Description
POST /{Project name}/sources/{Source name} Add a new RSS source to an unstructured data analysis project.
GET /{Project name}/sources/{Source name} Get the source data (url, description).
DELETE /{Project name}/sources/{Source name} Remove the source from the project (no further data will be gathered from there)

Response codes:

  • HTTP/1.1 200 - If the source is successfully processed by the GE
  • HTTP/1.1 500 - If there are some unidentified error.

Request example (Add source):

POST /TestProject/sources/bbc HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: localhost:8080
description=BBC News&url=http://feeds.bbci.co.uk/news/rss.xml

Response example:

HTTP/1.1 200 OK
{"message":"OK","success":true}

Request example (Get source metadata):

GET /TestProject/sources/bbc HTTP/1.1
Host: localhost:8080

Response example:

HTTP/1.1 200 OK
{"id":"1","name":"bbcnews","url":"http://feeds.bbci.co.uk/news/rss.xml"}

Request example (Remove source):

DELETE /TestProject/sources/bbc HTTP/1.1
Host: localhost:8080

Response example:

HTTP/1.1 200 OK
{"message":"OK","success":true}

Search data in a project

Verb URI Description
POST /{Project name}/search Search a specific pattern (following the Lucene query format) into the gathered document of the project.

Response codes:

  • HTTP/1.1 200 - If the search was executed successfully
  • HTTP/1.1 500 - If there are some unidentified error.

Request example:

POST /TestProject/search HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: localhost:8080
query=olympics

Response example:

 HTTP/1.1 200 OK
 [{
       "id": "0a4f5aa106ae09db16d61c45abc2db47",
       "pubDate": "2014-02-27T13:33:32.000",
       "text": "BBC Sport - Sochi 2014: Russia Winter Olympe site Terms of Use About the BBC Privacy Accessibility.....",
       "title": "Wada to act over Russia 'gas' claims",
       "projectid": "1",
       "digest": "1ce7ae0582621568a75dd3bfe6256ae8",
       "url": "http://www.bbc.co.uk/sport/0/winter-olympics/26369004"
   }]
 

Get Trending Terms

Verb URI Description
GET /{Project name}/terms Get the most relevant terms detected in the project between two dates

Response codes:

  • HTTP/1.1 200 - If the terms was retrieved correctly
  • HTTP/1.1 500 - If there are some unidentified error.

Request example:

GET /TestProject/terms?startdate=2013-01-01&enddate=2013-01-01
Host: localhost:8080

Response example:

 HTTP/1.1 200 OK
 [{"term":{"id":"6d9c4e52f18f604187a77eac8944d909","token":"Computer","lemma":"Computer","posTag":"NN"},"date":"2013-01-01","occurrences":1521},
{"term":{"id":"af8939e16b69103e2a063b30ce293dc8","token":"Test","lemma":"Test","posTag":"NN"},"date":"2013-01-01","occurrences":985}]
 

Get Term metadata

Verb URI Description
GET /{Project name}/terms/{term id} Get the metadata of a term

Response codes:

  • HTTP/1.1 200 - If the terms was retrieved correctly
  • HTTP/1.1 500 - If there are some unidentified error.

Request example:

GET /TestProject/terms/6d9c4e52f18f604187a77eac8944d909
Host: localhost:8080

Response example:

 HTTP/1.1 200 OK
 {"id":"6d9c4e52f18f604187a77eac8944d909","token":"Computer","lemma":"Computer","posTag":"NN"}
 

Get Term history

Verb URI Description
GET /{Project name}/terms/{term id}/history Get the occurrence history of a terms between two dates

Response codes:

  • HTTP/1.1 200 - If the terms was retrieved correctly
  • HTTP/1.1 500 - If there are some unidentified error.

Request example:

GET /TestProject/terms/6d9c4e52f18f604187a77eac8944d909/history?stardate=2013-01-01&endate=2013-01-01
Host: localhost:8080

Response example:

 HTTP/1.1 200 OK
 [{"date":"2013-01-01","occurrences":1521}]
 
Personal tools
Create a book