CODEWITHSUNDEEP
elasticsearchfiware-cygnus
Charlotte
Charlotte

Reputation: 93

FIWARE Cygnus sink to Elasticsearch/Kibana ?

I'm currently having a working flow :

Fiware Orion -> Fiware Cygnus -> Kafka -> Logstash -> Elasticsearch -> Kibana

I would like to push directly data from Cygnus to Elasticsearch, is there a sink available already ?

An Apache Flume/Elasticsearch sink already exist : https://flume.apache.org/releases/content/1.3.0/apidocs/org/apache/flume/sink/elasticsearch/ElasticSearchSink.html

I was wondering if it would be easy to use it for Cygnus ?

Upvotes: 1

Views: 408

Answers (1)

frb
frb

Reputation: 3798

Until Cygnus 1.5.0 (included) such a sink could be perfectly used (as any other Flume sink) in a Cygnus agent configuration.

From 1.6.0 (included, this is the last version) you will not be able since we internally replaced the usage of native Event objects with custom NGSIEvent ones. Why?:

  • An Event is a set of headers and an array of raw bytes for the body.
  • NGSIEvent inherits from Eventand is a set of headers, an already parsed version of the body (as an object) and an array of raw bytes for the body pointing to null (this last part is the one avoiding compatibility with native Flume sinks).

Anyway, this is "easy" to fix: new version a NGSIEvent will containg both the parsed version of the body and the body itself as raw bytes.

Upvotes: 1

Related Questions