CODEWITHSUNDEEP
elasticsearchknime
José Antonio Blanco
José Antonio Blanco

Reputation: 13

Load elasticsearch data in knime

I have a database loaded in elasticsearch and I would like to access it from KNIME to be able to make a recommendation system. How can I connect both applications?

Upvotes: 0

Views: 1328

Answers (2)

qqilihq
qqilihq

Reputation: 11454

This is an older topic, and back then there were the REST, Python, or JDBC workarounds. I vaguely remember that the REST way was fiddly because KNIME does not support GET requests with body.

Since a few years there's fortunately now a dedicated commercial extension which supports reading from and writing to Elasticsearch (currently for version 6 and 7).

You can find more information here: https://nodepit.com/product/elasticsearch

Upvotes: 1

Batuhan Kose
Batuhan Kose

Reputation: 101

You can do it with python code in knime. Code is above

from pandas import DataFrame
output_table = DataFrame()

from elasticsearch import Elasticsearch
import json
import pandas as pd

esclient = Elasticsearch(['http://XX.YYYY.ZZZ.TTT:9200'])
response = esclient.search(
index='rtimlog-2018.11.02',
body={
   "query": {
    "match": {
      "message.eventParameters.Msisdn.keyword": {
        "query": "532XXXXX",
      }
    }
  }
}
)

columns=['A']
df = pd.DataFrame( columns=columns)


for hit in response['hits']['hits']: 
    X= json.dumps(hit['_source']['message'])
    df.loc[i]=X

output_table = df

Upvotes: 1

Related Questions