CODEWITHSUNDEEP
jsonelasticsearchcurlkibanalogfile
mat's
mat's

Reputation: 49

import .log file in elasticsearch and Kibana

I have this .log file with JSON and it looks like this

    {"method":"GET","path":"/public/index","format":"html","controller":"public","action":"index","status":200,"duration":84.59,"view":33.7,"db":47.45,"ip":"127.0.0.1","route":"public#index","request_id":"4d7016832294bafa8f593453eed2adb1","source":"unknown","tags":["request"],"@timestamp":"2018-11-09T22:54:06Z","@version":"1"}
{"method":"GET","path":"/public/index","format":"html","controller":"public","action":"index","status":200,"duration":15.44,"view":13.85,"db":0.91,"ip":null,"route":null,"request_id":null,"source":"unknown","tags":["request"],"@timestamp":"2018-11-09T22:54:28Z","@version":"1"}
{"method":"GET","path":"/public/index","format":"html","controller":"public","action":"index","status":200,"duration":13.86,"view":12.47,"db":0.8,"ip":null,"route":null,"request_id":null,"source":"unknown","tags":["request"],"@timestamp":"2018-11-09T22:54:40Z","@version":"1"}

and i try to import this in elasticsearch and Kibana. I tried with this code in command line 

curl -XPOST "http://localhost:9200/test/test" -H "Content-Type: application/json" -d @logfile.log

and i get this error 

{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse"}],"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"illegal_argument_exception","reason":"Malformed content, found extra data after parsing: START_OBJECT"}},"status":400}

and when i look in Kibana I see this

this is what i have in Kibana

what i am doing wrong?

Upvotes: 0

Views: 1563

Answers (1)

ibexit
ibexit

Reputation: 3667

The bulk format is

action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....

You have all the optional_source lines in your file already.

Just add a line containing { "index" : { "_index" : "YOUR-INDEX-NAME", "_type" : "_doc"} } before each of your lines.

Then POST against the bulk api in ES https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html

Upvotes: 1

Related Questions