CODEWITHSUNDEEP
elasticsearchlogstash
bschaeffer
bschaeffer

Reputation: 2904

How to set document _id and _source with logstash and elasticsearch

I have a file where each line is a JSON string like so:

{"_id":171962,"doc":{"account_id":53,"email":"[email protected]"}}

And the following logstash config:

input {
  s3 {
    codec => json_lines
  }
}

output {
  elasticsearch {
    doc_as_upsert => true
    action => "update"
    document_id => "%{[_id]}"
  }
  stdout { 
    codec => json_lines 
  }
}

However, I am getting the following error:

Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters.

What I really want is to set the document id from _id and just set everything under doc property in the JSON as the document.

What configuration am I missing that will enable me to do this?

Upvotes: 1

Views: 3599

Answers (1)

nitzien
nitzien

Reputation: 1267

Have you tried this. 

 document_id => "%{_id}"

Also, you may want to use upsert instead of update if this is not really always update.

Upvotes: 2

Related Questions