CODEWITHSUNDEEP
jsonelasticsearchlogstashlogstash-configuration
Sneijky
Sneijky

Reputation: 11

logstash extract json field and overwrite index

I have the following json:

[
{
    "key": "Key-1",
    "field1": "hello",
    "field2": "world"
},
{

    "key": "Key-2",
    "field1": "hello",
    "field2": "world"
}
]

and the following logstash file:

input {
  http_poller {
    # List of urls to hit
    # URLs can either have a simple format for a get request
    # Or use more complex HTTP features
    urls => {
      myurl => {
        method => "GET"
        url => "http://localhost:8080/helloworld"
      }
    }
    # Decode the results as JSON
    codec => "json"
    # Schedule task
    schedule => { cron => "* * * * * UTC" }
  }

}

output {
  #debugging output

  stdout {
    codec => rubydebug
  }

  # elasticsearch output

  elasticsearch{
   hosts => "localhost"
   index => "helloworld"
  }

}

The problem is that this logstash creates multiple documents with the same key and I want logstash to replace existing document in the index. Anyway i can do that? Thanks in advance.

Upvotes: 0

Views: 200

Answers (1)

Manikandan
Manikandan

Reputation: 3165

You should define what is your id for the document, so that you can update existing based on the key.

elasticsearch {
  document_id => "%{key}"
  hosts => "localhost"
  index => "helloworld"
}

See the documentation here for more information.

Upvotes: 1

Related Questions