CODEWITHSUNDEEP
elasticsearchlogstash
Hyder B.
Hyder B.

Reputation: 12186

Update records in ElasticSearch

I had like to update the logdate column for ALL records in a specific index. From what I have read so far, it seems that it is not possible? I am correct?

Here's a sample of a document:

{
            "_index": "logstash-01-2015",
            "_type": "ufdb",
            "_id": "AU__EvrALg15uxY1Wxf9",
            "_score": 1,
            "_source": {
               "message": "2015-08-14 06:50:05 [31946] PASS  level2      10.249.10.70    level2     ads       http://ad.360yield.com/unpixel.... GET",
               "@version": "1",
               "@timestamp": "2015-09-24T11:17:57.389Z",
               "type": "ufdb",
               "file": "/usr/local/ufdbguard/logs/ufdbguardd.log",
               "host": "PROXY-DEV",
               "offset": "3983281700",
               "logdate": "2015-08-14T04:50:05.000Z",
               "status": "PASS",
               "group": "level2",
               "clientip": "10.249.10.70",
               "category": "ads",
               "url": "http://ad.360yield.com/unpixel....",
               "method": "GET",
               "tags": [
                  "_grokparsefailure"
               ]
            }
         }

Upvotes: 2

Views: 1257

Answers (2)

Mario Trucco
Mario Trucco

Reputation: 2011

You are correct, that is not possible.

There's been an open issue asking Update by Query for long time, and I'm not sure it's going to be implemented anytime soon since it is very problematic for the underlying lucene engine. It requires deleting all documents and reindexing them.

An Update by Query Plugin is available on github, but it's experimental and I never tried it.

UPDATE 2018-05-02

The original answer is quite old. Update By Query is now supported.

Upvotes: 2

Sloan Ahrens
Sloan Ahrens

Reputation: 8718

You can use the partial update API.

To test it, I created a trivial index:

PUT /test_index

Then created a document:

PUT /test_index/doc/1
{
   "message": "2015-08-14 06:50:05 [31946] PASS  level2      10.249.10.70    level2     ads       http://ad.360yield.com/unpixel.... GET",
   "@version": "1",
   "@timestamp": "2015-09-24T11:17:57.389Z",
   "type": "ufdb",
   "file": "/usr/local/ufdbguard/logs/ufdbguardd.log",
   "host": "PROXY-DEV",
   "offset": "3983281700",
   "logdate": "2015-08-14T04:50:05.000Z",
   "status": "PASS",
   "group": "level2",
   "clientip": "10.249.10.70",
   "category": "ads",
   "url": "http://ad.360yield.com/unpixel....",
   "method": "GET",
   "tags": [
      "_grokparsefailure"
   ]
}

Now I can do a partial update on the document with:

POST /test_index/doc/1/_update
{
    "doc": {
        "logdate": "2015-09-25T12:20:00.000Z"
    }
}

If I retrieve the document:

GET /test_index/doc/1

I will see that the logdate property has been updated:

{
   "_index": "test_index",
   "_type": "doc",
   "_id": "1",
   "_version": 2,
   "found": true,
   "_source": {
      "message": "2015-08-14 06:50:05 [31946] PASS  level2      10.249.10.70    level2     ads       http://ad.360yield.com/unpixel.... GET",
      "@version": "1",
      "@timestamp": "2015-09-24T11:17:57.389Z",
      "type": "ufdb",
      "file": "/usr/local/ufdbguard/logs/ufdbguardd.log",
      "host": "PROXY-DEV",
      "offset": "3983281700",
      "logdate": "2015-09-25T12:20:00.000Z",
      "status": "PASS",
      "group": "level2",
      "clientip": "10.249.10.70",
      "category": "ads",
      "url": "http://ad.360yield.com/unpixel....",
      "method": "GET",
      "tags": [
         "_grokparsefailure"
      ]
   }
}

Here is the code I used to test it:

http://sense.qbox.io/gist/236bf271df6d867f5f0c87eacab592e41d3095cf

Upvotes: 0

Related Questions