CODEWITHSUNDEEP
elasticsearch
Andrey Deineko
Andrey Deineko

Reputation: 52367

ES 1.5 Delete By Query API not working

I am using an old version on ElasticSearch - 1.5.

Problem: I need to delete a lot of documents, like few hundred thousands up to few millions. I have all the info about the records, including it's _ids - so array of _ids is what I want to use.

Scale problem: I had this deletion in the loop before, but ES is inconsistent when performing a lot of subsequent operations in a high speed. Thus I decided to look for a bulk delete.

I am trying to make use of delete by query API.

Docs states:

curl -XDELETE 'http://localhost:9200/twitter/tweet/_query' -d '{
    "query" : {
        "term" : { "user" : "kimchy" }
    }
}
'

What I'm doing:

curl -XDELETE 'http://localhost:9200/my_index/logs/_query' -d '{
  "query" : {
    "terms" : { "_id" : ["AVTD6fhLAn35BG25xbZz", "AVTD6fhLAn35BG25xbaC"] }
  }
}
'

The response is:

{
  "found":false,
  "_index":"my_index",
  "_type":"logs",
  "_id":"_query",
  "_version":1,
  "_shards":{"total":2, "successful":1, "failed":0}
}

And it does not remove any of documents. How do I make it work and actually delete these records?

Upvotes: 0

Views: 182

Answers (1)

Archit Saxena
Archit Saxena

Reputation: 1557

Not sure about the delete_by_query API in elasticsearch 1.5. Seems to me that elasticsearch is unable to understand your query as it is looking for "_id": "_query" (as evident from the response you posted).

What you can do is, use the Bulk API as documented here: https://www.elastic.co/guide/en/elasticsearch/reference/1.5/docs-bulk.html

As in the example in the doc page, you can do:

curl -s -XPOST localhost:9200/_bulk --data-binary @requests; echo
{ "delete" : { "_index" : "test", "_type" : "type1", "_id" : "2" } }
{ "delete" : { "_index" : "test", "_type" : "type1", "_id" : "3" } }
...

You need to make a file by any name ("requests" here) and add individual delete requests, each separated by a newline character.

Upvotes: 1

Related Questions