CODEWITHSUNDEEP
pythoncurlelasticsearchlogstashpyelasticsearch
Roshni Kasliwal
Roshni Kasliwal

Reputation: 229

How to delete documents from multiple indices using particular field name in elasticsearch python?

This is sample document. 

{
"_index": "mqtt-index-2018.01.23",
"_type": "iot_data",
"_id": "AWEjA7LNRU4cTUO-Lyj4",
"_score": null,
"_source": {
"message": "{\"datastream_name\": \"roshni\", \"value\": 12, 
\"context\": {\"latitude\": 0, \"elevation\": 0, \"longitude\": 0}, 
\"device_id\": 31}",
"@version": "1",
"@timestamp": "2018-01-23T12:34:59.687Z",
"host": "iot-elk",
"topic": "telemetry/f2a55827ef554475a41c3c96369957f0/roshni",
"datastream_name": "roshni",
"value": 12,
"context": {
  "latitude": 0,
  "elevation": 0,
  "longitude": 0
},
"device_id": 31,
"tstamp": "2018-01-23T12:34:59.687Z"
},
"fields": {
"tstamp": [
  1516710899687
],
"@timestamp": [
  1516710899687
]
},
"sort": [
 1516710899687
]
}

I want to delete document using device_id field. How to delete it using API call or using python client? I have tried it using Document _id and particular index but i want to delete it by using device_id field or other field.

Upvotes: 3

Views: 2673

Answers (4)

Akash Talole
Akash Talole

Reputation: 91

You can also delete documents with matching multiple fields.

curl -XDELETE 'http://localhost:9200/mqtt-index-*/logs/_query' -d '{
"query" : {
    "bool": {
        "must":[
    {"match" : {"device_id":31}}, 
    {"match":  {"datastream_name": "test"}}
    ]
}
}' -i

Upvotes: 0

Akash Talole
Akash Talole

Reputation: 91

Deleting document from multiple indices in elasticsearch using following DELETE API call.

curl -XDELETE 'http://localhost:9200/mqtt-index-*/logs/_query' -d '{
"query" : {
    "match" : {"device_id": 31}
}
}' -i

Upvotes: 1

MrName
MrName

Reputation: 2529

In terms of using the raw API, I believe this is what you are looking for.

Depending on which python library you are using, it is actually easier in python. I use elasticesearch-dsl-py, where you build up query objects. You can call delete on these query objects.

In regards to spanning multiple indexes, ElasticSearch does support this, either using a wildcard *, or separating the indexes with commas.

Upvotes: 0

OBu
OBu

Reputation: 5177

Short answer without sample code:

  • convert the json to a dict (see Converting JSON String to Dictionary Not List)
  • if you have many of those data structures, build a list of them if they match your search criteria (e.g. if data["parsed_message"]["device_id"] not in list_of_forbidden_ids):)

Upvotes: 0

Related Questions