CODEWITHSUNDEEP
pythonelasticsearchelasticsearch-py
Hal T
Hal T

Reputation: 547

Python elasticsearch return only entries that have a certain field

I'm trying to query an Elasticsearch, and only get results that have a certain field.

How do I query for documents that have field fields.EventData.PGID and ignore ones that don't?

datadict = es.search(index=idx1, 
                     q='run_id:"Run001" AND "fields.EventData.PGID exists"', 
                     sort='fields.System.TimeCreated.SystemTime',
                     size=1000)

The way events are logged in the ES is inconsistent and such I need to find only ones where a PGID was logged. I tried doing a try block in the Python code trying to access the field from the returned values and ignoring it if I get a KeyError, but due to the limit on how many items you can receive as a query result, in some cases I have all my results lacking a PGID so I just end up wasting a query and am unable to access actual results, so I would like this filtering to happen at the query level.

Upvotes: 0

Views: 531

Answers (1)

ML_TN
ML_TN

Reputation: 727

You can try with this filtered query

{
  "size": 1000,
  "query": {
    "filtered": {
      "filter": {
        "bool": {
          "must": {
            "exists": {
              "field": "fields.EventData.PGID"
            },
            "term": {
              "run_id": "Run001"
            }
          }
        }
      }
    }
  }
}

I think you can also add the sorting to the query

Upvotes: 1

Related Questions