CODEWITHSUNDEEP
pythonapielasticsearch
harper
harper

Reputation: 83

Elastic search to sort with timestamp using python

I have created a python api which will accept the *.json file.

for ex: abc.json

{
    "service_name": "httpd",
    "service_status": "DOWN",
    "host_name": "host1",
    "time": "1616600149.014236"
}

and, push the data to the ES using below python api.

@app.route('/add', methods=['POST']) def insert_data():
    #directory = '/home/user'
    dir = os.getcwd()
    os.chdir(dir)
    i = 1
    #This function will read all the json payload in the given dir and upload to ES.
    for filename in os.listdir(dir):
        if filename.endswith(".json"):
            f = open(filename)
            status_content = f.read()
            # Send the data into es
            result=(es.index(index='svc_index', ignore=400, doc_type='doc',
            id=i, body=json.loads(status_content)))
            i = i + 1
            print("result")

    return jsonify(result)

Output:

{
        "_id": "4", 
        "_index": "svc_index", 
        "_score": 1.0, 
        "_source": {
          "host_name": "host1", 
          "service_name": "httpd", 
          "service_status": "DOWN", 
          "time": "1616600143.5427265"
        }, 
        "_type": "doc"
      },

Since timestamp is being stored as string, sorting is not working. I wanted to bring the latest result on top. Could anyone please help into this.

Thanks!!

Upvotes: 2

Views: 2179

Answers (2)

harper
harper

Reputation: 83

I have found the solution.

timestamp is getting sorted in desc order with the following query in python.

result = es.search(index="svc_index", doc_type="doc", body={"query": {"match": {"service_name": query}},"sort":{"time": {'order': 'desc', 'mode':'max'}}}, size=1)

Upvotes: 2

illogicalhawk
illogicalhawk

Reputation: 11

If you want Elastic to know that the Time property is a timestamp and treat it as such, you can specify it as a Date field in the index mapping:

https://www.elastic.co/guide/en/elasticsearch/reference/current/date.html

That should enable Elastic to sort it correctly.

Upvotes: 1

Related Questions