CODEWITHSUNDEEP
djangoelasticsearch
Zed
Zed

Reputation: 379

Bulk index django model to elastic search

I am trying to bulk index my django model to elastic search 6, my plan is to run this as a cron once a day to update the index. import requests

data = serialize('json', CapitalSheet.objects.all())

data += "\n"

r = requests.post("http://127.0.0.1:9200/capitalsheet/_bulk", json = data)

print(r.content)

I am getting this error: b'{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"The bulk request must be terminated by a newline [\\n]"}],"type":"illegal_argument_exception","reason":"The bulk request must be terminated by a newline [\\n]"},"status":400}'

If you can suggest something better, I would be glad.

Upvotes: 0

Views: 1103

Answers (1)

Ryan Widmaier
Ryan Widmaier

Reputation: 8523

I would recommend looking at the python library provided by elasticsearch. It will make it easier to do the bulk inserts. Here is a link to the docs:

https://elasticsearch-py.readthedocs.io/en/master/helpers.html#bulk-helpers

If you want to do it manually though, the ES bulk API actually requires two lines for every record you want to insert. The first line details what index and the type of operation, and the second line is the record to insert. For example your request body would look something like this:

{ "index" : { "_index" : "test", "_type" : "type1" } }
{ "field1" : "value1" }
{ "index" : { "_index" : "test", "_type" : "type1" } }
{ "field1" : "value2" }

The ES documentation explains this well here: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html

Upvotes: 0

Related Questions