Export documents to either JSON or CSV

Question

I know this might be a basic question, but I believe everybody has to learn from some point.

I am trying hard to learn Elasticsearch and as I was going through the documentation at this page

https://www.elastic.co/guide/en/elasticsearch/reference/current/_exploring_your_data.html

I learned how to load a JSON data and form an index. But later an question arose in my head, how would I be able to export the whole index that I created (here the bank) to CSV or a JSON. Since I couldn't find a solution in Kibana, I searched Github and came across this python script 'es2csv'.

https://pypi.python.org/pypi/es2csv/2.4.11

I installed it, but I can't make an output to a format that it should appear in a CSV file. Please, could you help me understand what should I do and clarify the steps i should take because it is vital for me to learn.

Many thanks!

Answer 1

I recently posted this script which sounds like it has similar goals to es2csv.py above, but just as a single bash script using jq and json2csv.js.

Here it is as a gist:

https://gist.github.com/nzjess/23cbab0fbb3b1b3a63c30e468119612b

This is how I am scripting our exports of CSV from our ELK stack. Docs are inline.

Answer 2

This appears to be the most high level option - it would download all indices from elasticsearch, not just one (from the docs).

es2csv -q 'host: localhost' -o database.csv

or to specify an index, use -i and your index name:

es2csv -i indexname -q 'host: localhost' -o database.csv

A better way might be to use kibana to build visualisations. You can then export the data as Raw or Formatted using the buttons below the visualisation in kibana.

This might work better especially if you build a data table, as the data will be 'flattened' - rather than nested in a json structure which might not be much use when you convert directly to csv.