CODEWITHSUNDEEP
c#elasticsearchcurlnestelasticsearch.net
Changemyminds
Changemyminds

Reputation: 1377

How to use elasticsearch.net bulk API to import json file by btye array to elasticsearch?

I have some json file need to import to elasticsearch.

I use the curl api. The below is sample and it work fine for me.

curl -XPOST http://localhost:9200/index_local/_doc/_bulk -H "Content-Type: application/json" --data-binary @sample.json


I use the HttpWebRequest to simulate and it work fine for me too. 

public void Post(string fileName)
{
    try
    {
        // get the byte array 
        var data = File.ReadAllBytes(fileName);

        // create HttpRequest
        var httpRequest = (HttpWebRequest)WebRequest.Create(@"http://localhost:9200/index_local/_doc/_bulk");
        httpRequest.Method = "POST";
        httpRequest.ContentType = "application/json";
        httpRequest.ContentLength = data.Length;

        // set the file byte array to the stream
        using (var requestStream = httpRequest.GetRequestStream())
        {
            requestStream.Write(data, 0, data.Length);
        }

        // get the response
        using (var response = httpRequest.GetResponse() as HttpWebResponse)
        {
            using (var responseStream = new StreamReader(response.GetResponseStream()))
            {
                // read the result
                Console.WriteLine(responseStream.ReadToEnd());
            }
        }
    }
    catch (Exception e)
    {
        Console.WriteLine(e.Message);
    }
}

But I can't find the bulk api with import json file with elasticsearch.net.

Is there have some function equal to HttpWebRequest that can post the json file to elasticsearch ?

Is elasticsearch.net library ElasticLowLevelClient or ElasticClient support to import json file with btye array ?

Upvotes: 0

Views: 1239

Answers (1)

Russ Cam
Russ Cam

Reputation: 125518

Assuming sample.json is a JSON file with a valid structure for the bulk API, you can send the request with

var client = new ElasticClient();
var bytes = File.ReadAllBytes("sample.json");

var bulkResponse = client.LowLevel.Bulk<BulkResponse>(
    bytes,
    new BulkRequestParameters
    {
        RequestConfiguration = new RequestConfiguration
        {
            RequestTimeout = TimeSpan.FromMinutes(3)
        }
    });

if (!bulkResponse.IsValid)
{
    // handle failure
}

This sets a specific request timeout for this request, which you might set to something larger than normal if the bulk is larger than a normal request, which it typically is. If sample.json is larger than around 5MB, you might consider reading the file in line pairs (bulk action and document) in batches, and sending as multiple bulk requests.

Upvotes: 1

Related Questions