CODEWITHSUNDEEP
c#elasticsearchnest
Oliver Heil
Oliver Heil

Reputation: 64

Index dynamic Objects with ElasticSearch Nest - StackOverflow Exception

I want to index a collection of dynamic objects using the ElasticSearch Nest Client. The objects are first materialized by NewtonsoftJson.NET from a JSON file into dynamic objects and then manipulated by the program. All objects get a property "Id". This should serve as "_id" field for ElasticSearch. The "_id" field must be the same for identical data records in order to be able to update data later. Because attributes "IdProperty" cannot be added to a dynamic object and a mapping can also not be used, I was forced to the following solution. I would like to keep the dynamic objects because I manipulate only a few properties and the other properties are of no interest to me.

var values = new List<dynamic>();
dynamic obj = new System.Dynamic.ExpandoObject();
obj.Id = "ABC";
obj.SomeValue0 = "12";
obj.SomeValue1 = 99;

values.Add(obj);

var descriptor = new BulkDescriptor();

// Now i want to Index this List
foreach (var doc in values) {
    // Here the StackOverflowException will be thrown
    descriptor.Index<object>(i => i
        .Index("abc")
        .Id(doc.Id)
        .Document(doc));
}

client.Bulk(descriptor);

(Index a dynamic object using NEST - This was my inspiration)

This example raises a StackOverflow exception during indexing. It does not matter whether one or more objects are in the list.

Interestingly, the following method works without problems. The only thing that doesn't work is the ElasticSearch "_id" field was generated automatically and therefore does not correspond to the "Id" field.

client.IndexMany(value, index);

What am I doing wrong with the first possibility and is there a possibility to set a "_id" on a dynamic object?

Upvotes: 0

Views: 2615

Answers (1)

Russ Cam
Russ Cam

Reputation: 125498

You need to

  1. cast the Id property to string (or Nest.Id if it could be another type with an implicit conversion to Nest.Id, for example, Guid
  2. cast the object to object

Here's an example

var client = new ElasticClient();

var values = new List<dynamic>();
dynamic obj = new System.Dynamic.ExpandoObject();
obj.Id = "ABC";
obj.SomeValue0 = "12";
obj.SomeValue1 = 99;

values.Add(obj);

var descriptor = new BulkDescriptor();

// Now i want to Index this List
foreach (var doc in values)
{
    descriptor.Index<object>(i => i
        .Index("abc")
        .Id((Id)doc.Id)
        .Document((object)doc));
}

client.Bulk(descriptor);

which will send a request like the following

POST http://localhost:9200/_bulk 
{"index":{"_index":"abc","_type":"object","_id":"ABC"}}
{"Id":"ABC","SomeValue0":"12","SomeValue1":99}

Dynamic types don't play nicely with generic types and member access expressions, which looks to be related to runtime type resolution. In the example above, I would recommend using anonymous types and a List<object>

Upvotes: 4

Related Questions