CODEWITHSUNDEEP
elasticsearchlogstashnest
Cadmium
Cadmium

Reputation: 566

How to construct filter for Elastic Search on Log Stash with NEST?

I have a logstash/elasticsearch/kibana system set up and structured json type logs are getting into elastic search (not from a C# application) and visible in kibana.

I am using NEST because I would like to slice the data from logstash.

The following code is producing "hits" results that I can inspect in the debugger. 

ElasticClient client = new ElasticClient(settings);


var searchResults = client.Search( s  => s
                                      .From(0)
                                      .Size(100)
                                      .AllIndices()

                                      .SortDescending("@timestamp")
                                  );

However, if I try to expand the search to include something I believe to be present in the log (visible in Kibana), I get now results. 

 var searchResults = client.Search( s  => s
                                      .From(0)
                                      .Size(100)
                                      .AllIndices()
                                      .Query(q => q
                                         .Term("MySpecialFieldName", "ValueThatAppears")
                                       )
                                       .SortDescending("@timestamp")
                                   );

I would also like to take advantage of the type safety and other mechanisms shown in the samples. I am not sure if that is expected to be related.

( I am working on figuring that out separately: Adding a class like "client.Search( to the search seems to prevent the results as well, I am assuming that something about the class is not aligned with the data and is therefore unable to deserialize or is otherwise filtering...)

Upvotes: 3

Views: 1084

Answers (1)

Cadmium
Cadmium

Reputation: 566

Found the correct settings to make this work:

The logstash system puts this into a type (log or logs I think) when it gets indexed by elastic search. Also the term needs to be suffixed with ".raw". So the working example ends up being: 

 var searchResults = client.Search<MyClass>( s  => s
                                      .From(0)
                                      .Size(100)
                                      .AllIndices()
                                      .AllTypes()
                                      .Query(q => q
                                         .Term("MySpecialFieldName.raw", "ValueThatAppears")
                                       )
                                       .SortDescending("@timestamp")
                                   );

Upvotes: 1

Related Questions