CODEWITHSUNDEEP
loggingrequestelasticsearch
czerasz
czerasz

Reputation: 14280

Elasticsearch log explanation

I have enabled the slowlog for my elasticsearch server to see the traffic I get on it.
Here is some example output:

[2013-07-23 06:10:45,213][WARN ][index.search.slowlog.query] [Afari, Jamal] [index_3559_schema1][4] took[107.5micros], took_millis[0], types[some_type], stats[], search_type[QUERY_THEN_FETCH], total_shards[195], source[{"query":{"query_string":{"query":"*"}},"from":0,"size":"24","sort":{"updated_at":"desc"}}], extra_source[],
[2013-07-23 06:10:45,214][WARN ][index.search.slowlog.query] [Afari, Jamal] [index_3559_schema1][3] took[155.6micros], took_millis[0], types[some_type], stats[], search_type[QUERY_THEN_FETCH], total_shards[195], source[{"query":{"query_string":{"query":"*"}},"from":0,"size":"24","sort":{"updated_at":"desc"}}], extra_source[],
[2013-07-23 06:10:45,214][WARN ][index.search.slowlog.query] [Afari, Jamal] [index_3559_schema1][2] took[107.7micros], took_millis[0], types[some_type], stats[], search_type[QUERY_THEN_FETCH], total_shards[195], source[{"query":{"query_string":{"query":"*"}},"from":0,"size":"24","sort":{"updated_at":"desc"}}], extra_source[],
...
[2013-07-22 15:10:45,260][WARN ][index.search.slowlog.fetch] [Afari, Jamal] [index_42044_schema1][3] took[85.3micros], took_millis[0], types[some_type], stats[], search_type[QUERY_THEN_FETCH], total_shards[195], source[{"query":{"query_string":{"query":"*"}},"from":0,"size":"24","sort":{"updated_at":"desc"}}], extra_source[],

Does each of those lines mean (index_3559_schema1, index_3559_schema1, index_3559_schema1) I had an separate HTTP request?

What is the difference between index.search.slowlog.query and index.search.slowlog.fetch?

Upvotes: 1

Views: 1109

Answers (1)

javanna
javanna

Reputation: 60245

The slowlog doesn't refer to http requests but to queries, on the shard level. That means that if you query an index composed of 5 shards you'll have in the log 5 entries for the same query, one per shard, with a different shard id, which is the number that appears right after the index name. That way you can monitor the execution of each query for each shard.

The fetch line refers to the fetch phase. By default the query_then_fetch search type is used, which means that if you query an index composed of 5 primary shards (let's assume 0 replica to make it simpler), the query will be executed on each shard, which will return only the relevant information like the document ids and their scores. The node that you hit with your search request will then reduce those results in order to return only the top ones. After the reduce part that identifies which documents need to be returned, only those relevant documents are then fetched from their shard. That's when you see the fetch phase in your slowlog.

Upvotes: 3

Related Questions