CODEWITHSUNDEEP
elasticsearchlogstashkibana
Sebastian Sommerfeld
Sebastian Sommerfeld

Reputation: 407

ELK Type Conversion - Not a number but a string

I'm trying to set up an elk dashboard to see some numbers like total bytes, avg load time, etc. I'm forcing some conversions in logstash to make sure these fields aren't strings

convert => [ "bytes", "integer" ]
convert => [ "seconds", "float" ]
convert => [ "milliseconds", "integer" ]

Those Logstash conversions are working. See this excerpt from my logstash.log. Statuscode is a string, bytes, ... are numbers

"http_statuscode" => "200",
"bytes" => 2731,
"seconds" => 0.0,
"milliseconds" => 9059,

But when I try to build my dashboard with avg, min, max and total bytes for instance elasticsearch logs this:

Facet [stats]: field [bytes] isn't a number field, but a string

Am I missing some kind of conversion or something? Anybody already expierenced this behavior?

Thanks gus yand regards. Sebastian

Upvotes: 1

Views: 4073

Answers (2)

Alain Collins
Alain Collins

Reputation: 16362

One possible issue is that the mapping for fields in an index is set when the first document is inserted in the index. Changing the mapping will not update any old documents in the index, nor affect any new documents that are inserted into that index.

If you're in development, the easiest thing is to drop the index (thus deleting your earlier data). Any new documents would then use your new mapping.

If you can't toss the old data, you can wait for tomorrow, when you'll get a new index.

If necessary, you can also rebuild the index, but I've always felt it to be a pain.

One other possibility is that you have the same field name with different mappings in different types in the same index. [ repeat that a few times and it will make sense ]. Field [foo] must have the same mapping definition in each type of the same index.

Upvotes: 1

Quentin
Quentin

Reputation: 455

I recently solved this problem (I mean use bytes or request time as numbers in Kibana, I use v4 beta 3 and you ?). The three following points might help you :

  1. How do you parse your log ? Using Grok filter ? If yes, you can try matching your logs with the following patterns %{INT:bytes:int} instead of using the convert filter.
  2. Did you "reload field list" (yellow button) in Kibana 4 (settings->indices) after you've done your changes ?
  3. If you have old indexes in your ES cluster, did you correctly remove these indexes ? If not, you might have some conflicts between old types and new ones.

Hope it will help.

Upvotes: 0

Related Questions