CODEWITHSUNDEEP
rubylogstashscientific-notation
Dominik Sandjaja
Dominik Sandjaja

Reputation: 6476

Avoid scientific notation in JSON output when converting field to float

Running the following configuration in Logstash 2.3.4

input {
    generator {
        codec => json
        lines => [
            '{"value": "0.0007674"}'
            ]
        count => 1
    }
}

filter {
  mutate {
    convert => ["value", "float"]
  }
}

output {
    stdout { codec => json_lines }
    stdout { codec => json }
}

leads to the following output:

{"value":7.674E-4,"@version":"1","@timestamp":"2016-11-02T15:33:54.741Z","host":"myHost","sequence":0}
{"value":7.674E-4,"@version":"1","@timestamp":"2016-11-02T15:33:54.741Z","host":"myHost","sequence":0}

How can I prevent the conversion to scientific notation in the json output?

I have found this Logstash bug with the corresponding fix and thought, that all outputs and codecs automatically profit from it, since it was put in the event class. But either I was wrong or I am missing something, so any hint is appreciated.

Upvotes: 0

Views: 1969

Answers (1)

SteveTurczyn
SteveTurczyn

Reputation: 36870

The Logstash bug fix fixes values that are greater than or equal to 1000000000000000.0 and that are less than 0.0001. Unfortunately "0.0007674" doesn't fit into either category. You should see that if you set a smaller number, e.g. (0.00007674) then it works as expected.

Note that the solution they're following creates a string representation of the float... verify in your output if the smaller value (0.00007674) is satisfactory or whether it appears as a string regardless.

If it appears as a string regardless, you may be better off simply not doing the mutation.

You can override the sprintf method in an initializer.

app/config/initializers/fix_sprintf.rb

class LogStash::Event

  alias_method :old_sprintf, :sprintf

  public
  def sprintf(format)
    unchanged_format = format
    if format.is_a?(Float) and format.to_s.include?('E')
      format = ("%.15f" % format).sub(/0*$/,"")
    else
      format = format.to_s
    end
    if format.index("%").nil?
      return format
    end 

    old_sprintf(unchanged_format)
  end
end

Upvotes: 1

Related Questions