CODEWITHSUNDEEP
rubyfloating-point
ylluminate
ylluminate

Reputation: 12369

How to force a Float to display with full precision w/o scientific notation and not as a string?

How do you force a float to display with all significant places / full precision without scientific notation in Ruby?

Presently I convert a BigDecimal to Float, BigDecimal(0.000000001453).to_f, however this yields a resultant float of 1.453e-09. If I do something like "%14.12f" % BigDecimal("0.000000001453").to_f I get a string. In this case, however, a string as output is unacceptable as I need it as an actual numeric float without scientific notation.

--- Edit---

Alright, let me give some context here, which will probably require a change of my original question.

I'm attempting to create a graph with Highstock & lazy_high_chart. Earlier today I was able to draw graphs just fine when the floats were emitting to the resultant js as full precision floats vs showing up in scientific notation. Hence I felt that the problem resides in this issue.

But after the few inputs I'm getting here, perhaps I need some further review of the source and my assumption is misplaced. I'll let you decide:

@h = LazyHighCharts::HighChart.new('graph') do |f|
  hours_of_readings = 1
  reading_intervals = 1 #hour
  readings_per_hour = 60

  readings = ModelName.order("date DESC").select('data_readings.data2, data_readings.data1, data_readings.date').limit(hours_of_readings * readings_per_hour).all

  data1_and_date_series = Array.new
  data2_and_date_series = Array.new
  dates = Array.new

  # I have been thinking that the problem lies here in the "row.data1.to_f" and
  #   "row.data2.to_f" and thus this is the root of my initial question in terms 
  #   of it emitting scientific notation to the js output in the format of:
  #   [[1.0e-09], [1.04e-09],[9.4e-09], ... [3.68e-09]]
  data1_and_date_series = readings.map{|row| [(row.date.to_i * 1000), (row.data1.to_f if BigDecimal(row.data1) != BigDecimal("-1000.0"))] }
  data2_and_date_series = readings.map{|row| [(row.date.to_i * 1000), (row.data2.to_f if BigDecimal(row.data2) != BigDecimal("-1000.0"))] }


  f.series(
      :name => 'Data1',
      :data => data1_and_date_series,
      :pointStart => Time.now.to_i * 1000,
      :pointEnd => hours_of_readings.hours.ago.to_i * 1000,
      :pointInterval => reading_intervals.hour * 1000,
      :color => 'blue'
  )
  f.series(
      :name => 'Data2)',
      :data => data2_and_date_series,
      :pointStart => Time.now.to_i * 1000,
      :pointEnd => hours_of_readings.hours.ago.to_i * 1000,
      :pointInterval => reading_intervals.hour.to_i * 1000,
      :color => 'red'
  )

  f.chart({:defaultSeriesType=>"spline" })
  f.yAxis [
        {:title => { :text => "Label 1", :margin => 10} },
        {:title => { :text => "Label 2 (groups)"}, :opposite => true},
        {:max => 0},
        {:min => -0.000000001}
  ]
  f.options[:xAxis] = {
      :title => { :text => "Time"},
      :type => "datetime"
  }
  f.title(:text => "Title")
  f.legend(:align => 'right', :verticalAlign => 'top', :y => 75, :x => -50, :layout => 'vertical') # override the default values
end

Upvotes: 7

Views: 6645

Answers (2)

undur_gongor
undur_gongor

Reputation: 15954

The string representation and the actual value of a float are two different things.

What you see on screen/print-out is always a string representation, be it in scientific notation or "normal" notation. A float is converted to its string representation by to_s, puts, "%.10f" % and others.

The float value itself is independent of that. So your last sentence does not make much sense. The output is always a string.

To enforce a certain float format in Rails' to_json you can overwrite Float#encode_json, e.g.

class ::Float
  def encode_json(opts = nil)
    "%.10f" % self
  end
end

Put this before your code above. Note that -- depending on your actual values -- you might need more sophisticated logic to produce reasonable strings.

Upvotes: 3

jaypal singh
jaypal singh

Reputation: 77105

Will this work for you - 

>> 0.000000001453
=> 1.453e-09 # what you are getting right now
>> puts "%1.12f" % 0.000000001453
0.000000001453 # what you need
=> nil

Upvotes: 4

Related Questions