CODEWITHSUNDEEP
luanumbersnumber-formattingsignificant-digits
LuiGiovanni
LuiGiovanni

Reputation: 35

Lua significant figures

I'm trying to make a function that rounds a number up to a certain number of significant figures given by the user, for example if the user gives me the number

234.235534 with 5 significant numbers, the function should return 234.24

Upvotes: 1

Views: 1716

Answers (2)

Tanner Armstrong
Tanner Armstrong

Reputation: 11

I'm not much of a programmer, but I came up with this for my own use after I was disappointed by other rounding functions people recommended in lua. This should do what you asked.

function sigFig(num,figures)
    local x=figures - math.ceil(math.log10(math.abs(num)))
    return(math.floor(num*10^x+0.5)/10^x)
end

now in terms of significant digits, it won't add additional zeros to a number to signify precision. For example:

sigFig(234.235534,5) will yield 234.24

sigFig(234.0000001,6) will yield 234.0, not 234.000

Upvotes: 1

hjpotter92
hjpotter92

Reputation: 80649

I think you're looking for the [fs]?printf's %g modifier.

converts floating-point number to decimal or decimal exponent notation depending on the value and the precision.

where, the precision is defined by:

. followed by integer number or *, or neither that specifies precision of the conversion. In the case when * is used, the precision is specified by an additional argument of type int. If the value of this argument is negative, it is ignored. If neither a number nor * is used, the precision is taken as zero.

So, you want:

> return ("%.5g"):format(234.235534)
234.24
> return  ("%.6g"):format(x)
234.236

Upvotes: 2

Related Questions