Reputation: 15
I have programmed a function F which round the input following some specific rules. For example, if my input is F(0.1355)
it gives as output 1.4b-1
. The problem is that, when I do F(0.1355)^2
, it gives me 1.96382999420166b-2
, when it should give me 0.14^2, i.e, 1.96b-2
. Why is this happening? and, how can I fix it?
This is the code of my function:
F(x):=(block ([log10,aaa,fpprec],log10(x):=entier(log(x)/log(10)),
vnorm(x):= if x/(10^(log10(x)))<2 then 1/(10^(log10(x)-1)) else 1/10^(log10(x)),
aaa(x):= x*vnorm(x),
fpprec:if aaa(x)<20 then 2 else 1,bfloat((round(aaa(x)))*1/vnorm(x)))); `
More info: I can't declare fpprec as global variable. I know that would solve my problem. Best regards,
Lievet.
Upvotes: 0
Views: 283
Reputation: 17576
Well, the problem is that F(x)^2
is computed with the value of fpprec
outside of the block
, which is 16. First F(x)
is computed with fpprec
= 2; call that result foo
. Then foo^2
is computed with fpprec
= 16.
I don't know a way to limit the precision for bigfloat computations to the precision of the operands. It may be possible with simplification rules (tellsimp
or tellsimpafter
) but, if so, it is probably not easy.
Two ideas to make some progress here. (1) Consider using rational numbers instead of bigfloats. I.e. return 7/5 instead of 1.4b-1. (2) If you are just trying to limit the digits that are printed, use fpprintprec
.
Maybe if you say more about what is your larger goal here, someone will have some advice.
Upvotes: 1