CODEWITHSUNDEEP
c#.netxmlvb.netxpath
Mikkeee
Mikkeee

Reputation: 161

Dotnet XPathNavigator XPath Evaluate Incorrect Result

I'm attempting to evaluate an xpath expression but the results aren't correct. Take a look at the xml and expression below. The math (to me) seems that the calculation result should be:

500000 * (1 + -.80)

500000 * (.2)

I believe the result should be 100000 (and Excel thinks the same thing when I enter the formula). When I evaluate the xpath expression I get (Double: 99999.999999999971). I know some of you will tell me to change the (1 + -.8) to (.2) but these calculations are supplied to me and I can't change them. I dug a little deeper and evaluating (1 - .8) gives me 0.19999999999999996. What's going on here?

<editNode>    
  <cc_RCONB530 offset="0">500000</cc_RCONB530>  
</editNode>

editNode/cc_RCONB530[@offset='0'] *  (1 + -.80)

Upvotes: 1

Views: 170

Answers (1)

Martin Honnen
Martin Honnen

Reputation: 167561

The XPath 1.0 specification defines the XSLT/XPath 1.0 number type that way, ECMAScript/Javascript has the same type and the same problems, it is simply that for some values there is no finite and precise binary representation, the same as 1/3 has no finite and precise decimal representation. Move to XSLT 2.0 with xs:decimal if you can or if you are stuck with XSLT 1.0 then make sure you call format-number when outputting numbers, to ensure they are rounded to the precision you want.

Upvotes: 1

Related Questions