CODEWITHSUNDEEP
xmlxpathxquery
user3192053
user3192053

Reputation: 107

XPath in XQuery

Using the following output XSD type:

<xs:simpleType name="MoneyType">
  <xs:restriction base="xs:decimal">
       <xs:fractionDigits value="2"/>
   </xs:restriction>
</xs:simpleType>

I use a fn:sum command and get the following output:

1.0001865499999999E6

The value should be 

1000186.55

What would cause this lack of precision

Thanks

Upvotes: 1

Views: 85

Answers (1)

Michael Kay
Michael Kay

Reputation: 163595

Two issues here:

(a) floating point arithmetic (or conversion of decimal values to floating point) causes approximation, because most decimal fractions can't be represented exactly in binary

(b) by default, numbers outside the range one-millionth to one million are output in "scientific notation", that is 2e6 rather than 2000000.

If you explain more carefully what you are doing and what you want to achieve then we can tell you how to avoid these problems.

Upvotes: 3

Related Questions